American Institute of Mathematical Sciences

Advanced
February  2021, 15(1): 1-25. doi: 10.3934/ipi.2020048

Reproducible kernel Hilbert space based global and local image segmentation

Liam Burrows 1, , Weihong Guo 2, , Ke Chen 1, and  Francesco Torella 3,

1. 

Centre for Mathematical Imaging Techniques and Department of Mathematical Sciences, University of Liverpool, United Kingdom
2. 

Department of Mathematics, Applied Mathematics and Statistics, Case Western Reserve, University, Cleveland, OH 44106, USA
3. 

Liverpool Vascular & Endovascular Service, Royal Liverpool and Broadgreen University, Hospitals NHS Trust, Liverpool, L7 8XP, United Kingdom

Received  December 2019 Revised  May 2020 Published  August 2020

Image segmentation is the task of partitioning an image into individual objects, and has many important applications in a wide range of fields. The majority of segmentation methods rely on image intensity gradient to define edges between objects. However, intensity gradient fails to identify edges when the contrast between two objects is low. In this paper we aim to introduce methods to make such weak edges more prominent in order to improve segmentation results of objects of low contrast. This is done for two kinds of segmentation models: global and local. We use a combination of a reproducing kernel Hilbert space and approximated Heaviside functions to decompose an image and then show how this decomposition can be applied to a segmentation model. We show some results and robustness to noise, as well as demonstrating that we can combine the reconstruction and segmentation model together, allowing us to obtain both the decomposition and segmentation simultaneously.

Keywords: RKHS, Heaviside function, Image segmentation.
Mathematics Subject Classification: Primary: 58F15, 58F17; Secondary: 53C35.
Citation: Liam Burrows, Weihong Guo, Ke Chen, Francesco Torella. Reproducible kernel Hilbert space based global and local image segmentation. Inverse Problems & Imaging, 2021, 15 (1) : 1-25. doi: 10.3934/ipi.2020048
References:
[1]

L. Ambrosio and V. Tortorelli, On the approximation of free discontinuity problems, Bollettino Dell'Unione Matematica Italiana. B, 6 (1992), 105-123.   Google Scholar

[2]

L. Ambrosio and V. M. Tortorelli, Approximation of functional depending on jumps by elliptic functional via t-convergence, Communications on Pure and Applied Mathematics, 43 (1990), 999-1036.  doi: 10.1002/cpa.3160430805.  Google Scholar

[3]

B. Appleton and H. Talbot, Globally minimal surfaces by continuous maximal flows, IEEE Transactions on Pattern Analysis and Machine Intelligence, 28 (2006), 106-118.  doi: 10.1109/TPAMI.2006.12.  Google Scholar

[4]

N. Badshah and K. Chen, Image selective segmentation under geometrical constraints using an active contour approach, Communications in Computational Physics, 7 (2010), 759-778.  doi: 10.4208/cicp.2009.09.026.  Google Scholar

[5]

E. Bae, J. Yuan and X.-C. Tai, Global minimization for continuous multiphase partitioning problems using a dual approach, International Journal of Computer Vision, 92 (2011), 112-129.  doi: 10.1007/s11263-010-0406-y.  Google Scholar

[6]

Y. Boykov and V. Kolmogorov, An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision, Energy Minimization Methods in Computer Vision and Pattern Recognition, 2134 (2001), 359-374.  doi: 10.1007/3-540-44745-8_24.  Google Scholar

[7]

Y. Boykov, O. Veksler and R. Zabih, Fast approximate energy minimization via graph cuts, Proceedings of the Seventh IEEE International Conference on Computer Vision, 1 (1999), 377-384.  doi: 10.1109/ICCV.1999.791245.  Google Scholar

[8]

X. Bresson, S. Esedoḡlu, P. Vandergheynst, J.-P. Thiran and S. Osher, Fast global minimization of the active contour/snake model, Journal of Mathematical Imaging and Vision, 28 (2007), 151-167.  doi: 10.1007/s10851-007-0002-0.  Google Scholar

[9]

X. Cai, R. Chan and T. Zeng, A two-stage image segmentation method using a convex variant of the Mumford–Shah model and thresholding, SIAM Journal on Imaging Sciences, 6 (2013), 368-390.  doi: 10.1137/120867068.  Google Scholar

[10]

V. Caselles, R. Kimmel and G. Sapiro, Geodesic active contours, International Journal of Computer Vision, 22 (1997), 61-79.  doi: 10.1109/ICCV.1995.466871.  Google Scholar

[11]

A. Chambolle, An algorithm for total variation minimization and applications, Journal of Mathematical Imaging and Vision, 20 (2004), 89-97.   Google Scholar

[12]

T. F. Chan, S. Esedoglu and M. Nikolova, Algorithms for finding global minimizers of image segmentation and denoising models, SIAM Journal on Applied Mathematics, 66 (2006), 1632-1648.  doi: 10.1137/040615286.  Google Scholar

[13]

T. F. Chan and L. A. Vese, Active contours without edges, IEEE Transactions on Image Processing, 10 (2001), 266-277.  doi: 10.1109/83.902291.  Google Scholar

[14]

L.-J. Deng, W. Guo and T.-Z. Huang, Single-image super-resolution via an iterative reproducing kernel hilbert space method, IEEE Transactions on Circuits and Systems for Video Technology, 26 (2016), 2001-2014.  doi: 10.1109/TCSVT.2015.2475895.  Google Scholar

[15]

C. Gout, C. Le Guyader and L. Vese, Segmentation under geometrical conditions using geodesic active contours and interpolation using level set methods, Numerical Algorithms, 39 (2005), 155-173.  doi: 10.1007/s11075-004-3627-8.  Google Scholar

[16]

J.-B. Hiriart-Urruty and C. Lemar{é}chal, Convex Analysis and Minimization Algorithms I: Fundamentals, volume 305., Springer Science & Business Media, 2013. Google Scholar

[17]

P. Jaccard, The distribution of flora in the apline zone.1, New Phytologist, 11 (1912), 37-50.   Google Scholar

[18]

S. H. Kang, B. Shafei and G. Steidl, Supervised and transductive multi-class segmentation using p-Laplacians and RKHS methods, Journal of Visual Communication and Image Representation, 25 (2014), 1136-1148.  doi: 10.1016/j.jvcir.2014.03.010.  Google Scholar

[19]

M. Kass, A. Witkin and D. Terzopoulos, Snakes: Active contour models, International Journal of Computer Vision, 1 (1988), 321-331.  doi: 10.1007/BF00133570.  Google Scholar

[20]

C. Li, C.-Y. Kao, J. C. Gore and Z. Ding, Minimization of region-scalable fitting energy for image segmentation, IEEE Transactions on Image Processing, 17 (2008), 1940-1949.  doi: 10.1109/TIP.2008.2002304.  Google Scholar

[21]

C. Liu, M. Ng and T. Zeng, Weighted variational model for selective image segmentation with application to medical images, Pattern Recognition, 76 (2018), 367-379.  doi: 10.1016/j.patcog.2017.11.019.  Google Scholar

[22]

Z. Li and F. Malgouyres, Regularized non-local total variation and application in image restoration, Journal of Mathematical Imaging and Vision, 59 (2017), 296-317.  doi: 10.1007/s10851-017-0732-6.  Google Scholar

[23]

D. Mumford and J. Shah, Optimal approximations by piecewise smooth functions and associated variational problems, Communications on Pure and Applied Mathematics, 42 (1989), 577-685.  doi: 10.1002/cpa.3160420503.  Google Scholar

[24]

S. Osher and J. A. Sethian, Fronts propagating with curvature-dependent speed: Algorithms based on hamilton-jacobi formulations, Journal of Computational Physics, 79 (1988), 12-49.  doi: 10.1016/0021-9991(88)90002-2.  Google Scholar

[25]

R. B. Potts, Some generalized order-disorder transformations, Mathematical Proceedings of the Cambridge Philosophical Society, 48 (1952), 106-109.  doi: 10.1017/S0305004100027419.  Google Scholar

[26]

M. Roberts, K. Chen and K. L. Irion, A convex geodesic selective model for image segmentation, Journal of Mathematical Imaging and Vision, 61 (2019), 482-503.  doi: 10.1007/s10851-018-0857-2.  Google Scholar

[27]

M. Roberts and J. Spencer, Chan–vese reformulation for selective image segmentation, Journal of Mathematical Imaging and Vision, 61 (2019), 1173-1196.  doi: 10.1007/s10851-019-00893-0.  Google Scholar

[28]

J. Spencer and K. Chen, A convex and selective variational model for image segmentation, Communications in Mathematical Sciences, 13 (2015), 1453-1472.  doi: 10.4310/CMS.2015.v13.n6.a5.  Google Scholar

[29]

X.-F. Wang, D.-S. Huang and H. Xu, An efficient local chan–vese model for image segmentation, Pattern Recognition, 43 (2010), 603-618.  doi: 10.1016/j.patcog.2009.08.002.  Google Scholar

[30]

Y. Xu and W. Yin, A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion, SIAM Journal on Imaging Sciences, 6 (2013), 1758-1789.  doi: 10.1137/120887795.  Google Scholar

[31]

J. Yuan, E. Bae, X.-C. Tai and Y. Boykov, A continuous max-flow approach to potts model, In European Conference on Computer Vision, Springer, 2010,379–392. Google Scholar

[32]

J. Yuan, E. Bae, X.-C. Tai and Y. Boykov, A spatially continuous max-flow and min-cut framework for binary labeling problems, Numerische Mathematik, 126 (2014), 559-587.  doi: 10.1007/s00211-013-0569-x.  Google Scholar

[33]

H. Zhang, Y. Chen and J. Shi, Nonparametric image segmentation using Renyi's statistical dependence measure, Journal of Mathematical Imaging and Vision, 44 (2012), 330-340.  doi: 10.1007/s10851-012-0329-z.  Google Scholar

[34]

H. Zhao, A fast sweeping method for Eikonal equations, Mathematics of Computation, 74 (2005), 603-627.  doi: 10.1090/S0025-5718-04-01678-3.  Google Scholar

show all references

References:
[1]

L. Ambrosio and V. Tortorelli, On the approximation of free discontinuity problems, Bollettino Dell'Unione Matematica Italiana. B, 6 (1992), 105-123.   Google Scholar

[2]

L. Ambrosio and V. M. Tortorelli, Approximation of functional depending on jumps by elliptic functional via t-convergence, Communications on Pure and Applied Mathematics, 43 (1990), 999-1036.  doi: 10.1002/cpa.3160430805.  Google Scholar

[3]

B. Appleton and H. Talbot, Globally minimal surfaces by continuous maximal flows, IEEE Transactions on Pattern Analysis and Machine Intelligence, 28 (2006), 106-118.  doi: 10.1109/TPAMI.2006.12.  Google Scholar

[4]

N. Badshah and K. Chen, Image selective segmentation under geometrical constraints using an active contour approach, Communications in Computational Physics, 7 (2010), 759-778.  doi: 10.4208/cicp.2009.09.026.  Google Scholar

[5]

E. Bae, J. Yuan and X.-C. Tai, Global minimization for continuous multiphase partitioning problems using a dual approach, International Journal of Computer Vision, 92 (2011), 112-129.  doi: 10.1007/s11263-010-0406-y.  Google Scholar

[6]

Y. Boykov and V. Kolmogorov, An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision, Energy Minimization Methods in Computer Vision and Pattern Recognition, 2134 (2001), 359-374.  doi: 10.1007/3-540-44745-8_24.  Google Scholar

[7]

Y. Boykov, O. Veksler and R. Zabih, Fast approximate energy minimization via graph cuts, Proceedings of the Seventh IEEE International Conference on Computer Vision, 1 (1999), 377-384.  doi: 10.1109/ICCV.1999.791245.  Google Scholar

[8]

X. Bresson, S. Esedoḡlu, P. Vandergheynst, J.-P. Thiran and S. Osher, Fast global minimization of the active contour/snake model, Journal of Mathematical Imaging and Vision, 28 (2007), 151-167.  doi: 10.1007/s10851-007-0002-0.  Google Scholar

[9]

X. Cai, R. Chan and T. Zeng, A two-stage image segmentation method using a convex variant of the Mumford–Shah model and thresholding, SIAM Journal on Imaging Sciences, 6 (2013), 368-390.  doi: 10.1137/120867068.  Google Scholar

[10]

V. Caselles, R. Kimmel and G. Sapiro, Geodesic active contours, International Journal of Computer Vision, 22 (1997), 61-79.  doi: 10.1109/ICCV.1995.466871.  Google Scholar

[11]

A. Chambolle, An algorithm for total variation minimization and applications, Journal of Mathematical Imaging and Vision, 20 (2004), 89-97.   Google Scholar

[12]

T. F. Chan, S. Esedoglu and M. Nikolova, Algorithms for finding global minimizers of image segmentation and denoising models, SIAM Journal on Applied Mathematics, 66 (2006), 1632-1648.  doi: 10.1137/040615286.  Google Scholar

[13]

T. F. Chan and L. A. Vese, Active contours without edges, IEEE Transactions on Image Processing, 10 (2001), 266-277.  doi: 10.1109/83.902291.  Google Scholar

[14]

L.-J. Deng, W. Guo and T.-Z. Huang, Single-image super-resolution via an iterative reproducing kernel hilbert space method, IEEE Transactions on Circuits and Systems for Video Technology, 26 (2016), 2001-2014.  doi: 10.1109/TCSVT.2015.2475895.  Google Scholar

[15]

C. Gout, C. Le Guyader and L. Vese, Segmentation under geometrical conditions using geodesic active contours and interpolation using level set methods, Numerical Algorithms, 39 (2005), 155-173.  doi: 10.1007/s11075-004-3627-8.  Google Scholar

[16]

J.-B. Hiriart-Urruty and C. Lemar{é}chal, Convex Analysis and Minimization Algorithms I: Fundamentals, volume 305., Springer Science & Business Media, 2013. Google Scholar

[17]

P. Jaccard, The distribution of flora in the apline zone.1, New Phytologist, 11 (1912), 37-50.   Google Scholar

[18]

S. H. Kang, B. Shafei and G. Steidl, Supervised and transductive multi-class segmentation using p-Laplacians and RKHS methods, Journal of Visual Communication and Image Representation, 25 (2014), 1136-1148.  doi: 10.1016/j.jvcir.2014.03.010.  Google Scholar

[19]

M. Kass, A. Witkin and D. Terzopoulos, Snakes: Active contour models, International Journal of Computer Vision, 1 (1988), 321-331.  doi: 10.1007/BF00133570.  Google Scholar

[20]

C. Li, C.-Y. Kao, J. C. Gore and Z. Ding, Minimization of region-scalable fitting energy for image segmentation, IEEE Transactions on Image Processing, 17 (2008), 1940-1949.  doi: 10.1109/TIP.2008.2002304.  Google Scholar

[21]

C. Liu, M. Ng and T. Zeng, Weighted variational model for selective image segmentation with application to medical images, Pattern Recognition, 76 (2018), 367-379.  doi: 10.1016/j.patcog.2017.11.019.  Google Scholar

[22]

Z. Li and F. Malgouyres, Regularized non-local total variation and application in image restoration, Journal of Mathematical Imaging and Vision, 59 (2017), 296-317.  doi: 10.1007/s10851-017-0732-6.  Google Scholar

[23]

D. Mumford and J. Shah, Optimal approximations by piecewise smooth functions and associated variational problems, Communications on Pure and Applied Mathematics, 42 (1989), 577-685.  doi: 10.1002/cpa.3160420503.  Google Scholar

[24]

S. Osher and J. A. Sethian, Fronts propagating with curvature-dependent speed: Algorithms based on hamilton-jacobi formulations, Journal of Computational Physics, 79 (1988), 12-49.  doi: 10.1016/0021-9991(88)90002-2.  Google Scholar

[25]

R. B. Potts, Some generalized order-disorder transformations, Mathematical Proceedings of the Cambridge Philosophical Society, 48 (1952), 106-109.  doi: 10.1017/S0305004100027419.  Google Scholar

[26]

M. Roberts, K. Chen and K. L. Irion, A convex geodesic selective model for image segmentation, Journal of Mathematical Imaging and Vision, 61 (2019), 482-503.  doi: 10.1007/s10851-018-0857-2.  Google Scholar

[27]

M. Roberts and J. Spencer, Chan–vese reformulation for selective image segmentation, Journal of Mathematical Imaging and Vision, 61 (2019), 1173-1196.  doi: 10.1007/s10851-019-00893-0.  Google Scholar

[28]

J. Spencer and K. Chen, A convex and selective variational model for image segmentation, Communications in Mathematical Sciences, 13 (2015), 1453-1472.  doi: 10.4310/CMS.2015.v13.n6.a5.  Google Scholar

[29]

X.-F. Wang, D.-S. Huang and H. Xu, An efficient local chan–vese model for image segmentation, Pattern Recognition, 43 (2010), 603-618.  doi: 10.1016/j.patcog.2009.08.002.  Google Scholar

[30]

Y. Xu and W. Yin, A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion, SIAM Journal on Imaging Sciences, 6 (2013), 1758-1789.  doi: 10.1137/120887795.  Google Scholar

[31]

J. Yuan, E. Bae, X.-C. Tai and Y. Boykov, A continuous max-flow approach to potts model, In European Conference on Computer Vision, Springer, 2010,379–392. Google Scholar

[32]

J. Yuan, E. Bae, X.-C. Tai and Y. Boykov, A spatially continuous max-flow and min-cut framework for binary labeling problems, Numerische Mathematik, 126 (2014), 559-587.  doi: 10.1007/s00211-013-0569-x.  Google Scholar

[33]

H. Zhang, Y. Chen and J. Shi, Nonparametric image segmentation using Renyi's statistical dependence measure, Journal of Mathematical Imaging and Vision, 44 (2012), 330-340.  doi: 10.1007/s10851-012-0329-z.  Google Scholar

[34]

H. Zhao, A fast sweeping method for Eikonal equations, Mathematics of Computation, 74 (2005), 603-627.  doi: 10.1090/S0025-5718-04-01678-3.  Google Scholar

Figure 1.  Comparison of Geodesic and Euclidean distance constraints
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2.  Left panel: 3D surface images of $ \psi $ of nine parameter pairs $ (\theta, c) $; right panel: the corresponding 2D images. In each panel, from left to right and then from top to bottom: $ ( \frac{4 \pi}{5} , \frac{51}{1024} ), ( \frac{4\pi}{5} , \frac{25}{64} ), ( \frac{4\pi}{5} , \frac{175}{256} ), ( \frac{6\pi}{5}, \frac{135}{1024} ), ( \frac{6\pi}{5}, \frac{1}{2} ), ( \frac{6\pi}{5}, \frac{25}{32} ), ( \frac{8\pi}{5}, \frac{5}{64} ) $, $ ( \frac{8\pi}{5}, \frac{75}{256} ), ( \frac{8\pi}{5}, \frac{75}{128} ) $
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 3.  The decomposition of the input image (top left), into the smooth parts (top middle) and edge parts (top right), obtained from RKHS model (8), and edge indicator function $ g(\Psi \beta) $ (bottom left). In comparison, we display the gradient of the image, $ |\nabla z| $ and the edge indicator function $ g(|\nabla z|) $ in the bottom middle and bottom right respectively
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 4.  Segmentation of white matter from MRI brain image. We show both the contour overlaid on the original image, and the associated binary segmentation result for each example. First column: segmentation using edge weighted Chan-Vese using $ |\nabla z| $ model (24). Second column: Edge weighted Chan-Vese model (24) using Canny ($ g = g_{\mathcal{C}} $). Third column: Cai et al. [9]. Fourth column: Proposed two stage global model (8) and (25). Final column: The proposed combined global segmentation model (26)
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 5.  Row 1: 3% noise, Row 2: 5% noise, Row 3: 7% noise, Row 4: 9% noise. We show both the contour overlaid on the original image, and the associated binary segmentation result for each example. First two columns: Using model (24). Final two columns: Combined model, algorithm (1)
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 6.  Geodesic distances computed using gradient (middle) and $ \Psi \beta $ (right)
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 7.  Row one: Original clean image and the user input. Row two: Image and result with 20% noise. Row three: Image and result with 40% noise. Row four: Image and result with 60% noise
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 8.  Segmentation of blood vessel
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 9.  Segmentation of blood vessel
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 10.  Result of the combined selective model
Figure Options

Download full-size image

Download as PowerPoint slide

[1]

Lekbir Afraites, Abdelghafour Atlas, Fahd Karami, Driss Meskine. Some class of parabolic systems applied to image processing. Discrete & Continuous Dynamical Systems - B, 2016, 21 (6) : 1671-1687. doi: 10.3934/dcdsb.2016017
[2]

Zhihua Zhang, Naoki Saito. PHLST with adaptive tiling and its application to antarctic remote sensing image approximation. Inverse Problems & Imaging, 2014, 8 (1) : 321-337. doi: 10.3934/ipi.2014.8.321
[3]

Israa Mohammed Khudher, Yahya Ismail Ibrahim, Suhaib Abduljabbar Altamir. Individual biometrics pattern based artificial image analysis techniques. Numerical Algebra, Control & Optimization, 2021  doi: 10.3934/naco.2020056
[4]

Sara Munday. On the derivative of the $\alpha$-Farey-Minkowski function. Discrete & Continuous Dynamical Systems - A, 2014, 34 (2) : 709-732. doi: 10.3934/dcds.2014.34.709
[5]

Ralf Hielscher, Michael Quellmalz. Reconstructing a function on the sphere from its means along vertical slices. Inverse Problems & Imaging, 2016, 10 (3) : 711-739. doi: 10.3934/ipi.2016018
[6]

Jia Cai, Guanglong Xu, Zhensheng Hu. Sketch-based image retrieval via CAT loss with elastic net regularization. Mathematical Foundations of Computing, 2020, 3 (4) : 219-227. doi: 10.3934/mfc.2020013
[7]

Charles Fulton, David Pearson, Steven Pruess. Characterization of the spectral density function for a one-sided tridiagonal Jacobi matrix operator. Conference Publications, 2013, 2013 (special) : 247-257. doi: 10.3934/proc.2013.2013.247
[8]

Dayalal Suthar, Sunil Dutt Purohit, Haile Habenom, Jagdev Singh. Class of integrals and applications of fractional kinetic equation with the generalized multi-index Bessel function. Discrete & Continuous Dynamical Systems - S, 2021  doi: 10.3934/dcdss.2021019
[9]

Saima Rashid, Fahd Jarad, Zakia Hammouch. Some new bounds analogous to generalized proportional fractional integral operator with respect to another function. Discrete & Continuous Dynamical Systems - S, 2021  doi: 10.3934/dcdss.2021020
[10]

Davide La Torre, Simone Marsiglio, Franklin Mendivil, Fabio Privileggi. Public debt dynamics under ambiguity by means of iterated function systems on density functions. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021070

2019 Impact Factor: 1.373

Tools

Metrics

  • PDF downloads (138)
  • HTML views (304)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences