# American Institute of Mathematical Sciences

June  2017, 11(3): 577-600. doi: 10.3934/ipi.2017027

## Image segmentation with dynamic artifacts detection and bias correction

 1 University of California, Los Angeles, Department of Mathematics, 520 Portola Plaza, Box 951555, Los Angeles, CA 90095-1555, USA 2 University of California, Los Angeles, California NanoSystems Institute (CNSI), 570 Westwood Plaza, Building 114, Los Angeles, CA 90095, USA 3 University of California, Los Angeles, Department of Chemistry and Biochemistry, 607 Charles E. Young Drive, Los Angeles, CA 90095, USA 4 University of California, Los Angeles, Department of Materials Science and Engineering, 410 Westwood Plaza, Los Angeles, CA 90095, USA 5 University of California, Los Angeles, Department of Mathematics, 520 Portola Plaza, Box 951555, Los Angeles, CA 90095-1555, USA

1 To whom correspondence should be addressed. Current affiliation and address: Montana State University, Department of Mathematical Sciences, Wilson Hall 2-214, P.O. Box 172400, Bozeman, MT 59717-2400, USA

Received  January 2015 Revised  February 2017 Published  April 2017

Fund Project: This work is supported by the Swiss National Science Foundation under grant P300P2-147778, the California Research Training Program in Computational and Applied Mathematics under grant NSF DMS-1045536, the W. M. Keck Foundation, ONR N00014-16-1-2119, and the Merkin Family Foundation.

Region-based image segmentation is well-addressed by the Chan-Vese (CV) model. However, this approach fails when images are affected by artifacts (outliers) and illumination bias that outweigh the actual image contrast. Here, we introduce a model for segmenting such images. In a single energy functional, we introduce 1) a dynamic artifact class preventing intensity outliers from skewing the segmentation, and 2), in Retinex-fashion, we decompose the image into a piecewise-constant structural part and a smooth bias part. The CV-segmentation terms then only act on the structure, and only in regions not identified as artifacts. The segmentation is parameterized using a phase-field, and efficiently minimized using threshold dynamics.

We demonstrate the proposed model on a series of sample images from diverse modalities exhibiting artifacts and/or bias. Our algorithm typically converges within 10-50 iterations and takes fractions of a second on standard equipment to produce meaningful results. We expect our method to be useful for damaged images, and anticipate use in applications where artifacts and bias are actual features of interest, such as lesion detection and bias field correction in medical imaging, e.g., in magnetic resonance imaging (MRI).

Citation: Dominique Zosso, Jing An, James Stevick, Nicholas Takaki, Morgan Weiss, Liane S. Slaughter, Huan H. Cao, Paul S. Weiss, Andrea L. Bertozzi. Image segmentation with dynamic artifacts detection and bias correction. Inverse Problems & Imaging, 2017, 11 (3) : 577-600. doi: 10.3934/ipi.2017027
##### References:

show all references

##### References:
Example cases 1 & 2. Top: Coronal MRI slice. Input image and initial segmentation contour are shown in the top-right corner. The image is heavily affected by intensity bias, such that the classical CV model fails. The superior parts of the white-matter are undersegmented, while the inferior regions are markedly oversegmented. CV+X is not very helpful, here. In contrast, CV+B fixes the problem: the extracted structure is nearly flat (the brain is essentially two-phase piecewise constant), while CV+XB marks some non-brain pixels as outliers. Bottom: This synthethic image is a combination of piecewise constant regions affected by strong noise and oscillating bias. Again, CV fails, and artifacts detection not appropriate. Bias correction greatly improves the segmentation, but errors persist since from the simple initialization the algorithm converges to a wrong local minimum (upper part of left hand structure)
Example cases 3 & 4. Top: The seemingly simple scene is not segmentable by the CV model alone, due to strong bias. CV+X wrongly classifies bright regions as artifacts. Bias correction (CV+B) results in accurate segmentation of the T-object. Bottom: The vessel structure is not accurately segmented by classical CV: superior parts are oversegmented due to brightening, inferior parts are undersegmented due to darkening. CV+X is inappropriate, while CV+B fixes the problem and leads to much improved vessel segmentation
.Bottom: Three-phase piecewise constant synthetic image. The goal is to separate the two black ellipses from the gray background, considering the white ring to be an occlusion artifact. CV, however, groups the white ring with the light background. CV+X successfully identifies the ring as artifact, and closes the black ellipses thanks to the interface regularization. The bias correction is misleading, since much of the white ring will be considered overly illuminated background (CV+B, CV+XB), the corners being captured as transient artifacts (CV+XB)">Figure 3.  Example cases 5 & 6. Top: Compare to ex. 4 in figure 2.Bottom: Three-phase piecewise constant synthetic image. The goal is to separate the two black ellipses from the gray background, considering the white ring to be an occlusion artifact. CV, however, groups the white ring with the light background. CV+X successfully identifies the ring as artifact, and closes the black ellipses thanks to the interface regularization. The bias correction is misleading, since much of the white ring will be considered overly illuminated background (CV+B, CV+XB), the corners being captured as transient artifacts (CV+XB)
]. Left: Starting from generic initialization (as used in our method), RSF fails to capture the correct bias/segmentation result.Right: Starting from a tuned initialization (as provided in [24]), RSF produces the desired bias resistant region-based image segmentation result. Note that for optimally chosen number of iterations, $i$ (as provided in [24]), the computation time is about an order of magnitude slower than our proposed method, as reported in table 1">Figure 4.  Results by region-scalable-fitting (RSF) [24]. Left: Starting from generic initialization (as used in our method), RSF fails to capture the correct bias/segmentation result.Right: Starting from a tuned initialization (as provided in [24]), RSF produces the desired bias resistant region-based image segmentation result. Note that for optimally chosen number of iterations, $i$ (as provided in [24]), the computation time is about an order of magnitude slower than our proposed method, as reported in table 1
Microscopy example cases. Top: This AFM image is severely affected by inhomogeneity compared to pattern contrast. As a result, classical CV segmentation fails. The proposed CV+XB model is able to capture some of the bias as such, and correctly segments some of the actual pattern. The central-square contour initialization, however, provokes an incorrect bias field estimate at early stages of the optimization, and leads to an incorrect local minimum, misclassifying the central portions. Middle: Starting from a near-optimal contour initialization of a single pattern element, the CV model still fails entirely. The proposed CV+XB model is not misled into incorrect minima, anymore, and successfully separates bias from actual pattern contrast. Bottom: The CV+XB model captures most of the inhomogeneity present in this AFM sample image and leads to reasonable segmentation of the diamond pattern. (Note: The inversion of foreground/background between CV and CV+XB models is arbitrary and triggered by domain size
Further microscopy examples. Top: Seemingly "easy" AFM sample, actually affected by strong inhomogeneity at different scales. CV+XB yields a flattened structural image and correct pattern segmentation. Second set: Fluorescence microscopy image suffering from strong bias and artifacts, failing classical CV. CV+XB corrects bias and captures most artifacts. Third set: AFM sample with crack (white) and bend (darkening). The darkening is beyond recovery, but the crack is correctly identified as artifact. Bottom: Artifact detection collects bright spots and the dark line, resulting in correct stripe-pattern segmentation
Algorithm convergence: number of iterations and required computer time. The algorithm is deterministic. Convergence is defined as the phase-field $u$ not changing during its update. Computation time is statistical depending on CPU scheduling; here, we report average numbers over 50 repetitions. The extra cost of artifact is negligible compared to basic CV, in particular since it may speed up convergence in appropriate images. Bias correction roughly doubles the computational load per iteration, which is an acceptable price for its benefits when appropriate
 I M × N CV CV+X CV+B CV+XB i [1] t [s] i [1] t [s] i [1] t [s] i [1] t [s] 1 78 × 119 11 0.02 20 0.03 11 0.05 11 0.05 2 75 × 79 14 0.02 16 0.02 65 0.24 66 0.25 3 96 × 127 22 0.06 45 0.12 19 0.11 35 0.21 4 110 × 111 15 0.04 30 0.08 71 0.41 100 0.58 5 131 × 103 33 0.11 33 0.12 30 0.26 36 0.32 6 124 × 184 11 0.05 18 0.07 55 0.45 26 0.23
 I M × N CV CV+X CV+B CV+XB i [1] t [s] i [1] t [s] i [1] t [s] i [1] t [s] 1 78 × 119 11 0.02 20 0.03 11 0.05 11 0.05 2 75 × 79 14 0.02 16 0.02 65 0.24 66 0.25 3 96 × 127 22 0.06 45 0.12 19 0.11 35 0.21 4 110 × 111 15 0.04 30 0.08 71 0.41 100 0.58 5 131 × 103 33 0.11 33 0.12 30 0.26 36 0.32 6 124 × 184 11 0.05 18 0.07 55 0.45 26 0.23
 [1] Maika Goto, Kazunori Kuwana, Yasuhide Uegata, Shigetoshi Yazaki. A method how to determine parameters arising in a smoldering evolution equation by image segmentation for experiment's movies. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 881-891. doi: 10.3934/dcdss.2020233 [2] Fan Jia, Xue-Cheng Tai, Jun Liu. Nonlocal regularized CNN for image segmentation. Inverse Problems & Imaging, 2020, 14 (5) : 891-911. doi: 10.3934/ipi.2020041 [3] Yuan Wang, Zhi-Feng Pang, Yuping Duan, Ke Chen. Image retinex based on the nonconvex TV-type regularization. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2020050 [4] Ye Yuan, Yan Ren, Xiaodong Liu, Jing Wang. Approach to image segmentation based on interval neutrosophic set. Numerical Algebra, Control & Optimization, 2020, 10 (1) : 1-11. doi: 10.3934/naco.2019028 [5] Matthew S. Keegan, Berta Sandberg, Tony F. Chan. A multiphase logic framework for multichannel image segmentation. Inverse Problems & Imaging, 2012, 6 (1) : 95-110. doi: 10.3934/ipi.2012.6.95 [6] Shi Yan, Jun Liu, Haiyang Huang, Xue-Cheng Tai. A dual EM algorithm for TV regularized Gaussian mixture model in image segmentation. Inverse Problems & Imaging, 2019, 13 (3) : 653-677. doi: 10.3934/ipi.2019030 [7] Jianping Zhang, Ke Chen, Bo Yu, Derek A. Gould. A local information based variational model for selective image segmentation. Inverse Problems & Imaging, 2014, 8 (1) : 293-320. doi: 10.3934/ipi.2014.8.293 [8] Lu Tan, Ling Li, Senjian An, Zhenkuan Pan. Nonlinear diffusion based image segmentation using two fast algorithms. Mathematical Foundations of Computing, 2019, 2 (2) : 149-168. doi: 10.3934/mfc.2019011 [9] Ruiliang Zhang, Xavier Bresson, Tony F. Chan, Xue-Cheng Tai. Four color theorem and convex relaxation for image segmentation with any number of regions. Inverse Problems & Imaging, 2013, 7 (3) : 1099-1113. doi: 10.3934/ipi.2013.7.1099 [10] Balázs Kósa, Karol Mikula, Markjoe Olunna Uba, Antonia Weberling, Neophytos Christodoulou, Magdalena Zernicka-Goetz. 3D image segmentation supported by a point cloud. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 971-985. doi: 10.3934/dcdss.2020351 [11] Jie Huang, Xiaoping Yang, Yunmei Chen. A fast algorithm for global minimization of maximum likelihood based on ultrasound image segmentation. Inverse Problems & Imaging, 2011, 5 (3) : 645-657. doi: 10.3934/ipi.2011.5.645 [12] Liam Burrows, Weihong Guo, Ke Chen, Francesco Torella. Reproducible kernel Hilbert space based global and local image segmentation. Inverse Problems & Imaging, 2021, 15 (1) : 1-25. doi: 10.3934/ipi.2020048 [13] Mehrdad Nojoumian, Douglas R. Stinson. On dealer-free dynamic threshold schemes. Advances in Mathematics of Communications, 2013, 7 (1) : 39-56. doi: 10.3934/amc.2013.7.39 [14] Xinli Hu. Threshold dynamics for a Tuberculosis model with seasonality. Mathematical Biosciences & Engineering, 2012, 9 (1) : 111-122. doi: 10.3934/mbe.2012.9.111 [15] Mehdi Bastani, Davod Khojasteh Salkuyeh. On the GSOR iteration method for image restoration. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 27-43. doi: 10.3934/naco.2020013 [16] Zhenguo Bai, Yicang Zhou. Threshold dynamics of a bacillary dysentery model with seasonal fluctuation. Discrete & Continuous Dynamical Systems - B, 2011, 15 (1) : 1-14. doi: 10.3934/dcdsb.2011.15.1 [17] Jinhuo Luo, Jin Wang, Hao Wang. Seasonal forcing and exponential threshold incidence in cholera dynamics. Discrete & Continuous Dynamical Systems - B, 2017, 22 (6) : 2261-2290. doi: 10.3934/dcdsb.2017095 [18] Baoli Shi, Zhi-Feng Pang, Jing Xu. Image segmentation based on the hybrid total variation model and the K-means clustering strategy. Inverse Problems & Imaging, 2016, 10 (3) : 807-828. doi: 10.3934/ipi.2016022 [19] Yupeng Li, Wuchen Li, Guo Cao. Image segmentation via $L_1$ Monge-Kantorovich problem. Inverse Problems & Imaging, 2019, 13 (4) : 805-826. doi: 10.3934/ipi.2019037 [20] Qianting Ma, Tieyong Zeng, Dexing Kong, Jianwei Zhang. Weighted area constraints-based breast lesion segmentation in ultrasound image analysis. Inverse Problems & Imaging, , () : -. doi: 10.3934/ipi.2021057

2020 Impact Factor: 1.639