# American Institute of Mathematical Sciences

June  2017, 11(3): 577-600. doi: 10.3934/ipi.2017027

## Image segmentation with dynamic artifacts detection and bias correction

 1 University of California, Los Angeles, Department of Mathematics, 520 Portola Plaza, Box 951555, Los Angeles, CA 90095-1555, USA 2 University of California, Los Angeles, California NanoSystems Institute (CNSI), 570 Westwood Plaza, Building 114, Los Angeles, CA 90095, USA 3 University of California, Los Angeles, Department of Chemistry and Biochemistry, 607 Charles E. Young Drive, Los Angeles, CA 90095, USA 4 University of California, Los Angeles, Department of Materials Science and Engineering, 410 Westwood Plaza, Los Angeles, CA 90095, USA 5 University of California, Los Angeles, Department of Mathematics, 520 Portola Plaza, Box 951555, Los Angeles, CA 90095-1555, USA

1 To whom correspondence should be addressed. Current affiliation and address: Montana State University, Department of Mathematical Sciences, Wilson Hall 2-214, P.O. Box 172400, Bozeman, MT 59717-2400, USA

Received  January 2015 Revised  February 2017 Published  April 2017

Fund Project: This work is supported by the Swiss National Science Foundation under grant P300P2-147778, the California Research Training Program in Computational and Applied Mathematics under grant NSF DMS-1045536, the W. M. Keck Foundation, ONR N00014-16-1-2119, and the Merkin Family Foundation.

Region-based image segmentation is well-addressed by the Chan-Vese (CV) model. However, this approach fails when images are affected by artifacts (outliers) and illumination bias that outweigh the actual image contrast. Here, we introduce a model for segmenting such images. In a single energy functional, we introduce 1) a dynamic artifact class preventing intensity outliers from skewing the segmentation, and 2), in Retinex-fashion, we decompose the image into a piecewise-constant structural part and a smooth bias part. The CV-segmentation terms then only act on the structure, and only in regions not identified as artifacts. The segmentation is parameterized using a phase-field, and efficiently minimized using threshold dynamics.

We demonstrate the proposed model on a series of sample images from diverse modalities exhibiting artifacts and/or bias. Our algorithm typically converges within 10-50 iterations and takes fractions of a second on standard equipment to produce meaningful results. We expect our method to be useful for damaged images, and anticipate use in applications where artifacts and bias are actual features of interest, such as lesion detection and bias field correction in medical imaging, e.g., in magnetic resonance imaging (MRI).

Citation: Dominique Zosso, Jing An, James Stevick, Nicholas Takaki, Morgan Weiss, Liane S. Slaughter, Huan H. Cao, Paul S. Weiss, Andrea L. Bertozzi. Image segmentation with dynamic artifacts detection and bias correction. Inverse Problems & Imaging, 2017, 11 (3) : 577-600. doi: 10.3934/ipi.2017027
##### References:

show all references

##### References:
Example cases 1 & 2. Top: Coronal MRI slice. Input image and initial segmentation contour are shown in the top-right corner. The image is heavily affected by intensity bias, such that the classical CV model fails. The superior parts of the white-matter are undersegmented, while the inferior regions are markedly oversegmented. CV+X is not very helpful, here. In contrast, CV+B fixes the problem: the extracted structure is nearly flat (the brain is essentially two-phase piecewise constant), while CV+XB marks some non-brain pixels as outliers. Bottom: This synthethic image is a combination of piecewise constant regions affected by strong noise and oscillating bias. Again, CV fails, and artifacts detection not appropriate. Bias correction greatly improves the segmentation, but errors persist since from the simple initialization the algorithm converges to a wrong local minimum (upper part of left hand structure)
Example cases 3 & 4. Top: The seemingly simple scene is not segmentable by the CV model alone, due to strong bias. CV+X wrongly classifies bright regions as artifacts. Bias correction (CV+B) results in accurate segmentation of the T-object. Bottom: The vessel structure is not accurately segmented by classical CV: superior parts are oversegmented due to brightening, inferior parts are undersegmented due to darkening. CV+X is inappropriate, while CV+B fixes the problem and leads to much improved vessel segmentation
Example cases 5 & 6. Top: Compare to ex. 4 in figure 2.Bottom: Three-phase piecewise constant synthetic image. The goal is to separate the two black ellipses from the gray background, considering the white ring to be an occlusion artifact. CV, however, groups the white ring with the light background. CV+X successfully identifies the ring as artifact, and closes the black ellipses thanks to the interface regularization. The bias correction is misleading, since much of the white ring will be considered overly illuminated background (CV+B, CV+XB), the corners being captured as transient artifacts (CV+XB)
Results by region-scalable-fitting (RSF) [24]. Left: Starting from generic initialization (as used in our method), RSF fails to capture the correct bias/segmentation result.Right: Starting from a tuned initialization (as provided in [24]), RSF produces the desired bias resistant region-based image segmentation result. Note that for optimally chosen number of iterations, $i$ (as provided in [24]), the computation time is about an order of magnitude slower than our proposed method, as reported in table 1
Microscopy example cases. Top: This AFM image is severely affected by inhomogeneity compared to pattern contrast. As a result, classical CV segmentation fails. The proposed CV+XB model is able to capture some of the bias as such, and correctly segments some of the actual pattern. The central-square contour initialization, however, provokes an incorrect bias field estimate at early stages of the optimization, and leads to an incorrect local minimum, misclassifying the central portions. Middle: Starting from a near-optimal contour initialization of a single pattern element, the CV model still fails entirely. The proposed CV+XB model is not misled into incorrect minima, anymore, and successfully separates bias from actual pattern contrast. Bottom: The CV+XB model captures most of the inhomogeneity present in this AFM sample image and leads to reasonable segmentation of the diamond pattern. (Note: The inversion of foreground/background between CV and CV+XB models is arbitrary and triggered by domain size
Further microscopy examples. Top: Seemingly "easy" AFM sample, actually affected by strong inhomogeneity at different scales. CV+XB yields a flattened structural image and correct pattern segmentation. Second set: Fluorescence microscopy image suffering from strong bias and artifacts, failing classical CV. CV+XB corrects bias and captures most artifacts. Third set: AFM sample with crack (white) and bend (darkening). The darkening is beyond recovery, but the crack is correctly identified as artifact. Bottom: Artifact detection collects bright spots and the dark line, resulting in correct stripe-pattern segmentation
Algorithm convergence: number of iterations and required computer time. The algorithm is deterministic. Convergence is defined as the phase-field $u$ not changing during its update. Computation time is statistical depending on CPU scheduling; here, we report average numbers over 50 repetitions. The extra cost of artifact is negligible compared to basic CV, in particular since it may speed up convergence in appropriate images. Bias correction roughly doubles the computational load per iteration, which is an acceptable price for its benefits when appropriate
 I M × N CV CV+X CV+B CV+XB i [1] t [s] i [1] t [s] i [1] t [s] i [1] t [s] 1 78 × 119 11 0.02 20 0.03 11 0.05 11 0.05 2 75 × 79 14 0.02 16 0.02 65 0.24 66 0.25 3 96 × 127 22 0.06 45 0.12 19 0.11 35 0.21 4 110 × 111 15 0.04 30 0.08 71 0.41 100 0.58 5 131 × 103 33 0.11 33 0.12 30 0.26 36 0.32 6 124 × 184 11 0.05 18 0.07 55 0.45 26 0.23
 I M × N CV CV+X CV+B CV+XB i [1] t [s] i [1] t [s] i [1] t [s] i [1] t [s] 1 78 × 119 11 0.02 20 0.03 11 0.05 11 0.05 2 75 × 79 14 0.02 16 0.02 65 0.24 66 0.25 3 96 × 127 22 0.06 45 0.12 19 0.11 35 0.21 4 110 × 111 15 0.04 30 0.08 71 0.41 100 0.58 5 131 × 103 33 0.11 33 0.12 30 0.26 36 0.32 6 124 × 184 11 0.05 18 0.07 55 0.45 26 0.23
 [1] Mehdi Bastani, Davod Khojasteh Salkuyeh. On the GSOR iteration method for image restoration. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 27-43. doi: 10.3934/naco.2020013 [2] Weiwei Liu, Jinliang Wang, Yuming Chen. Threshold dynamics of a delayed nonlocal reaction-diffusion cholera model. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020316 [3] Cuicui Li, Lin Zhou, Zhidong Teng, Buyu Wen. The threshold dynamics of a discrete-time echinococcosis transmission model. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020339 [4] Abdelghafour Atlas, Mostafa Bendahmane, Fahd Karami, Driss Meskine, Omar Oubbih. A nonlinear fractional reaction-diffusion system applied to image denoising and decomposition. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020321 [5] Bahaaeldin Abdalla, Thabet Abdeljawad. Oscillation criteria for kernel function dependent fractional dynamic equations. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020443 [6] Chao Xing, Jiaojiao Pan, Hong Luo. Stability and dynamic transition of a toxin-producing phytoplankton-zooplankton model with additional food. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020275 [7] Haixiang Yao, Ping Chen, Miao Zhang, Xun Li. Dynamic discrete-time portfolio selection for defined contribution pension funds with inflation risk. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020166 [8] Jia Cai, Guanglong Xu, Zhensheng Hu. Sketch-based image retrieval via CAT loss with elastic net regularization. Mathematical Foundations of Computing, 2020, 3 (4) : 219-227. doi: 10.3934/mfc.2020013 [9] Manil T. Mohan. First order necessary conditions of optimality for the two dimensional tidal dynamics system. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020045 [10] Shao-Xia Qiao, Li-Jun Du. Propagation dynamics of nonlocal dispersal equations with inhomogeneous bistable nonlinearity. Electronic Research Archive, , () : -. doi: 10.3934/era.2020116 [11] Ebraheem O. Alzahrani, Muhammad Altaf Khan. Androgen driven evolutionary population dynamics in prostate cancer growth. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020426 [12] Hong Niu, Zhijiang Feng, Qijin Xiao, Yajun Zhang. A PID control method based on optimal control strategy. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 117-126. doi: 10.3934/naco.2020019 [13] Li-Bin Liu, Ying Liang, Jian Zhang, Xiaobing Bao. A robust adaptive grid method for singularly perturbed Burger-Huxley equations. Electronic Research Archive, 2020, 28 (4) : 1439-1457. doi: 10.3934/era.2020076 [14] Zexuan Liu, Zhiyuan Sun, Jerry Zhijian Yang. A numerical study of superconvergence of the discontinuous Galerkin method by patch reconstruction. Electronic Research Archive, 2020, 28 (4) : 1487-1501. doi: 10.3934/era.2020078 [15] Yuxia Guo, Shaolong Peng. A direct method of moving planes for fully nonlinear nonlocal operators and applications. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020462 [16] Noah Stevenson, Ian Tice. A truncated real interpolation method and characterizations of screened Sobolev spaces. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5509-5566. doi: 10.3934/cpaa.2020250 [17] Yue Feng, Yujie Liu, Ruishu Wang, Shangyou Zhang. A conforming discontinuous Galerkin finite element method on rectangular partitions. Electronic Research Archive, , () : -. doi: 10.3934/era.2020120 [18] José Luis López. A quantum approach to Keller-Segel dynamics via a dissipative nonlinear Schrödinger equation. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020376 [19] A. M. Elaiw, N. H. AlShamrani, A. Abdel-Aty, H. Dutta. Stability analysis of a general HIV dynamics model with multi-stages of infected cells and two routes of infection. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020441 [20] Leilei Wei, Yinnian He. A fully discrete local discontinuous Galerkin method with the generalized numerical flux to solve the tempered fractional reaction-diffusion equation. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020319

2019 Impact Factor: 1.373