Quasiconformal model with CNN features for large deformation image registration

Ho Law; Gary P. T. Choi; Ka Chun Lam; Lok Ming Lui

doi:10.3934/ipi.2022010

Previous Article
IPI Home
This Issue
Next Article
A non-iterative sampling method for inverse elastic wave scattering by rough surfaces

August 2022, 16(4): 1019-1046. doi: 10.3934/ipi.2022010

Quasiconformal model with CNN features for large deformation image registration

Ho Law ^1, , Gary P. T. Choi ^2, , Ka Chun Lam ^3, and Lok Ming Lui ^4,

1.	School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332, USA
2.	Department of Mathematics, Massachusetts Institute of Technology, Cambridge, MA 02139, USA
3.	Machine Learning Team, National Institute of Mental Health, Bethesda, MD 20892, USA
4.	Department of Mathematics, The Chinese University of Hong Kong, Shatin, NT, Hong Kong, China

Received June 2021 Revised January 2022 Published August 2022 Early access March 2022

Fund Project: This work was supported in part by the National Science Foundation under Grant No. DMS-2002103 (to Gary P. T. Choi), and HKRGC GRF under project ID 14305919 (to Lok Ming Lui)

Image registration has been widely studied over the past several decades, with numerous applications in science, engineering and medicine. Most of the conventional mathematical models for large deformation image registration rely on prescribed landmarks, which usually require tedious manual labeling. In recent years, there has been a surge of interest in the use of machine learning for image registration. In this paper, we develop a novel method for large deformation image registration by a fusion of quasiconformal theory and convolutional neural network (CNN). More specifically, we propose a quasiconformal energy model with a novel fidelity term that incorporates the features extracted using a pre-trained CNN, thereby allowing us to obtain meaningful registration results without any guidance of prescribed landmarks. Moreover, unlike many prior image registration methods, the bijectivity of our method is guaranteed by quasiconformal theory. Experimental results are presented to demonstrate the effectiveness of the proposed method. More broadly, our work sheds light on how rigorous mathematical theories and practical machine learning approaches can be integrated for developing computational methods with improved performance.

Keywords: Image registration, convolutional neural networks, quasiconformal theory.

Mathematics Subject Classification: 65D18, 68U05, 68U10, 68T07.

Citation: Ho Law, Gary P. T. Choi, Ka Chun Lam, Lok Ming Lui. Quasiconformal model with CNN features for large deformation image registration. Inverse Problems and Imaging, 2022, 16 (4) : 1019-1046. doi: 10.3934/ipi.2022010

References:

[1]	The National Lung Screening Trial (NLST), https://cdas.cancer.gov/nlst/.
[2]	Open Access Biomedical Image Search Engine, https://openi.nlm.nih.gov/.
[3]	G. Balakrishnan, A. Zhao, M. R. Sabuncu, J. Guttag and A. V. Dalca, VoxelMorph: A learning framework for deformable medical image registration, IEEE Transactions on Medical Imaging, 38 (2019), 1788-1800. doi: 10.1109/TMI.2019.2897538.
[4]	V. Balntas, E. Johns, L. Tang and K. Mikolajczyk, PN-Net: Conjoined triple deep network for learning local image descriptors, Preprint, arXiv: 1601.05030.
[5]	M. F. Beg, M. I. Miller, A. Trouvé and L. Younes, Computing large deformation metric mappings via geodesic flows of diffeomorphisms, International Journal of Computer Vision, 61 (2005), 139-157. doi: 10.1023/B:VISI.0000043755.93987.aa.
[6]	F. L. Bookstein, Principal warps: Thin-plate splines and the decomposition of deformations, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11 (1989), 567-585. doi: 10.1109/34.24792.
[7]	L. G. Brown, A survey of image registration techniques, ACM Computing Surveys (CSUR), 24 (1992), 325-376. doi: 10.1145/146370.146374.
[8]	G. P. T. Choi, H. L. Chan, R. Yong, S. Ranjitkar, A. Brook, G. Townsend, K. Chen and L. M. Lui, Tooth morphometry using quasi-conformal theory, Pattern Recognition, 99 (2020), 107064. doi: 10.1016/j.patcog.2019.107064.
[9]	G. P. T. Choi, Y. Leung-Liu, X. Gu and L. M. Lui, Parallelizable global conformal parameterization of simply-connected surfaces via partial welding, SIAM J. Imaging Sci., 13 (2020), 1049-1083. doi: 10.1137/19M125337X.
[10]	G. P. T. Choi, Y. Liu and L. M. Lui, Free-boundary conformal parameterization of point clouds, J. Sci. Comput., 90 (2022), Paper No. 14, 26 pp. doi: 10.1007/s10915-021-01641-6.
[11]	G. P.-T. Choi and L. M. Lui, A linear formulation for disk conformal parameterization of simply-connected open surfaces, Adv. Comput. Math., 44 (2018), 87-114. doi: 10.1007/s10444-017-9536-x.
[12]	G. P. T. Choi, D. Qiu and L. M. Lui, Shape analysis via inconsistent surface registration, Proc. A., 476 (2020), 20200147, 21 pp. doi: 10.1098/rspa.2020.0147.
[13]	P. T. Choi, K. C. Lam and L. M. Lui, FLASH: Fast landmark aligned spherical harmonic parameterization for genus-0 closed brain surfaces, SIAM J. Imaging Sci., 8 (2015), 67-94. doi: 10.1137/130950008.
[14]	P. T. Choi and L. M. Lui, Fast disk conformal parameterization of simply-connected open surfaces, J. Sci. Comput., 65 (2015), 1065-1090. doi: 10.1007/s10915-015-9998-2.
[15]	Z. Daoping and K. Chen, 3D orientation-preserving variational models for accurate image registration, SIAM J. Imaging Sci., 13 (2020), 1653-1691. doi: 10.1137/20M1320006.
[16]	B. D. de Vos, F. F. Berendsen, M. A. Viergever, H. Sokooti, M. Staring and I. Išgum, A deep learning framework for unsupervised affine and deformable image registration, Medical Image Analysis, 52 (2019), 128-143.
[17]	F. P. Gardiner and N. Lakic, Quasiconformal Teichmüller Theory, 76, American Mathematical Society, 2000. doi: 10.1090/surv/076.
[18]	B. Glocker, Drop - Deformable registration using discrete optimization, http://campar.in.tum.de/Main/Drop.
[19]	B. Glocker, A. Sotiras, N. Komodakis and N. Paragios, Deformable medical image registration: Setting the state of the art with discrete methods, Annual Review of Biomedical Engineering, 13 (2011), 219-244. doi: 10.1146/annurev-bioeng-071910-124649.
[20]	I. Goodfellow, Y. Bengio and A. Courville, Deep Learning, MIT Press, 2016, http://www.deeplearningbook.org.
[21]	X. Han, T. Leung, Y. Jia, R. Sukthankar and A. C. Berg, MatchNet: Unifying feature and metric learning for patch-based matching, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2015), 3279–3286.
[22]	K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2016), 770–778. doi: 10.1109/CVPR.2016.90.
[23]	B. K. P. Horn and B. G. Schunck, Determining optical flow, Artificial Intelligence, 17 (1981), 185-203. doi: 10.1016/0004-3702(81)90024-2.
[24]	G. Huang, Z. Liu, L. Van Der Maaten and K. Q. Weinberger, Densely connected convolutional networks, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2017), 4700–4708. doi: 10.1109/CVPR.2017.243.
[25]	M. Jahrer, M. Grabner and H. Bischof, Learned local descriptors for recognition and matching, In Computer Vision Winter Workshop, 2 (2008).
[26]	F. Jia, J. Liu and X.-C. Tai, A regularized convolutional neural network for semantic image segmentation, Anal. Appl., 19 (2021), 147-165. doi: 10.1142/S0219530519410148.
[27]	H. J. Johnson and G. E. Christensen, Consistent landmark and intensity-based image registration, IEEE Transactions on Medical Imaging, 21 (2002), 450-461. doi: 10.1109/TMI.2002.1009381.
[28]	S. C. Joshi and M. I. Miller, Landmark matching via large deformation diffeomorphisms, IEEE Transactions on Image Processing, 9 (2000), 1357-1370. doi: 10.1109/83.855431.
[29]	S. Klein and M. Staring, Elastix: A toolbox for rigid and nonrigid registration of images, https://elastix.lumc.nl/.
[30]	S. Klein, M. Staring, K. Murphy, M. A. Viergever and J. P. Pluim, Elastix: A toolbox for intensity-based medical image registration, IEEE Transactions on Medical Imaging, 29 (2010), 196-205. doi: 10.1109/TMI.2009.2035616.
[31]	D.-J. Kroon, Multimodality non-rigid demon algorithm image registration, MATLAB Central File Exchange, https://www.mathworks.com/matlabcentral/fileexchange/21451-multimodality-non-rigid-demon-algorithm-image-registration.
[32]	D. Kuang, Cycle-consistent training for reducing negative jacobian determinant in deep registration networks, In International Workshop on Simulation and Synthesis in Medical Imaging, Springer, 11827 (2019), 120–129. doi: 10.1007/978-3-030-32778-1_13.
[33]	D. Kuang and T. Schmah, FAIM–a ConvNet method for unsupervised 3D medical image registration, In International Workshop on Machine Learning in Medical Imaging, Springer, 11861 (2019), 646–654. doi: 10.1007/978-3-030-32692-0_74.
[34]	K. C. Lam and L. M. Lui, Landmark- and intensity-based registration with large deformations via quasi-conformal maps, SIAM J. Imaging Sci., 7 (2014), 2364-2392. doi: 10.1137/130943406.
[35]	Y. T. Lee, K. C. Lam and L. M. Lui, Landmark-matching transformation with large deformation via n-dimensional quasi-conformal maps, J. Sci. Comput., 67 (2016), 926-954. doi: 10.1007/s10915-015-0113-5.
[36]	O. Lehto and K. I. Virtanen, Quasiconformal Mappings in the Plane, vol. 126, 2$^{nd}$ edition, Springer-Verlag Berlin Heidelberg, 1973.
[37]	H. Lombaert, Diffeomorphic log demons image registration, MATLAB Central File Exchange, https://www.mathworks.com/matlabcentral/fileexchange/39194-diffeomorphic-log-demons-image-registration.
[38]	J. B. A. Maintz and M. A. Viergever, A survey of medical image registration, Medical Image Analysis, 2 (1998), 1-36. doi: 10.1016/S1361-8415(01)80026-8.
[39]	T. W. Meng, G. P.-T. Choi and L. M. Lui, TEMPO: Feature-endowed Teichmüller extremal mappings of point clouds, SIAM J. Imaging Sci., 9 (2016), 1922-1962. doi: 10.1137/15M1049117.
[40]	J. Modersitzki, FAIR: Flexible Algorithms for Image Registration, SIAM, 2009. doi: 10.1137/1.9780898718843.
[41]	I. Rocco, R. Arandjelovic and J. Sivic, Convolutional neural network architecture for geometric matching, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2017), 6148–6157. doi: 10.1109/CVPR.2017.12.
[42]	E. Simo-Serra, E. Trulls, L. Ferraz, I. Kokkinos, P. Fua and F. Moreno-Noguer, Discriminative learning of deep convolutional feature point descriptors, In Proceedings of the IEEE International Conference on Computer Vision, (2015), 118–126. doi: 10.1109/ICCV.2015.22.
[43]	K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, Preprint, arXiv: 1409.1556.
[44]	S. Sommer, Segframe image registration, https://github.com/nefan/segframe.
[45]	A. Sotiras, C. Davatzikos and N. Paragios, Deformable medical image registration: A survey, IEEE Transactions on Medical Imaging, 32 (2013), 1153-1190. doi: 10.1109/TMI.2013.2265603.
[46]	J.-P. Thirion, Image matching as a diffusion process: An analogy with Maxwell's demons, Medical Image Analysis, 2 (1998), 243-260. doi: 10.1016/S1361-8415(98)80022-4.
[47]	T. Vercauteren, X. Pennec, A. Perchant and N. Ayache, Diffeomorphic demons: Efficient non-parametric image registration, NeuroImage, 45 (2009), 61-72. doi: 10.1016/j.neuroimage.2008.10.040.
[48]	H. Wang, L. Dong, J. O'Daniel, R. Mohan, A. S. Garden, K. K. Ang, D. A. Kuban, M. Bonnen, J. Y. Chang and R. Cheung, Validation of an accelerated 'Demons' algorithm for deformable image registration in radiation therapy, Physics in Medicine & Biology, 50 (2005), 2887. doi: 10.1088/0031-9155/50/12/011.
[49]	X. Yang, R. Kwitt, M. Styner and M. Niethammer, Quicksilver: Fast predictive image registration–a deep learning approach, NeuroImage, 158 (2017), 378-396. doi: 10.1016/j.neuroimage.2017.07.008.
[50]	C. P. Yung, G. P. T. Choi, K. Chen and L. M. Lui, Efficient feature-based image registration by mapping sparsified surfaces, Journal of Visual Communication and Image Representation, 55 (2018), 561-571. doi: 10.1016/j.jvcir.2018.07.005.
[51]	S. Zagoruyko and N. Komodakis, Learning to compare image patches via convolutional neural networks, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2015), 4353–4361. doi: 10.1109/CVPR.2015.7299064.
[52]	B. Zitova and J. Flusser, Image registration methods: A survey, Image and Vision Computing, 21 (2003), 977-1000. doi: 10.1016/S0262-8856(03)00137-9.

show all references

References:

[1]	The National Lung Screening Trial (NLST), https://cdas.cancer.gov/nlst/.
[2]	Open Access Biomedical Image Search Engine, https://openi.nlm.nih.gov/.
[3]	G. Balakrishnan, A. Zhao, M. R. Sabuncu, J. Guttag and A. V. Dalca, VoxelMorph: A learning framework for deformable medical image registration, IEEE Transactions on Medical Imaging, 38 (2019), 1788-1800. doi: 10.1109/TMI.2019.2897538.
[4]	V. Balntas, E. Johns, L. Tang and K. Mikolajczyk, PN-Net: Conjoined triple deep network for learning local image descriptors, Preprint, arXiv: 1601.05030.
[5]	M. F. Beg, M. I. Miller, A. Trouvé and L. Younes, Computing large deformation metric mappings via geodesic flows of diffeomorphisms, International Journal of Computer Vision, 61 (2005), 139-157. doi: 10.1023/B:VISI.0000043755.93987.aa.
[6]	F. L. Bookstein, Principal warps: Thin-plate splines and the decomposition of deformations, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11 (1989), 567-585. doi: 10.1109/34.24792.
[7]	L. G. Brown, A survey of image registration techniques, ACM Computing Surveys (CSUR), 24 (1992), 325-376. doi: 10.1145/146370.146374.
[8]	G. P. T. Choi, H. L. Chan, R. Yong, S. Ranjitkar, A. Brook, G. Townsend, K. Chen and L. M. Lui, Tooth morphometry using quasi-conformal theory, Pattern Recognition, 99 (2020), 107064. doi: 10.1016/j.patcog.2019.107064.
[9]	G. P. T. Choi, Y. Leung-Liu, X. Gu and L. M. Lui, Parallelizable global conformal parameterization of simply-connected surfaces via partial welding, SIAM J. Imaging Sci., 13 (2020), 1049-1083. doi: 10.1137/19M125337X.
[10]	G. P. T. Choi, Y. Liu and L. M. Lui, Free-boundary conformal parameterization of point clouds, J. Sci. Comput., 90 (2022), Paper No. 14, 26 pp. doi: 10.1007/s10915-021-01641-6.
[11]	G. P.-T. Choi and L. M. Lui, A linear formulation for disk conformal parameterization of simply-connected open surfaces, Adv. Comput. Math., 44 (2018), 87-114. doi: 10.1007/s10444-017-9536-x.
[12]	G. P. T. Choi, D. Qiu and L. M. Lui, Shape analysis via inconsistent surface registration, Proc. A., 476 (2020), 20200147, 21 pp. doi: 10.1098/rspa.2020.0147.
[13]	P. T. Choi, K. C. Lam and L. M. Lui, FLASH: Fast landmark aligned spherical harmonic parameterization for genus-0 closed brain surfaces, SIAM J. Imaging Sci., 8 (2015), 67-94. doi: 10.1137/130950008.
[14]	P. T. Choi and L. M. Lui, Fast disk conformal parameterization of simply-connected open surfaces, J. Sci. Comput., 65 (2015), 1065-1090. doi: 10.1007/s10915-015-9998-2.
[15]	Z. Daoping and K. Chen, 3D orientation-preserving variational models for accurate image registration, SIAM J. Imaging Sci., 13 (2020), 1653-1691. doi: 10.1137/20M1320006.
[16]	B. D. de Vos, F. F. Berendsen, M. A. Viergever, H. Sokooti, M. Staring and I. Išgum, A deep learning framework for unsupervised affine and deformable image registration, Medical Image Analysis, 52 (2019), 128-143.
[17]	F. P. Gardiner and N. Lakic, Quasiconformal Teichmüller Theory, 76, American Mathematical Society, 2000. doi: 10.1090/surv/076.
[18]	B. Glocker, Drop - Deformable registration using discrete optimization, http://campar.in.tum.de/Main/Drop.
[19]	B. Glocker, A. Sotiras, N. Komodakis and N. Paragios, Deformable medical image registration: Setting the state of the art with discrete methods, Annual Review of Biomedical Engineering, 13 (2011), 219-244. doi: 10.1146/annurev-bioeng-071910-124649.
[20]	I. Goodfellow, Y. Bengio and A. Courville, Deep Learning, MIT Press, 2016, http://www.deeplearningbook.org.
[21]	X. Han, T. Leung, Y. Jia, R. Sukthankar and A. C. Berg, MatchNet: Unifying feature and metric learning for patch-based matching, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2015), 3279–3286.
[22]	K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2016), 770–778. doi: 10.1109/CVPR.2016.90.
[23]	B. K. P. Horn and B. G. Schunck, Determining optical flow, Artificial Intelligence, 17 (1981), 185-203. doi: 10.1016/0004-3702(81)90024-2.
[24]	G. Huang, Z. Liu, L. Van Der Maaten and K. Q. Weinberger, Densely connected convolutional networks, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2017), 4700–4708. doi: 10.1109/CVPR.2017.243.
[25]	M. Jahrer, M. Grabner and H. Bischof, Learned local descriptors for recognition and matching, In Computer Vision Winter Workshop, 2 (2008).
[26]	F. Jia, J. Liu and X.-C. Tai, A regularized convolutional neural network for semantic image segmentation, Anal. Appl., 19 (2021), 147-165. doi: 10.1142/S0219530519410148.
[27]	H. J. Johnson and G. E. Christensen, Consistent landmark and intensity-based image registration, IEEE Transactions on Medical Imaging, 21 (2002), 450-461. doi: 10.1109/TMI.2002.1009381.
[28]	S. C. Joshi and M. I. Miller, Landmark matching via large deformation diffeomorphisms, IEEE Transactions on Image Processing, 9 (2000), 1357-1370. doi: 10.1109/83.855431.
[29]	S. Klein and M. Staring, Elastix: A toolbox for rigid and nonrigid registration of images, https://elastix.lumc.nl/.
[30]	S. Klein, M. Staring, K. Murphy, M. A. Viergever and J. P. Pluim, Elastix: A toolbox for intensity-based medical image registration, IEEE Transactions on Medical Imaging, 29 (2010), 196-205. doi: 10.1109/TMI.2009.2035616.
[31]	D.-J. Kroon, Multimodality non-rigid demon algorithm image registration, MATLAB Central File Exchange, https://www.mathworks.com/matlabcentral/fileexchange/21451-multimodality-non-rigid-demon-algorithm-image-registration.
[32]	D. Kuang, Cycle-consistent training for reducing negative jacobian determinant in deep registration networks, In International Workshop on Simulation and Synthesis in Medical Imaging, Springer, 11827 (2019), 120–129. doi: 10.1007/978-3-030-32778-1_13.
[33]	D. Kuang and T. Schmah, FAIM–a ConvNet method for unsupervised 3D medical image registration, In International Workshop on Machine Learning in Medical Imaging, Springer, 11861 (2019), 646–654. doi: 10.1007/978-3-030-32692-0_74.
[34]	K. C. Lam and L. M. Lui, Landmark- and intensity-based registration with large deformations via quasi-conformal maps, SIAM J. Imaging Sci., 7 (2014), 2364-2392. doi: 10.1137/130943406.
[35]	Y. T. Lee, K. C. Lam and L. M. Lui, Landmark-matching transformation with large deformation via n-dimensional quasi-conformal maps, J. Sci. Comput., 67 (2016), 926-954. doi: 10.1007/s10915-015-0113-5.
[36]	O. Lehto and K. I. Virtanen, Quasiconformal Mappings in the Plane, vol. 126, 2$^{nd}$ edition, Springer-Verlag Berlin Heidelberg, 1973.
[37]	H. Lombaert, Diffeomorphic log demons image registration, MATLAB Central File Exchange, https://www.mathworks.com/matlabcentral/fileexchange/39194-diffeomorphic-log-demons-image-registration.
[38]	J. B. A. Maintz and M. A. Viergever, A survey of medical image registration, Medical Image Analysis, 2 (1998), 1-36. doi: 10.1016/S1361-8415(01)80026-8.
[39]	T. W. Meng, G. P.-T. Choi and L. M. Lui, TEMPO: Feature-endowed Teichmüller extremal mappings of point clouds, SIAM J. Imaging Sci., 9 (2016), 1922-1962. doi: 10.1137/15M1049117.
[40]	J. Modersitzki, FAIR: Flexible Algorithms for Image Registration, SIAM, 2009. doi: 10.1137/1.9780898718843.
[41]	I. Rocco, R. Arandjelovic and J. Sivic, Convolutional neural network architecture for geometric matching, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2017), 6148–6157. doi: 10.1109/CVPR.2017.12.
[42]	E. Simo-Serra, E. Trulls, L. Ferraz, I. Kokkinos, P. Fua and F. Moreno-Noguer, Discriminative learning of deep convolutional feature point descriptors, In Proceedings of the IEEE International Conference on Computer Vision, (2015), 118–126. doi: 10.1109/ICCV.2015.22.
[43]	K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, Preprint, arXiv: 1409.1556.
[44]	S. Sommer, Segframe image registration, https://github.com/nefan/segframe.
[45]	A. Sotiras, C. Davatzikos and N. Paragios, Deformable medical image registration: A survey, IEEE Transactions on Medical Imaging, 32 (2013), 1153-1190. doi: 10.1109/TMI.2013.2265603.
[46]	J.-P. Thirion, Image matching as a diffusion process: An analogy with Maxwell's demons, Medical Image Analysis, 2 (1998), 243-260. doi: 10.1016/S1361-8415(98)80022-4.
[47]	T. Vercauteren, X. Pennec, A. Perchant and N. Ayache, Diffeomorphic demons: Efficient non-parametric image registration, NeuroImage, 45 (2009), 61-72. doi: 10.1016/j.neuroimage.2008.10.040.
[48]	H. Wang, L. Dong, J. O'Daniel, R. Mohan, A. S. Garden, K. K. Ang, D. A. Kuban, M. Bonnen, J. Y. Chang and R. Cheung, Validation of an accelerated 'Demons' algorithm for deformable image registration in radiation therapy, Physics in Medicine & Biology, 50 (2005), 2887. doi: 10.1088/0031-9155/50/12/011.
[49]	X. Yang, R. Kwitt, M. Styner and M. Niethammer, Quicksilver: Fast predictive image registration–a deep learning approach, NeuroImage, 158 (2017), 378-396. doi: 10.1016/j.neuroimage.2017.07.008.
[50]	C. P. Yung, G. P. T. Choi, K. Chen and L. M. Lui, Efficient feature-based image registration by mapping sparsified surfaces, Journal of Visual Communication and Image Representation, 55 (2018), 561-571. doi: 10.1016/j.jvcir.2018.07.005.
[51]	S. Zagoruyko and N. Komodakis, Learning to compare image patches via convolutional neural networks, In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2015), 4353–4361. doi: 10.1109/CVPR.2015.7299064.
[52]	B. Zitova and J. Flusser, Image registration methods: A survey, Image and Vision Computing, 21 (2003), 977-1000. doi: 10.1016/S0262-8856(03)00137-9.

Figure 1. An illustration of how the Beltrami coefficient $ \mu $ determines the conformality distortion. Under a quasiconformal map $ f $, an infinitesimal circle around a point $ p $ is mapped to an infinitesimal ellipse centered at $ f(p) $, where the major axis length and the minor axis length are given by $ |f_z(p)|(1+|\mu(p)|) $ and $ |f_z(p)|(1-|\mu(p)|) $, respectively. Therefore, the maximal dilation of $ f $ is $ K(f) = \frac{1+||\mu||_{\infty}}{1-||\mu||_{\infty}} $. Also, the orientation change of the major axis of the ellipse is given by $ \arg(\mu(p))/2 $

Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2. An illustration of receptive field

Figure Options

Download full-size image

Download as PowerPoint slide

Figure 3. The process of obtaining feature vectors from the images. The two images on the left are the moving image and the fixed image. We first partition both images into smaller patches, and then feed each patch into the truncated classification network to obtain a 3D array of size $ m \times n \times d $, where $ m, n $ are the number of receptive fields along the width and height of the input image depending on stride, kernel and padding size, and $ d $ is the dimension of the feature vector depending on the architecture of the network. We can then transform this 3D array into a vector by vertically stacking along one direction. Here we partition each image into $ 3\times 3 = 9 $ patches for illustrative purposes, and hence 9 vectors in $ \mathbb{R}^{mnd} $ are produced for each of the two images as shown on the right. In practice, a finer partition is often used

Figure Options

Download full-size image

Download as PowerPoint slide

Figure 4. An illustration of the multiresolution Scheme. We first coarsen both input source and target images as shown in Fig. 4a and Fig. 4b. Then, we run our proposed algorithm on this coarsen pair. From the obtained mapping on this coarsest level, with linear interpolation, we warp the source image on second level to yield Fig. 4c and register it against Fig. 4d using our proposed algorithm. Finally, with the mapping from the last level, we interpolate it to warp the source image on the finest level as shown in Fig. 4e and register it against Fig. 4f. Refer to Fig. 6 for the final registration result

Figure Options

Download full-size image

Download as PowerPoint slide

Figure 5. The 'Z' to '2' example

Figure Options

Download full-size image

Download as PowerPoint slide

Figure 6. The eagle example

Figure Options

Download full-size image

Download as PowerPoint slide

Figure 7. The rabbit example

Figure Options

Download full-size image

Download as PowerPoint slide

Figure 8. The first hand X-ray example

Figure Options

Download full-size image

Download as PowerPoint slide

Figure 9. The second hand X-ray example

Figure Options

Download full-size image

Download as PowerPoint slide

Figure 10. The lung CT example

Figure Options

Download full-size image

Download as PowerPoint slide

Figure 11. The chest CT example

Figure Options

Download full-size image

Download as PowerPoint slide

Table 1. The performance of different image registration methods for various synthetic and real medical images. Here, $ E_{\text{sim}} $ measures the accuracy of the registration mapping as described in Equation (37), $ E_{\text{smooth}} $ measures the smoothness of the mapping as described in Equation (38), $ E_{\text{total}} $ measures the overall quality of the mapping as described in Equation (39), and the number of flips reflects the bijectivity of the mapping. For each example and each measure, the best entry among all methods is in bold

${{bf Method}}$	Results ($E_{\text{sim}}$/$E_{\text{smooth}}$/$E_{\text{total}}$/#Flips)
${{bf Method}}$	'Z' to '2' (Fig. 5)	Eagle (Fig. 6)	Rabbit (Fig. 7)	Hand 1 (Fig. 8)	Hand 2 (Fig. 9)	Lung (Fig. 10)	Chest (Fig. 11)
Our method	0.3099 0.4476 0.7575 0	0.0916 0.1558 0.2474 0	0.1953 0.1436 0.3389 0	0.1417 0.1075 0.2492 0	0.1317 0.1536 0.2853 0	0.2435 0.4774 0.7210 0	0.0368 0.1414 0.1783 0
DDemons^[47]	2.0660 0.0481 2.1141 121	0.3724 0.2470 0.6194 3697	0.1806 0.1278 0.3083 174	0.4528 0.2651 0.7180 6036	0.4273 0.2718 0.6991 8925	0.6332 0.2271 0.8602 4191	0.0725 0.3565 0.4290 16316
LDDMM^[5]	2.0488 0.2941 2.3629 105	0.3570 0.2388 0.5968 26	0.5973 0.0738 0.6711 1	0.7018 0.1842 0.8860 0	0.9563 0.1902 1.1465 6	0.6843 0.2568 0.9410 209	0.2446 0.0717 0.3164 0
Elastix^[30]	3.6377 0.9185 4.5562 68280	0.1324 0.1779 0.3103 1158	0.1808 0.1679 0.3487 0	0.1103 0.1513 0.2617 0	0.1856 0.1406 0.3263 0	0.3432 0.3203 0.6635 3579	0.0530 0.1408 0.1938 0
DROP^[19]	2.0554 0.3084 2.3638 2515	0.2509 0.1518 0.4027 0	0.1788 0.1634 0.3423 0	0.3021 0.1186 0.4207 0	1.2514 0.1406 1.3920 0	0.5065 0.6193 1.1259 3663	0.2415 0.1841 0.4256 0

Table Options

Download as excel

${{bf Method}}$	Results ($E_{\text{sim}}$/$E_{\text{smooth}}$/$E_{\text{total}}$/#Flips)
${{bf Method}}$	'Z' to '2' (Fig. 5)	Eagle (Fig. 6)	Rabbit (Fig. 7)	Hand 1 (Fig. 8)	Hand 2 (Fig. 9)	Lung (Fig. 10)	Chest (Fig. 11)
Our method	0.3099 0.4476 0.7575 0	0.0916 0.1558 0.2474 0	0.1953 0.1436 0.3389 0	0.1417 0.1075 0.2492 0	0.1317 0.1536 0.2853 0	0.2435 0.4774 0.7210 0	0.0368 0.1414 0.1783 0
DDemons^[47]	2.0660 0.0481 2.1141 121	0.3724 0.2470 0.6194 3697	0.1806 0.1278 0.3083 174	0.4528 0.2651 0.7180 6036	0.4273 0.2718 0.6991 8925	0.6332 0.2271 0.8602 4191	0.0725 0.3565 0.4290 16316
LDDMM^[5]	2.0488 0.2941 2.3629 105	0.3570 0.2388 0.5968 26	0.5973 0.0738 0.6711 1	0.7018 0.1842 0.8860 0	0.9563 0.1902 1.1465 6	0.6843 0.2568 0.9410 209	0.2446 0.0717 0.3164 0
Elastix^[30]	3.6377 0.9185 4.5562 68280	0.1324 0.1779 0.3103 1158	0.1808 0.1679 0.3487 0	0.1103 0.1513 0.2617 0	0.1856 0.1406 0.3263 0	0.3432 0.3203 0.6635 3579	0.0530 0.1408 0.1938 0
DROP^[19]	2.0554 0.3084 2.3638 2515	0.2509 0.1518 0.4027 0	0.1788 0.1634 0.3423 0	0.3021 0.1186 0.4207 0	1.2514 0.1406 1.3920 0	0.5065 0.6193 1.1259 3663	0.2415 0.1841 0.4256 0

[1]	Zhuwei Qin, Fuxun Yu, Chenchen Liu, Xiang Chen. How convolutional neural networks see the world --- A survey of convolutional neural network visualization methods. Mathematical Foundations of Computing, 2018, 1 (2) : 149-180. doi: 10.3934/mfc.2018008
[2]	Yuantian Xia, Juxiang Zhou, Tianwei Xu, Wei Gao. An improved deep convolutional neural network model with kernel loss function in image classification. Mathematical Foundations of Computing, 2020, 3 (1) : 51-64. doi: 10.3934/mfc.2020005
[3]	Dana Paquin, Doron Levy, Eduard Schreibmann, Lei Xing. Multiscale Image Registration. Mathematical Biosciences & Engineering, 2006, 3 (2) : 389-418. doi: 10.3934/mbe.2006.3.389
[4]	Zbigniew Gomolka, Boguslaw Twarog, Jacek Bartman. Improvement of image processing by using homogeneous neural networks with fractional derivatives theorem. Conference Publications, 2011, 2011 (Special) : 505-514. doi: 10.3934/proc.2011.2011.505
[5]	Zhao Yi, Justin W. L. Wan. An inviscid model for nonrigid image registration. Inverse Problems and Imaging, 2011, 5 (1) : 263-284. doi: 10.3934/ipi.2011.5.263
[6]	Dana Paquin, Doron Levy, Lei Xing. Hybrid multiscale landmark and deformable image registration. Mathematical Biosciences & Engineering, 2007, 4 (4) : 711-737. doi: 10.3934/mbe.2007.4.711
[7]	Christiane Pöschl, Jan Modersitzki, Otmar Scherzer. A variational setting for volume constrained image registration. Inverse Problems and Imaging, 2010, 4 (3) : 505-522. doi: 10.3934/ipi.2010.4.505
[8]	Seonho Park, Maciej Rysz, Kaitlin L. Fair, Panos M. Pardalos. Synthetic-Aperture Radar image based positioning in GPS-denied environments using Deep Cosine Similarity Neural Networks. Inverse Problems and Imaging, 2021, 15 (4) : 763-785. doi: 10.3934/ipi.2021013
[9]	Ying Sue Huang. Resynchronization of delayed neural networks. Discrete and Continuous Dynamical Systems, 2001, 7 (2) : 397-401. doi: 10.3934/dcds.2001.7.397
[10]	Marcus Wagner. A direct method for the solution of an optimal control problem arising from image registration. Numerical Algebra, Control and Optimization, 2012, 2 (3) : 487-510. doi: 10.3934/naco.2012.2.487
[11]	Angel Angelov, Marcus Wagner. Multimodal image registration by elastic matching of edge sketches via optimal control. Journal of Industrial and Management Optimization, 2014, 10 (2) : 567-590. doi: 10.3934/jimo.2014.10.567
[12]	Yangang Chen, Justin W. L. Wan. Numerical method for image registration model based on optimal mass transport. Inverse Problems and Imaging, 2018, 12 (2) : 401-432. doi: 10.3934/ipi.2018018
[13]	Tatyana S. Turova. Structural phase transitions in neural networks. Mathematical Biosciences & Engineering, 2014, 11 (1) : 139-148. doi: 10.3934/mbe.2014.11.139
[14]	Kateřina Škardová, Tomáš Oberhuber, Jaroslav Tintěra, Radomír Chabiniok. Signed-distance function based non-rigid registration of image series with varying image intensity. Discrete and Continuous Dynamical Systems - S, 2021, 14 (3) : 1145-1160. doi: 10.3934/dcdss.2020386
[15]	Editorial Office. Retraction: Honggang Yu, An efficient face recognition algorithm using the improved convolutional neural network. Discrete and Continuous Dynamical Systems - S, 2019, 12 (4&5) : 901-901. doi: 10.3934/dcdss.2019060
[16]	Benedict Leimkuhler, Charles Matthews, Tiffany Vlaar. Partitioned integrators for thermodynamic parameterization of neural networks. Foundations of Data Science, 2019, 1 (4) : 457-489. doi: 10.3934/fods.2019019
[17]	Ricai Luo, Honglei Xu, Wu-Sheng Wang, Jie Sun, Wei Xu. A weak condition for global stability of delayed neural networks. Journal of Industrial and Management Optimization, 2016, 12 (2) : 505-514. doi: 10.3934/jimo.2016.12.505
[18]	Benedetta Lisena. Average criteria for periodic neural networks with delay. Discrete and Continuous Dynamical Systems - B, 2014, 19 (3) : 761-773. doi: 10.3934/dcdsb.2014.19.761
[19]	Larry Turyn. Cellular neural networks: asymmetric templates and spatial chaos. Conference Publications, 2003, 2003 (Special) : 864-871. doi: 10.3934/proc.2003.2003.864
[20]	Lars Grüne. Computing Lyapunov functions using deep neural networks. Journal of Computational Dynamics, 2021, 8 (2) : 131-152. doi: 10.3934/jcd.2021006

2021 Impact Factor: 1.483

Tools

Metrics

Tools

Article outline

Figures and Tables

2021 Impact Factor: 1.483

Tools

Figures and Tables

2021 Impact Factor: 1.483

American Institute of Mathematical Sciences

Quasiconformal model with CNN features for large deformation image registration

References:

References:

Tools

Metrics

Other articles
by authors

Tools

Tools

Tools

Metrics

Other articles
by authors

American Institute of Mathematical Sciences

Quasiconformal model with CNN features for large deformation image registration

References:

References:

Tools

Metrics

Other articlesby authors

Tools

Tools

Tools

Metrics

Other articlesby authors

Other articles
by authors

Other articles
by authors