Figure 1.
The Architecture of PDR U-Net. f represents the number of filters.d represents the dilated rate.The keep rate of dropout is 0.7
-
Previous Article
A stable non-iterative reconstruction algorithm for the acoustic inverse boundary value problem
- IPI Home
- This Issue
-
Next Article
Near-field imaging for an obstacle above rough surfaces with limited aperture data
doi: 10.3934/ipi.2020057
Automatic segmentation of the femur and tibia bones from X-ray images based on pure dilated residual U-Net
| 1. | School of Artificial Intelligence, Hebei University of Technology, Tianjin 300401, China |
| 2. | Tianjin Institute of Orthopaedics, Tianjin Hospital, Tianjin University, Tianjin 300211, China |
| 3. | Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China |
| 4. | School of Artificial Intelligence, Hebei Key Laboratory of Robot Perception and Human-Robot Interaction, Hebei University of Technology, Tianjin 300401, China |
X-ray images of the lower limb bone are the most commonly used imaging modality for clinical studies, and segmentation of the femur and tibia in an X-ray image is helpful for many medical studies such as diagnosis, surgery and treatment. In this paper, we propose a new approach based on pure dilated residual U-Net for the segmentation of the femur and tibia bones. The proposed approach employs dilated convolution completely to increase the receptive field, in this way, we can make full use of the advantages of dilated convolution. We conducted experiments and evaluations on datasets provided by Tianjin hospital. Comparison with the classical U-net and FusionNet, our method has fewer parameters, higher accuracy, and converges more rapidly, which means the high performance of the proposed method.
Mathematics Subject Classification:
Primary: 68T07; Secondary: 68T20.
Citation:
Weihao Shen, Wenbo Xu, Hongyang Zhang, Zexin Sun, Jianxiong Ma, Xinlong Ma, Shoujun Zhou, Shijie Guo, Yuanquan Wang.
Automatic segmentation of the femur and tibia bones from X-ray images based on pure dilated residual U-Net. Inverse Problems & Imaging, doi: 10.3934/ipi.2020057
![]()
References:
| [1] |
S. Y. Ababneh, J. W. Prescott and M. N. Gurcan,
Automatic graph-cut based segmentation of bones from knee magnetic resonance images for osteoarthritis research, Medical Image Anal., 15 (2011), 438-448.
doi: 10.1016/j.media.2011.01.007. |
| [2] |
V. Badrinarayanan, A. Kendall and R. Cipolla, Segnet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE Trans. Pattern Anal. Mach. Intell., 39 (2017), 2481–2495.
doi: 10.1109/TPAMI.2016.2644615. |
| [3] |
O. Bandyopadhyay, A. Biswas and B. B. Bhattacharya,
Long-bone fracture detection in digital x-ray images based on digital-geometric techniques, Comput. Methods Programs Biomed., 123 (2016), 2-14.
doi: 10.1016/j.cmpb.2015.09.013. |
| [4] |
J. Carballido-Gamio, et al., Automatic multi-parametric quantification of the proximal femur with quantitative computed tomography, Quantitative Imaging in Medicine and Surgery, 5 (2015), 552-568. Google Scholar |
| [5] |
L. Chen, G. Papandreou, I. Kokkinos, K. Murphy and A. L. Yuille, Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs, IEEE Trans. Pattern Anal. Mach. Intell., 40 (2018), 834–848.
doi: 10.1109/TPAMI.2017.2699184. |
| [6] |
Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox and O. Ronneberger, 3d u-net: Learning dense volumetric segmentation from sparse annotation, in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II, Lecture Notes in Computer Science, 9901, 2016,424–432.
doi: 10.1007/978-3-319-46723-8_49. |
| [7] |
C. M. Deniz, S. Hallyburton, A. Welbeck, S. Honig, K. Cho and G. Chang, Segmentation of the proximal femur from MR images using deep convolutional neural networks, Sci. Rep., 8 (2018), 16485.
doi: 10.1038/s41598-018-34817-6. |
| [8] |
F. Ding, W. K. Leow and T. S. Howe, Automatic segmentation of femur bones in anterior-posterior pelvis x-ray images, in Computer Analysis of Images and Patterns, 12th International Conference, CAIP 2007, Vienna, Austria, August 27-29, 2007, Proceedings (eds. W. G. Kropatsch, M. Kampel and A. Hanbury), Lecture Notes in Computer Science, 4673, Springer, 2007,205–212.
doi: 10.1007/978-3-540-74272-2_26. |
| [9] |
L.-H. Fan, J.-G. Han, Y. Jia, C. Zhao and B. Yang, Segmentation of femurs in x-ray image with generative adversarial networks, DEStech Transactions on Engineering and Technology Research, 289–295.
doi: 10.12783/dtetr/ecae2018/27745. |
| [10] |
I. J. Goodfellow, et al., Generative adversarial nets, in Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada (eds. Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence and K. Q. Weinberger), 2014, 2672–2680 Google Scholar |
| [11] |
S. Guan, A. A. Khan, S. Sikdar and P. V. Chitnis, Fully dense unet for 2d sparse photoacoustic tomography artifact removal, preprint, arXiv: 1808.10848.
doi: 10.1109/JBHI.2019.2912935. |
| [12] |
I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin and A. C. Courville, Improved training of wasserstein gans, in Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA (eds. I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan and R. Garnett), 2017, 5767–5777 Google Scholar |
| [13] |
K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, in 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016
doi: 10.1109/CVPR.2016.90. |
| [14] |
G. Huang, Z. Liu, L. van der Maaten and K. Q. Weinberger, Densely connected convolutional networks, in 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017
doi: 10.1109/CVPR.2017.243. |
| [15] |
R. Jiang, J. Meng and P. Babyn, X-ray image segmentation using active contour model with global constraints, 2007 IEEE Symposium on Computational Intelligence in Image and Signal Processing, (2007), 240–245. Google Scholar |
| [16] |
A. Krizhevsky, I. Sutskever and G. E. Hinton, Imagenet classification with deep convolutional neural networks, in Advances in Neural Information Processing Systems, 2012
doi: 10.1145/3065386. |
| [17] |
H. Li, A. Zhygallo and B. H. Menze, Automatic brain structures segmentation using deep residual dilated u-net, in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part I (eds. A. Crimi, S. Bakas, H. J. Kuijf, F. Keyvan, M. Reyes and T. van Walsum), Lecture Notes in Computer Science, 11383, Springer, 2018,385–393.
doi: 10.1007/978-3-030-11723-8_39. |
| [18] |
M. Lin, Q. Chen and S. Yan, Network in network, in 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings (eds. Y. Bengio and Y. LeCun), preprint Google Scholar |
| [19] |
M. Liu, T. Breuel and J. Kautz, Unsupervised image-to-image translation networks, in Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA (eds. I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan and R. Garnett), 2017,700–708 Google Scholar |
| [20] |
X. Liu, et al., Msdf-net: Multi-scale deep fusion network for stroke lesion segmentation, IEEE Access, 7 (2019), 178486–178495.
doi: 10.1109/ACCESS.2019.2958384. |
| [21] |
J. Long, E. Shelhamer and T. Darrell, Fully convolutional networks for semantic segmentation, in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015
doi: 10.1109/CVPR.2015.7298965. |
| [22] |
X. Mao, Q. Li, H. Xie, R. Y. K. Lau, Z. Wang and S. P. Smolley, Least squares generative adversarial networks, in IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017
doi: 10.1109/ICCV.2017.304. |
| [23] |
F. Milletari, N. Navab and S. Ahmadi, V-net: Fully convolutional neural networks for volumetric medical image segmentation, in Fourth International Conference on 3D Vision, 3DV 2016, Stanford, CA, USA, October 25-28, 2016
doi: 10.1109/3DV.2016.79. |
| [24] |
O. Oktay, et al., Attention u-net: Learning where to look for the pancreas, preprint, arXiv: 1804.03999. Google Scholar |
| [25] |
C. N. Öztürk and S. Albayrak,
Automatic segmentation of cartilage in high-field magnetic resonance images of the knee joint with an improved voxel-classification-driven region-growing algorithm using vicinity-correlated subsampling, Comp. Bio. Med., 72 (2016), 90-107.
doi: 10.1016/j.compbiomed.2016.03.011. |
| [26] |
T. T. Peng, et al., Detection of femur fractures in x-ray images, Master of Science Thesis, National University of Singapore. Google Scholar |
| [27] |
A. Pries, P. J. Schreier, A. Lamm, S. Pede and J. Schmidt, Deep morphing: Detecting bone structures in fluoroscopic x-ray images with prior knowledge, preprint, arXiv: 1808.04441. Google Scholar |
| [28] |
T. M. Quan, D. G. C. Hildebrand and W. Jeong, Fusionnet: A deep fully residual convolutional neural network for image segmentation in connectomics, preprint, arXiv: 1612.05360. Google Scholar |
| [29] |
O. Ronneberger, P. Fischer and T. Brox, U-net: Convolutional networks for biomedical image segmentation, in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015-18th International Conference Munich, Germany, October 5-9, 2015, Proceedings, Part III (eds. N. Navab, J. Hornegger, W. M. W. III and A. F. Frangi), Lecture Notes in Computer Science, 9351, Springer, 2015,234–241.
doi: 10.1007/978-3-319-24574-4_28. |
| [30] |
T. Salimans, I. J. Goodfellow, W. Zaremba, V. Cheung, A. Radford and X. Chen, Improved techniques for training gans, in Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain (eds. D. D. Lee, M. Sugiyama, U. von Luxburg, I. Guyon and R. Garnett), 2016, 2226–2234 Google Scholar |
| [31] |
P. Santhoshini, R. Tamilselvi and R. Sivakumar, Automatic segmentation of femur bone features and analysis of osteoporosis, Lecture Notes on Software Engineering, 194–198.
doi: 10.7763/LNSE.2013.V1.44. |
| [32] |
K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings (eds. Y. Bengio and Y. LeCun), preprint Google Scholar |
| [33] |
R. Smith, Segmentation and fracture detection in x-ray images for traumatic pelvic injury., Google Scholar |
| [34] |
C. Stolojescu-Crisan and S. Holban, An interactive x-ray image segmentation technique for bone extraction, in International Work-Conference on Bioinformatics and Biomedical Engineering, IWBBIO 2014, Granada, Spain, April 7-9, 2014 (eds. I. Rojas and F. M. O. Guzman), Copicentro Editorial, 2014, 1164–1171 Google Scholar |
| [35] |
H. Sun, et al., Aunet: Attention-guided dense-upsampling networks for breast mass segmentation in whole mammograms, Phys. Med. Biol., 65 (2020), 055005.
doi: 10.1088/1361-6560/ab5745. |
| [36] |
C. Szegedy, et al., Going deeper with convolutions, in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015
doi: 10.1109/CVPR.2015.7298594. |
| [37] |
A. Tack, A. Mukhopadhyay and S. Zachow,
Knee menisci segmentation using convolutional neural networks: Data from the osteoarthritis initiative, Osteoarthritis and Cartilage, 26 (2018), 680-688.
doi: 10.1016/j.joca.2018.02.907. |
| [38] |
W. Wang, Y. Wang, Y. Wu, T. Lin, S. Li and B. Chen,
Quantification of full left ventricular metrics via deep regression learning with contour-guidance, IEEE Access, 7 (2019), 47918-47928.
doi: 10.1109/ACCESS.2019.2907564. |
| [39] |
J. Wu, A. Belle, R. H. Hargraves, C. Cockrell, Y. Tang and K. Najarian,
Bone segmentation and 3d visualization of CT images for traumatic pelvic injuries, Int. J. Imaging Syst. Technol., 24 (2014), 29-38.
doi: 10.1002/ima.22076. |
| [40] |
X. Xiao, S. Lian, Z. Luo and S. Li, Weighted res-unet for high-quality retina vessel segmentation, in 2018 9th International Conference on Information Technology in Medicine and Education (ITME), 2018,327–331. Google Scholar |
| [41] |
Y. Xue, T. Xu, H. Zhang, L. R. Long and X. Huang,
Segan: Adversarial network with multi-scale L 1 loss for medical image segmentation, Neuroinformatics, 16 (2018), 383-392.
doi: 10.1007/s12021-018-9377-x. |
| [42] |
F. Yokota, T. Okada, M. Takao, N. Sugano, Y. Tada and Y. Sato, Automated segmentation of the femur and pelvis from 3d CT data of diseased hip using hierarchical statistical shape model of joint structure, in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2009, 12th International Conference, London, UK, September 20-24, 2009, Proceedings, Part II (eds. G. Yang, D. J. Hawkes, D. Rueckert, J. A. Noble and C. J. Taylor), Lecture Notes in Computer Science, 5762, Springer, 2009,811–818.
doi: 10.1007/978-3-642-04271-3_98. |
| [43] |
F. Yu and V. Koltun, Multi-scale context aggregation by dilated convolutions, in 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings (eds. Y. Bengio and Y. LeCun), preprint Google Scholar |
| [44] |
K. Zhang, W. Lu and P. Marziliano,
Automatic knee cartilage segmentation from multi-contrast mr images using support vector machine classification with spatial dependencies, Magnetic Resonance Imaging, 31 (2013), 1731-1743.
doi: 10.1016/j.mri.2013.06.005. |
| [45] |
Z. Zhang, C. Duan, T. Lin, S. Zhou, Y. Wang and X. Gao,
GVFOM: A novel external force for active contour based image segmentation, Inf. Sci., 506 (2020), 1-18.
doi: 10.1016/j.ins.2019.08.003. |
| [46] |
Y. Zhou, W. Huang, P. Dong, Y. Xia and S. Wang, D-unet: A dimension-fusion U shape network for chronic stroke lesion segmentation, preprint, arXiv: 1908.05104.
doi: 10.1109/TCBB.2019.2939522. |
| [47] |
J. Zhu, T. Park, P. Isola and A. A. Efros, Unpaired image-to-image translation using cycle-consistent adversarial networks, in IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017
doi: 10.1109/ICCV.2017.244. |
| [48] |
Keras: Deep learning library for theano and tensorflow, https://github.com/keras-team/keras, 2015. Google Scholar |
| [49] |
Lableme, http://labelme.csail.mit.edu/Release3.0/. Google Scholar |
show all references
References:
| [1] |
S. Y. Ababneh, J. W. Prescott and M. N. Gurcan,
Automatic graph-cut based segmentation of bones from knee magnetic resonance images for osteoarthritis research, Medical Image Anal., 15 (2011), 438-448.
doi: 10.1016/j.media.2011.01.007. |
| [2] |
V. Badrinarayanan, A. Kendall and R. Cipolla, Segnet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE Trans. Pattern Anal. Mach. Intell., 39 (2017), 2481–2495.
doi: 10.1109/TPAMI.2016.2644615. |
| [3] |
O. Bandyopadhyay, A. Biswas and B. B. Bhattacharya,
Long-bone fracture detection in digital x-ray images based on digital-geometric techniques, Comput. Methods Programs Biomed., 123 (2016), 2-14.
doi: 10.1016/j.cmpb.2015.09.013. |
| [4] |
J. Carballido-Gamio, et al., Automatic multi-parametric quantification of the proximal femur with quantitative computed tomography, Quantitative Imaging in Medicine and Surgery, 5 (2015), 552-568. Google Scholar |
| [5] |
L. Chen, G. Papandreou, I. Kokkinos, K. Murphy and A. L. Yuille, Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs, IEEE Trans. Pattern Anal. Mach. Intell., 40 (2018), 834–848.
doi: 10.1109/TPAMI.2017.2699184. |
| [6] |
Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox and O. Ronneberger, 3d u-net: Learning dense volumetric segmentation from sparse annotation, in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II, Lecture Notes in Computer Science, 9901, 2016,424–432.
doi: 10.1007/978-3-319-46723-8_49. |
| [7] |
C. M. Deniz, S. Hallyburton, A. Welbeck, S. Honig, K. Cho and G. Chang, Segmentation of the proximal femur from MR images using deep convolutional neural networks, Sci. Rep., 8 (2018), 16485.
doi: 10.1038/s41598-018-34817-6. |
| [8] |
F. Ding, W. K. Leow and T. S. Howe, Automatic segmentation of femur bones in anterior-posterior pelvis x-ray images, in Computer Analysis of Images and Patterns, 12th International Conference, CAIP 2007, Vienna, Austria, August 27-29, 2007, Proceedings (eds. W. G. Kropatsch, M. Kampel and A. Hanbury), Lecture Notes in Computer Science, 4673, Springer, 2007,205–212.
doi: 10.1007/978-3-540-74272-2_26. |
| [9] |
L.-H. Fan, J.-G. Han, Y. Jia, C. Zhao and B. Yang, Segmentation of femurs in x-ray image with generative adversarial networks, DEStech Transactions on Engineering and Technology Research, 289–295.
doi: 10.12783/dtetr/ecae2018/27745. |
| [10] |
I. J. Goodfellow, et al., Generative adversarial nets, in Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada (eds. Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence and K. Q. Weinberger), 2014, 2672–2680 Google Scholar |
| [11] |
S. Guan, A. A. Khan, S. Sikdar and P. V. Chitnis, Fully dense unet for 2d sparse photoacoustic tomography artifact removal, preprint, arXiv: 1808.10848.
doi: 10.1109/JBHI.2019.2912935. |
| [12] |
I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin and A. C. Courville, Improved training of wasserstein gans, in Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA (eds. I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan and R. Garnett), 2017, 5767–5777 Google Scholar |
| [13] |
K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, in 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016
doi: 10.1109/CVPR.2016.90. |
| [14] |
G. Huang, Z. Liu, L. van der Maaten and K. Q. Weinberger, Densely connected convolutional networks, in 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017
doi: 10.1109/CVPR.2017.243. |
| [15] |
R. Jiang, J. Meng and P. Babyn, X-ray image segmentation using active contour model with global constraints, 2007 IEEE Symposium on Computational Intelligence in Image and Signal Processing, (2007), 240–245. Google Scholar |
| [16] |
A. Krizhevsky, I. Sutskever and G. E. Hinton, Imagenet classification with deep convolutional neural networks, in Advances in Neural Information Processing Systems, 2012
doi: 10.1145/3065386. |
| [17] |
H. Li, A. Zhygallo and B. H. Menze, Automatic brain structures segmentation using deep residual dilated u-net, in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part I (eds. A. Crimi, S. Bakas, H. J. Kuijf, F. Keyvan, M. Reyes and T. van Walsum), Lecture Notes in Computer Science, 11383, Springer, 2018,385–393.
doi: 10.1007/978-3-030-11723-8_39. |
| [18] |
M. Lin, Q. Chen and S. Yan, Network in network, in 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings (eds. Y. Bengio and Y. LeCun), preprint Google Scholar |
| [19] |
M. Liu, T. Breuel and J. Kautz, Unsupervised image-to-image translation networks, in Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA (eds. I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan and R. Garnett), 2017,700–708 Google Scholar |
| [20] |
X. Liu, et al., Msdf-net: Multi-scale deep fusion network for stroke lesion segmentation, IEEE Access, 7 (2019), 178486–178495.
doi: 10.1109/ACCESS.2019.2958384. |
| [21] |
J. Long, E. Shelhamer and T. Darrell, Fully convolutional networks for semantic segmentation, in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015
doi: 10.1109/CVPR.2015.7298965. |
| [22] |
X. Mao, Q. Li, H. Xie, R. Y. K. Lau, Z. Wang and S. P. Smolley, Least squares generative adversarial networks, in IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017
doi: 10.1109/ICCV.2017.304. |
| [23] |
F. Milletari, N. Navab and S. Ahmadi, V-net: Fully convolutional neural networks for volumetric medical image segmentation, in Fourth International Conference on 3D Vision, 3DV 2016, Stanford, CA, USA, October 25-28, 2016
doi: 10.1109/3DV.2016.79. |
| [24] |
O. Oktay, et al., Attention u-net: Learning where to look for the pancreas, preprint, arXiv: 1804.03999. Google Scholar |
| [25] |
C. N. Öztürk and S. Albayrak,
Automatic segmentation of cartilage in high-field magnetic resonance images of the knee joint with an improved voxel-classification-driven region-growing algorithm using vicinity-correlated subsampling, Comp. Bio. Med., 72 (2016), 90-107.
doi: 10.1016/j.compbiomed.2016.03.011. |
| [26] |
T. T. Peng, et al., Detection of femur fractures in x-ray images, Master of Science Thesis, National University of Singapore. Google Scholar |
| [27] |
A. Pries, P. J. Schreier, A. Lamm, S. Pede and J. Schmidt, Deep morphing: Detecting bone structures in fluoroscopic x-ray images with prior knowledge, preprint, arXiv: 1808.04441. Google Scholar |
| [28] |
T. M. Quan, D. G. C. Hildebrand and W. Jeong, Fusionnet: A deep fully residual convolutional neural network for image segmentation in connectomics, preprint, arXiv: 1612.05360. Google Scholar |
| [29] |
O. Ronneberger, P. Fischer and T. Brox, U-net: Convolutional networks for biomedical image segmentation, in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015-18th International Conference Munich, Germany, October 5-9, 2015, Proceedings, Part III (eds. N. Navab, J. Hornegger, W. M. W. III and A. F. Frangi), Lecture Notes in Computer Science, 9351, Springer, 2015,234–241.
doi: 10.1007/978-3-319-24574-4_28. |
| [30] |
T. Salimans, I. J. Goodfellow, W. Zaremba, V. Cheung, A. Radford and X. Chen, Improved techniques for training gans, in Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain (eds. D. D. Lee, M. Sugiyama, U. von Luxburg, I. Guyon and R. Garnett), 2016, 2226–2234 Google Scholar |
| [31] |
P. Santhoshini, R. Tamilselvi and R. Sivakumar, Automatic segmentation of femur bone features and analysis of osteoporosis, Lecture Notes on Software Engineering, 194–198.
doi: 10.7763/LNSE.2013.V1.44. |
| [32] |
K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings (eds. Y. Bengio and Y. LeCun), preprint Google Scholar |
| [33] |
R. Smith, Segmentation and fracture detection in x-ray images for traumatic pelvic injury., Google Scholar |
| [34] |
C. Stolojescu-Crisan and S. Holban, An interactive x-ray image segmentation technique for bone extraction, in International Work-Conference on Bioinformatics and Biomedical Engineering, IWBBIO 2014, Granada, Spain, April 7-9, 2014 (eds. I. Rojas and F. M. O. Guzman), Copicentro Editorial, 2014, 1164–1171 Google Scholar |
| [35] |
H. Sun, et al., Aunet: Attention-guided dense-upsampling networks for breast mass segmentation in whole mammograms, Phys. Med. Biol., 65 (2020), 055005.
doi: 10.1088/1361-6560/ab5745. |
| [36] |
C. Szegedy, et al., Going deeper with convolutions, in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015
doi: 10.1109/CVPR.2015.7298594. |
| [37] |
A. Tack, A. Mukhopadhyay and S. Zachow,
Knee menisci segmentation using convolutional neural networks: Data from the osteoarthritis initiative, Osteoarthritis and Cartilage, 26 (2018), 680-688.
doi: 10.1016/j.joca.2018.02.907. |
| [38] |
W. Wang, Y. Wang, Y. Wu, T. Lin, S. Li and B. Chen,
Quantification of full left ventricular metrics via deep regression learning with contour-guidance, IEEE Access, 7 (2019), 47918-47928.
doi: 10.1109/ACCESS.2019.2907564. |
| [39] |
J. Wu, A. Belle, R. H. Hargraves, C. Cockrell, Y. Tang and K. Najarian,
Bone segmentation and 3d visualization of CT images for traumatic pelvic injuries, Int. J. Imaging Syst. Technol., 24 (2014), 29-38.
doi: 10.1002/ima.22076. |
| [40] |
X. Xiao, S. Lian, Z. Luo and S. Li, Weighted res-unet for high-quality retina vessel segmentation, in 2018 9th International Conference on Information Technology in Medicine and Education (ITME), 2018,327–331. Google Scholar |
| [41] |
Y. Xue, T. Xu, H. Zhang, L. R. Long and X. Huang,
Segan: Adversarial network with multi-scale L 1 loss for medical image segmentation, Neuroinformatics, 16 (2018), 383-392.
doi: 10.1007/s12021-018-9377-x. |
| [42] |
F. Yokota, T. Okada, M. Takao, N. Sugano, Y. Tada and Y. Sato, Automated segmentation of the femur and pelvis from 3d CT data of diseased hip using hierarchical statistical shape model of joint structure, in Medical Image Computing and Computer-Assisted Intervention - MICCAI 2009, 12th International Conference, London, UK, September 20-24, 2009, Proceedings, Part II (eds. G. Yang, D. J. Hawkes, D. Rueckert, J. A. Noble and C. J. Taylor), Lecture Notes in Computer Science, 5762, Springer, 2009,811–818.
doi: 10.1007/978-3-642-04271-3_98. |
| [43] |
F. Yu and V. Koltun, Multi-scale context aggregation by dilated convolutions, in 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings (eds. Y. Bengio and Y. LeCun), preprint Google Scholar |
| [44] |
K. Zhang, W. Lu and P. Marziliano,
Automatic knee cartilage segmentation from multi-contrast mr images using support vector machine classification with spatial dependencies, Magnetic Resonance Imaging, 31 (2013), 1731-1743.
doi: 10.1016/j.mri.2013.06.005. |
| [45] |
Z. Zhang, C. Duan, T. Lin, S. Zhou, Y. Wang and X. Gao,
GVFOM: A novel external force for active contour based image segmentation, Inf. Sci., 506 (2020), 1-18.
doi: 10.1016/j.ins.2019.08.003. |
| [46] |
Y. Zhou, W. Huang, P. Dong, Y. Xia and S. Wang, D-unet: A dimension-fusion U shape network for chronic stroke lesion segmentation, preprint, arXiv: 1908.05104.
doi: 10.1109/TCBB.2019.2939522. |
| [47] |
J. Zhu, T. Park, P. Isola and A. A. Efros, Unpaired image-to-image translation using cycle-consistent adversarial networks, in IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017
doi: 10.1109/ICCV.2017.244. |
| [48] |
Keras: Deep learning library for theano and tensorflow, https://github.com/keras-team/keras, 2015. Google Scholar |
| [49] |
Lableme, http://labelme.csail.mit.edu/Release3.0/. Google Scholar |
Figure 2.
The details of the standard block and residual block. f represents the number of filters. d represents the dilated rate
Figure 3.
the illustration on the left is an unfilled femur label and on the right is a filled tibia label
Figure 4.
The whole process of data augmentation
Figure 5.
The caption on the left is the loss of PDRU-Net on the training set, and that on the right is the loss of PDRU-Net on the validation set
Figure 6.
Segmentation results of the first three training epochs
Figure 7.
The first three columns show the segmentation results of U-Net, FusionNet and PDR U-Net respectively.The third and fourth columns show the corresponding ground truth images and input images respectively
Table 1.
The receptive field of each block in the encoding path of PDRU-Net
| Block Type | Convolutional Layer | Receptive Field |
| standard block 1 | conv1_1 | 1-1+1 |
| dilated rate = 1 | conv1_2 | 3-1+1 |
| residual block 2 | conv2_1 | 5-1+2 |
| dilated rate = 2 | conv2_2 | 9-1+2 |
| residual block 3 | conv3_1 | 13-1+4 |
| dilated rate = 4 | conv3_2 | 21-1+4 |
| residual block 4 | conv4_1 | 29-1+8 |
| dilated rate = 8 | conv4_2 | 45-1+8 |
| residual block 5 | conv5_1 | 61-1+16 |
| dilated rate = 16 | conv5_2 | 93-1+16 |
| residual block 6 | conv6_1 | 125-1+32 |
| dilated rate = 32 | conv6_2 | 189-1+32 |
| residual block 7 | conv7_1 | 253-1+32 |
| dilated rate = 32 | conv7_2 | 317-1+32 |
| residual block 8 | conv8_1 | 381-1+32 |
| dilated rate = 32 | conv8_2 | 445-1+32 |
| Block Type | Convolutional Layer | Receptive Field |
| standard block 1 | conv1_1 | 1-1+1 |
| dilated rate = 1 | conv1_2 | 3-1+1 |
| residual block 2 | conv2_1 | 5-1+2 |
| dilated rate = 2 | conv2_2 | 9-1+2 |
| residual block 3 | conv3_1 | 13-1+4 |
| dilated rate = 4 | conv3_2 | 21-1+4 |
| residual block 4 | conv4_1 | 29-1+8 |
| dilated rate = 8 | conv4_2 | 45-1+8 |
| residual block 5 | conv5_1 | 61-1+16 |
| dilated rate = 16 | conv5_2 | 93-1+16 |
| residual block 6 | conv6_1 | 125-1+32 |
| dilated rate = 32 | conv6_2 | 189-1+32 |
| residual block 7 | conv7_1 | 253-1+32 |
| dilated rate = 32 | conv7_2 | 317-1+32 |
| residual block 8 | conv8_1 | 381-1+32 |
| dilated rate = 32 | conv8_2 | 445-1+32 |
Table 2.
Comparison of the PDRU-Net, U-Net and FusionNet
| model | parameters | Dice Coefficient | Pixel Accuracy | Recall | Precision | F1 score |
| U-Net | ~33M | 0.918 | 0.943 | 0.839 | 0.987 | 0.907 |
| FusionNet | ~78M | 0.944 | 0.969 | 0.877 | 0.997 | 0.933 |
| PDRU-Net | ~0.36M | 0.973 | 0.987 | 0.953 | 0.976 | 0.964 |
| model | parameters | Dice Coefficient | Pixel Accuracy | Recall | Precision | F1 score |
| U-Net | ~33M | 0.918 | 0.943 | 0.839 | 0.987 | 0.907 |
| FusionNet | ~78M | 0.944 | 0.969 | 0.877 | 0.997 | 0.933 |
| PDRU-Net | ~0.36M | 0.973 | 0.987 | 0.953 | 0.976 | 0.964 |
| [1] |
Wenzhong Zhu, Huanlong Jiang, Erli Wang, Yani Hou, Lidong Xian, Joyati Debnath. X-ray image global enhancement algorithm in medical image classification. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 1297-1309. doi: 10.3934/dcdss.2019089 |
| [2] |
Silvia Allavena, Michele Piana, Federico Benvenuto, Anna Maria Massone. An interpolation/extrapolation approach to X-ray imaging of solar flares. Inverse Problems & Imaging, 2012, 6 (2) : 147-162. doi: 10.3934/ipi.2012.6.147 |
| [3] |
Nuutti Hyvönen, Martti Kalke, Matti Lassas, Henri Setälä, Samuli Siltanen. Three-dimensional dental X-ray imaging by combination of panoramic and projection data. Inverse Problems & Imaging, 2010, 4 (2) : 257-271. doi: 10.3934/ipi.2010.4.257 |
| [4] |
Dan Jane, Gabriel P. Paternain. On the injectivity of the X-ray transform for Anosov thermostats. Discrete & Continuous Dynamical Systems, 2009, 24 (2) : 471-487. doi: 10.3934/dcds.2009.24.471 |
| [5] |
François Rouvière. X-ray transform on Damek-Ricci spaces. Inverse Problems & Imaging, 2010, 4 (4) : 713-720. doi: 10.3934/ipi.2010.4.713 |
| [6] |
Aleksander Denisiuk. On range condition of the tensor x-ray transform in $ \mathbb R^n $. Inverse Problems & Imaging, 2020, 14 (3) : 423-435. doi: 10.3934/ipi.2020020 |
| [7] |
Arun K. Kulshreshth, Andreas Alpers, Gabor T. Herman, Erik Knudsen, Lajos Rodek, Henning F. Poulsen. A greedy method for reconstructing polycrystals from three-dimensional X-ray diffraction data. Inverse Problems & Imaging, 2009, 3 (1) : 69-85. doi: 10.3934/ipi.2009.3.69 |
| [8] |
Zhenhua Zhao, Yining Zhu, Jiansheng Yang, Ming Jiang. Mumford-Shah-TV functional with application in X-ray interior tomography. Inverse Problems & Imaging, 2018, 12 (2) : 331-348. doi: 10.3934/ipi.2018015 |
| [9] |
Jakob S. Jørgensen, Emil Y. Sidky, Per Christian Hansen, Xiaochuan Pan. Empirical average-case relation between undersampling and sparsity in X-ray CT. Inverse Problems & Imaging, 2015, 9 (2) : 431-446. doi: 10.3934/ipi.2015.9.431 |
| [10] |
Yijing Sun. Estimates for extremal values of $-\Delta u= h(x) u^{q}+\lambda W(x) u^{p}$. Communications on Pure & Applied Analysis, 2010, 9 (3) : 751-760. doi: 10.3934/cpaa.2010.9.751 |
| [11] |
Fan Jia, Xue-Cheng Tai, Jun Liu. Nonlocal regularized CNN for image segmentation. Inverse Problems & Imaging, 2020, 14 (5) : 891-911. doi: 10.3934/ipi.2020041 |
| [12] |
Ye Yuan, Yan Ren, Xiaodong Liu, Jing Wang. Approach to image segmentation based on interval neutrosophic set. Numerical Algebra, Control & Optimization, 2020, 10 (1) : 1-11. doi: 10.3934/naco.2019028 |
| [13] |
Dominique Zosso, Jing An, James Stevick, Nicholas Takaki, Morgan Weiss, Liane S. Slaughter, Huan H. Cao, Paul S. Weiss, Andrea L. Bertozzi. Image segmentation with dynamic artifacts detection and bias correction. Inverse Problems & Imaging, 2017, 11 (3) : 577-600. doi: 10.3934/ipi.2017027 |
| [14] |
Matthew S. Keegan, Berta Sandberg, Tony F. Chan. A multiphase logic framework for multichannel image segmentation. Inverse Problems & Imaging, 2012, 6 (1) : 95-110. doi: 10.3934/ipi.2012.6.95 |
| [15] |
Nicolas Lermé, François Malgouyres, Dominique Hamoir, Emmanuelle Thouin. Bayesian image restoration for mosaic active imaging. Inverse Problems & Imaging, 2014, 8 (3) : 733-760. doi: 10.3934/ipi.2014.8.733 |
| [16] |
Jia Cai, Guanglong Xu, Zhensheng Hu. Sketch-based image retrieval via CAT loss with elastic net regularization. Mathematical Foundations of Computing, 2020, 3 (4) : 219-227. doi: 10.3934/mfc.2020013 |
| [17] |
Shi Yan, Jun Liu, Haiyang Huang, Xue-Cheng Tai. A dual EM algorithm for TV regularized Gaussian mixture model in image segmentation. Inverse Problems & Imaging, 2019, 13 (3) : 653-677. doi: 10.3934/ipi.2019030 |
| [18] |
Jianping Zhang, Ke Chen, Bo Yu, Derek A. Gould. A local information based variational model for selective image segmentation. Inverse Problems & Imaging, 2014, 8 (1) : 293-320. doi: 10.3934/ipi.2014.8.293 |
| [19] |
Lu Tan, Ling Li, Senjian An, Zhenkuan Pan. Nonlinear diffusion based image segmentation using two fast algorithms. Mathematical Foundations of Computing, 2019, 2 (2) : 149-168. doi: 10.3934/mfc.2019011 |
| [20] |
Ruiliang Zhang, Xavier Bresson, Tony F. Chan, Xue-Cheng Tai. Four color theorem and convex relaxation for image segmentation with any number of regions. Inverse Problems & Imaging, 2013, 7 (3) : 1099-1113. doi: 10.3934/ipi.2013.7.1099 |
2019 Impact Factor: 1.373
Tools
Article outline
Figures and Tables
[Back to Top]