# American Institute of Mathematical Sciences

• Previous Article
Inverse problems for a half-order time-fractional diffusion equation in arbitrary dimension by Carleman estimates
• IPI Home
• This Issue
• Next Article
Refined stability estimates in electrical impedance tomography with multi-layer structure
doi: 10.3934/ipi.2021045
Online First

Online First articles are published articles within a journal that have not yet been assigned to a formal issue. This means they do not yet have a volume number, issue number, or page numbers assigned to them, however, they can still be found and cited using their DOI (Digital Object Identifier). Online First publication benefits the research community by making new scientific discoveries known as quickly as possible.

Readers can access Online First articles via the “Online First” tab for the selected journal.

## Learning to scan: A deep reinforcement learning approach for personalized scanning in CT imaging

 1 Academy for Advanced Interdisciplinary Studies, Peking University, China 2 Computer Science Department, Carnegie Mellon University, USA 3 Center for Advanced Medical Computing and Analysis, Massachusetts General Hospital and Harvard Medical School, USA 4 Department of Mathematics, University of California, Santa Barbara, USA 5 Beijing International Center for Mathematical Research, Center for Data Science, Institute for Artificial Intelligence, Peking University, China

Corresponding Author

* Equal Contribution

Received  October 2020 Revised  April 2021 Early access July 2021

Computed Tomography (CT) takes X-ray measurements on the subjects to reconstruct tomographic images. As X-ray is radioactive, it is desirable to control the total amount of dose of X-ray for safety concerns. Therefore, we can only select a limited number of measurement angles and assign each of them limited amount of dose. Traditional methods such as compressed sensing usually randomly select the angles and equally distribute the allowed dose on them. In most CT reconstruction models, the emphasize is on designing effective image representations, while much less emphasize is on improving the scanning strategy. The simple scanning strategy of random angle selection and equal dose distribution performs well in general, but they may not be ideal for each individual subject. It is more desirable to design a personalized scanning strategy for each subject to obtain better reconstruction result. In this paper, we propose to use Reinforcement Learning (RL) to learn a personalized scanning policy to select the angles and the dose at each chosen angle for each individual subject. We first formulate the CT scanning process as an Markov Decision Process (MDP), and then use modern deep RL methods to solve it. The learned personalized scanning strategy not only leads to better reconstruction results, but also shows strong generalization to be combined with different reconstruction algorithms.

Citation: Ziju Shen, Yufei Wang, Dufan Wu, Xu Yang, Bin Dong. Learning to scan: A deep reinforcement learning approach for personalized scanning in CT imaging. Inverse Problems & Imaging, doi: 10.3934/ipi.2021045
##### References:
 [1] W. van Aarle, W. J. Palenstijn, J. Cant, E. Janssens, F. Bleichrodt, A. Dabravolski, J. De Beenhouwer, K. J. Batenburg and J. Sijbers, Fast and flexible x-ray tomography using the astra toolbox, Optics Express, 22 (2016), 25129-25147.   Google Scholar [2] W. van Aarle, W. J. Palenstijn, J. D. Beenhouwer, T. Altantzis, S. Bals, K. J. Batenburg and J. Sijbers, The astra toolbox: A platform for advanced algorithm development in electron tomography, Ultramicroscopy, 24 (2015), 35-47.   Google Scholar [3] J. Adler and O. Oktem, Learned primal-dual reconstruction, IEEE Transactions on Medical Imaging, 37 (2018), 1322-1332.   Google Scholar [4] K. J. Batenburg, W. J. Palenstijn, P. Balázs and J. Sijbers, Dynamic angle selection in binary tomography, Computer Vision and Image Understanding, 117 (2013), 306-318.   Google Scholar [5] I. Bello, H. Pham, Q. V Le, M. Norouzi and S. Bengio, Neural combinatorial optimization with reinforcement learning, preprint, arXiv: 1611.09940, 2016. Google Scholar [6] S. Boyd and N. Parikh, Distributed optimization and statistical learning via the alternating direction method of multipliers, Foundations and Trends in Machine Learning, 3 (2011), 1-122.   Google Scholar [7] A. Buades, B. Coll and J. M. Morel, A review of image denoising algorithms, with a new one, Multiscale Model. Simul., 2 (2005), 490-530.  doi: 10.1137/040616024.  Google Scholar [8] J.-F. Cai, S. Osher and Z. Shen, Split bregman methods and frame based image restoration, Multiscale Modeling and Simulation, 8 (2009), 337-369.  doi: 10.1137/090753504.  Google Scholar [9] J.-F. Cai, H. Ji, Z. Shen and G. B. Ye, Data-driven tight frame construction and image denoising, Applied and Computational Harmonic Analysis, 37 (2014), 89-105.  doi: 10.1016/j.acha.2013.10.001.  Google Scholar [10] E. J. Candes, Y. C. Eldar, et al., Compressed Sensing With Coherent and Redundant Dictionaries, 2010. Google Scholar [11] E. J. Candes, J. Romberg and T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Transactions on Information Theory, 52 (2006), 489-509.  doi: 10.1109/TIT.2005.862083.  Google Scholar [12] A. Chambolle and T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging, Journal of Mathematical Imaging and Vision, 40 (2011), 120-145.  doi: 10.1007/s10851-010-0251-1.  Google Scholar [13] H. Chen, Y. Zhang, M. K. Kalra, F. Lin, Y. Chen, P. Liao, J. Zhou and G. Wang, Low-dose CT with a residual encoder-decoder convolutional neural network, IEEE Transactions on Medical Imaging, 36 (2017), 2524-2535.   Google Scholar [14] K. Dabov, A. Foi, V. Katkovnik and K. Egiazarian, Image denoising by sparse 3-d transform-domain collaborative filtering, IEEE Transactions on Image Processing, 16 (2007), 2080-2095.  doi: 10.1109/TIP.2007.901238.  Google Scholar [15] A. Dabravolski, K. J. Batenburg and J. Sijbers, Dynamic angle selection in x-ray computed tomography, Nuclear Instruments and Methods in Physics Research Section B: Beam Interactions with Materials and Atoms, 324 (2014), 17-24.   Google Scholar [16] I. Daubechies, Ten Lectures on Wavelets, CBMS-NSF Regional Conference Series in Applied Mathematics, 61. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1992. doi: 10.1137/1.9781611970104.  Google Scholar [17] B. Dong, J. Li and Z. Shen, X-ray CT image reconstruction via wavelet frame based regularization and radon domain inpainting, Journal of Scientific Computing, 54 (2013), 333-349.  doi: 10.1007/s10915-012-9579-6.  Google Scholar [18] B. Dong and Z. Shen, Mra based wavelet frames and applications, IAS Lecture Notes Series, Summer Program on "The Mathematics of Image Processing", Park City Mathematics Institute, 19 (2010), 9-158.  doi: 10.1090/pcms/019/02.  Google Scholar [19] D. Donoho, Compressed sensing, IEEE Transactions on Information Theory, 52 (2006), 1289-1306.   Google Scholar [20] J. M. Ede, Adaptive partial scanning transmission electron microscopy with reinforcement learning, preprint, arXiv: 2004.02786. Google Scholar [21] M. Elad and M. Aharon, Image denoising via sparse and redundant representations over learned dictionaries, IEEE Transactions on Image processing, 15 (2006), 3736-3745.  doi: 10.1109/TIP.2006.881969.  Google Scholar [22] E. Esser and X. Zhang, A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science, SIAM Journal on Imaging Sciences, 3 (2010), 1015-1046.  doi: 10.1137/09076934X.  Google Scholar [23] M. Gies, W. A. Kalender, H. Wolf and C. Suess, Dose reduction in CT by anatomically adapted tube current modulation. i. Simulation studies, Medical Physics, 26 (1999), 2235-2247.   Google Scholar [24] G. D. Godaliyadda, M. A. Uchic, D. H. Ye, M. A. Groeber, G. T. Buzzard and C. A. Bouman, A supervised learning approach for dynamic sampling, S & T Imaging. International Society for Optics and Photonics, 2016. Google Scholar [25] G. M. D. P. Godaliyadda, D. H. Ye, M. D. Uchic, M. A. Groeber, G. T. Buzzard and C. A. Bouman, A framework for dynamic image sampling based on supervised learning (slads), IEEE Trans. Comput. Imaging, 4 (2018), 1-16.  doi: 10.1109/TCI.2017.2777482.  Google Scholar [26] T. Goldstein and S. Osher, The split bregman method for l1-regularized problems, SIAM Journal Imaging Sciences, 2 (2009), 323-343.  doi: 10.1137/080725891.  Google Scholar [27] T. Goldstein and S. Osher, The split bregman method for $l_1$-regularized problems, SIAM Journal on Imaging Sciences, 2 (2009), 323-343.  doi: 10.1137/080725891.  Google Scholar [28] R. Gordon, R. Benderab and G. T. Herman, Algebraic Reconstruction Techniques (ART) for Three-Dimensional Electron Microscopy and X-ray Photography, Journal of Theoretical Biology, 1970. Google Scholar [29] S. Gu, L. Zhang, W. Zuo and X. Feng, Weighted nuclear norm minimization with application to image denoising, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2014), 2862–2869. Google Scholar [30] A. Halimi, P. Ciuciu, A. Mccarthy, S. Mclaughlin and G. Buller, Fast adaptive scene sampling for single-photon 3d lidar images, IEEE CAMSAP 2019 - International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, 2019. Google Scholar [31] S. Ji, Y. Xue and L. Carin, Bayesian compressive sensing, IEEE Transactions on Signal Processing, 56 (2008), 2346-2356.  doi: 10.1109/TSP.2007.914345.  Google Scholar [32] K. H. Jin, M. T. McCann, E. Froustey and M. Unser, Deep convolutional neural network for inverse problems in imaging, IEEE Transactions on Image Processing, 26 (2017), 4509-4522.  doi: 10.1109/TIP.2017.2713099.  Google Scholar [33] W. A. Kalender, H. Wolf and C. Suess, Dose reduction in CT by anatomically adapted tube current modulation. ii. Phantom measurements, Medical Physics, 26 (1999), 2248-2253.   Google Scholar [34] E. Kang, J. Min and J. C. Ye, A deep conversational neural network using directional wavelets for low-dose x-ray ct reconstruction, Medical Physics, 44 (2017), e360–e375. doi: 10.1002/mp.12344.  Google Scholar [35] A. Katsevich, Theoretically exact filtered backprojection-type inversion algorithm for spiral CT, SIAM Journal on Applied Mathematics, 62 (2002), 2012-2026.  doi: 10.1137/S0036139901387186.  Google Scholar [36] D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, arXiv: 1412.6980, 2014. Google Scholar [37] W. Kool, H. V. Hoof and M. Welling, Attention, learn to solve routing problems!, preprint, arXiv: 1803.08475, 2018. Google Scholar [38] Y. Li, Deep Reinforcement Learning: An overview, arXiv: 1701.07274, 2017. Google Scholar [39] L. Ly and Y.-H. R. Tsai, Autonomous exploration, reconstruction, and surveillance of 3d environments aided by deep learning, arXiv: 1809.06025, 2018. Google Scholar [40] S. Mallat, A Wavelet Tour of Signal Processing, Academic Press, Inc., San Diego, CA, 1998.   Google Scholar [41] M. T. McCann, K. H. Jin and M. Unser, Convolutional neural networks for inverse problems in imaging: A review, IEEE Signal Processing Magazine, 34 (2017), 85-95.   Google Scholar [42] C. McCollough, Tu-fg-207a-04: Overview of the low dose ct grand challenge, Medical Physics, 43 (2016), 3760-3760.  doi: 10.1118/1.4957556.  Google Scholar [43] A. Mittal, A. Dhawan, S. Manchanda, S. Medya, S. Ranu and A. Singh, Learning heuristics over large graphs via deep reinforcement learning, preprint, arXiv: 1903.03332, 2019. Google Scholar [44] V. Mnih, K. Kavukcuoglu and D. Silver, Human-level control through deep reinforcement learning, Nature, 518 (2015). Google Scholar [45] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland and G. Ostrovski, Human-level control through deep reinforcement learning, Nature, 518 (2015), 529-533.   Google Scholar [46] K. A. Mohan, S. V. Venkatakrishnan, E. B. Gulsoy, J. W. Gibbs, X. Xiao, M. D. Graef, P. W. Voorhees and C. A. Bouman, Timbir: A method for time-space reconstruction from interlaced views, IEEE Transactions on Computational Imaging, 1 (2015), 96-111.  doi: 10.1109/TCI.2015.2431913.  Google Scholar [47] E. Monier, N. Brun, T. Oberlin, X. Li, M. Tenc and N. Dobigeon, Fast reconstruction of atomic-scale stem-eels images from sparse sampling, Ultramicroscopy, 2020. Google Scholar [48] K. Mueller, Selection of optimal views for computed tomography reconstruction, Patent WO, Jan, 28 (2011). Google Scholar [49] K. Mueller, R. Yagel and J. J. Wheller, Anti-aliased three-dimensional cone-beam reconstruction of low-contrast objects with algebraic methods, IEEE Transactions On Medical Imaging, 6 (1999), 519-537.   Google Scholar [50] R. Ohbuchi and M. Aono, Quasi-Monte Carlo Rendering With Adaptive Sampling, 1996. Google Scholar [51] S. Osher, Z. Shi and W. Zhu, Low dimensional manifold model for image processing, SIAM Journal on Imaging Sciences, 10 (2017), 1669-1690.  doi: 10.1137/16M1058686.  Google Scholar [52] J. Park, J. Jung, A. P. Gupta, J. Soh, C. Jeong, J. Ahn, S. Cho, K. -H. Yoon, D. Kim, M. Mativenga, et al., Multi-beam x-ray source based on carbon nanotube emitters for tomosynthesis system, in Medical Imaging 2020: Physics of Medical Imaging, International Society for Optics and Photonics, 11312 (2020), 113122. Google Scholar [53] G. Placidi, M. Alecci and A. Sotgiu, Theory of adaptive acquisition method for image reconstruction from projections and application to epr image, Journal of Magnetic Resonance, (1995), 50–57. Google Scholar [54] S. RL, Fast calculation of the exact radiological path for a three-dimensional CT array, Medical Physics, 2 (1985), 252-5.   Google Scholar [55] A. Ron and Z. Shen, Affine systems in $L_{2}(\mathbb{R}^{d})$: The analysis of the analysis operator, Journal of Functional Analysis, 148 (1997), 408-447.  doi: 10.1006/jfan.1996.3079.  Google Scholar [56] L. I. Rudin, S. Osher and E. Fatemi, Nonlinear total variation based noise removal algorithms, Physical D: Nonlinear Phenomena, 60 (1992), 259-268.  doi: 10.1016/0167-2789(92)90242-F.  Google Scholar [57] J. Schulman and F. Wolski, Proximal policy optimization algorithms, arXiv: 1707.06347v2, 2017. Google Scholar [58] M. W. Seeger and H. Nickisch, Compressed sensing and bayesian experimental design, in Proceedings of the 25th International Conference on Machine Learning, (2008), 912–919. Google Scholar [59] B. Settles, Active Iearning Literature Survey, Technical report, University of Wisconsin-Madison Department of Computer Sciences, 2009. Google Scholar [60] C. Shen, Y. Gonzalez, L. Chen, S. B. Jiang and X. Jia, Intelligent parameter tuning in optimization-based iterative CT reconstruction via deep reinforcement learning, IEEE Transactions on Medical Imaging, 37 (2018), 1430-1439.   Google Scholar [61] E. Y. Sidky and X. Pan, Image reconstruction in circular cone-beam computed tomography by constrained, total-variation minimization, Physics in medicine and biology, 4777, 2008. Google Scholar [62] D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra and M. Riedmiller, Deterministic policy gradient algorithms, In International Conference on Machine Learning, (2014), 387–395. Google Scholar [63] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, MIT Press, Cambridge, MA, 2018.   Google Scholar [64] R. S. Sutton, D. A. McAllester, S. P. Singh and Y. Mansour, Policy gradient methods for reinforcement learning with function approximation, In Advances in Neural Information Processing Systems, (2000), 1057–1063. Google Scholar [65] C. Tai and E. Weinan, Multiscale adaptive representation of signals: I. The basic framework, The Journal of Machine Learning Research, 17 (2016), 4875-4912.   Google Scholar [66] G. Wang, A perspective on deep imaging, IEEE Access, 4 (2016), 8914-8924.   Google Scholar [67] Z. Wang and G. R. Arce, Variable density compressed image sampling, Image Processing, IEEE Transactions, 19 (2010), 264-270.  doi: 10.1109/TIP.2009.2032889.  Google Scholar [68] G. Wang, M. Kalra and C. G.Orton, Machine learning will transform radiology significantly within the next 5 years, Medical Physics, 44 (2017), 2041-2044.   Google Scholar [69] G. Wang, J. Chu Ye, K. Mueller and J. A Fessler, Image reconstruction is a new frontier of machine learning, IEEE Transactions on Medical Imaging, 37 (2018), 1289-1296.   Google Scholar [70] G. Wang and H. Yu, A scheme for multisource interior tomography, Medical physics, 36 (2009), 3575-3581.   Google Scholar [71] C. J. Watkins and P. Dayan, Q-learning, Machine learning, 8 (1992), 279-292.   Google Scholar [72] Q. Yang, P. Yan, Y. Zhang, H. Yu, Y. Shi, X. Mou, M. K. Kalra, Y. Zhang, L. Sun and G. Wang, Low-dose CT image denoising using a generative adversarial network with wasserstein distance and perceptual loss, IEEE Transactions on Medical Imaging, 37 (2018) 1348–1357. Google Scholar [73] L. Yu, M. Shiung, D. Jondal and C. H. McCollough, Development and validation of a practical lower-dose-simulation tool for optimizing computed tomography scan protocols, Journal of Computer Assisted Tomography, 36 (20124), 477-487.   Google Scholar [74] J. Zhang, G. Yang, Y. Lee, Y. Cheng, B. Gao, Q. Qiu, J. Lu and O. Zhou, A multi-beam x-ray imaging system based on carbon nanotube field emitters, in Medical Imaging 2006: Physics of Medical Imaging, International Society for Optics and Photonics, 6142 (2006), 614204. Google Scholar [75] S. Zhang, Z. Song, G. D. P. Godaliyadda, D. H. Ye, A. U. Chowdhury, A. Sengupta, G. T. Buzzard, C. A. Bouman and G. J. Simpson, Dynamic sparse sampling for confocal raman microscopy, Analytical Chemistry, 90 (2018), 4461-4469.   Google Scholar [76] Y. Zhang, G. M. D. Godaliyadda, N. Ferrier, E. B. Gulsoy, C. A. Bouman and C. Phatak, Slads-Net: Supervised Learning Approach for Dynamic Sampling Using Deep Neural Networks, Electronic Imaging, Computational Imaging XVI, 2018. Google Scholar [77] H. -M. Zhang and B. Dong, A review on deep learning in medical image reconstruction, Journal of the Operations Research Society of China, 8 (2020) 311–340. doi: 10.1007/s40305-019-00287-4.  Google Scholar [78] Z. Zhang, X. Liang, X. Dong, Y. Xie and G. Cao, A sparse-view CT reconstruction method based on combination of DenseNet and deconvolution, IEEE Transactions on Medical Imaging, 37 (2018), 1407-1417.   Google Scholar [79] M. Zhu and T. Chan, An efficient primal-dual hybrid gradient algorithm for total variation image restoration, UCLA CAM Report, 34 (2008), 8-34.   Google Scholar

show all references

##### References:
 [1] W. van Aarle, W. J. Palenstijn, J. Cant, E. Janssens, F. Bleichrodt, A. Dabravolski, J. De Beenhouwer, K. J. Batenburg and J. Sijbers, Fast and flexible x-ray tomography using the astra toolbox, Optics Express, 22 (2016), 25129-25147.   Google Scholar [2] W. van Aarle, W. J. Palenstijn, J. D. Beenhouwer, T. Altantzis, S. Bals, K. J. Batenburg and J. Sijbers, The astra toolbox: A platform for advanced algorithm development in electron tomography, Ultramicroscopy, 24 (2015), 35-47.   Google Scholar [3] J. Adler and O. Oktem, Learned primal-dual reconstruction, IEEE Transactions on Medical Imaging, 37 (2018), 1322-1332.   Google Scholar [4] K. J. Batenburg, W. J. Palenstijn, P. Balázs and J. Sijbers, Dynamic angle selection in binary tomography, Computer Vision and Image Understanding, 117 (2013), 306-318.   Google Scholar [5] I. Bello, H. Pham, Q. V Le, M. Norouzi and S. Bengio, Neural combinatorial optimization with reinforcement learning, preprint, arXiv: 1611.09940, 2016. Google Scholar [6] S. Boyd and N. Parikh, Distributed optimization and statistical learning via the alternating direction method of multipliers, Foundations and Trends in Machine Learning, 3 (2011), 1-122.   Google Scholar [7] A. Buades, B. Coll and J. M. Morel, A review of image denoising algorithms, with a new one, Multiscale Model. Simul., 2 (2005), 490-530.  doi: 10.1137/040616024.  Google Scholar [8] J.-F. Cai, S. Osher and Z. Shen, Split bregman methods and frame based image restoration, Multiscale Modeling and Simulation, 8 (2009), 337-369.  doi: 10.1137/090753504.  Google Scholar [9] J.-F. Cai, H. Ji, Z. Shen and G. B. Ye, Data-driven tight frame construction and image denoising, Applied and Computational Harmonic Analysis, 37 (2014), 89-105.  doi: 10.1016/j.acha.2013.10.001.  Google Scholar [10] E. J. Candes, Y. C. Eldar, et al., Compressed Sensing With Coherent and Redundant Dictionaries, 2010. Google Scholar [11] E. J. Candes, J. Romberg and T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Transactions on Information Theory, 52 (2006), 489-509.  doi: 10.1109/TIT.2005.862083.  Google Scholar [12] A. Chambolle and T. Pock, A first-order primal-dual algorithm for convex problems with applications to imaging, Journal of Mathematical Imaging and Vision, 40 (2011), 120-145.  doi: 10.1007/s10851-010-0251-1.  Google Scholar [13] H. Chen, Y. Zhang, M. K. Kalra, F. Lin, Y. Chen, P. Liao, J. Zhou and G. Wang, Low-dose CT with a residual encoder-decoder convolutional neural network, IEEE Transactions on Medical Imaging, 36 (2017), 2524-2535.   Google Scholar [14] K. Dabov, A. Foi, V. Katkovnik and K. Egiazarian, Image denoising by sparse 3-d transform-domain collaborative filtering, IEEE Transactions on Image Processing, 16 (2007), 2080-2095.  doi: 10.1109/TIP.2007.901238.  Google Scholar [15] A. Dabravolski, K. J. Batenburg and J. Sijbers, Dynamic angle selection in x-ray computed tomography, Nuclear Instruments and Methods in Physics Research Section B: Beam Interactions with Materials and Atoms, 324 (2014), 17-24.   Google Scholar [16] I. Daubechies, Ten Lectures on Wavelets, CBMS-NSF Regional Conference Series in Applied Mathematics, 61. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1992. doi: 10.1137/1.9781611970104.  Google Scholar [17] B. Dong, J. Li and Z. Shen, X-ray CT image reconstruction via wavelet frame based regularization and radon domain inpainting, Journal of Scientific Computing, 54 (2013), 333-349.  doi: 10.1007/s10915-012-9579-6.  Google Scholar [18] B. Dong and Z. Shen, Mra based wavelet frames and applications, IAS Lecture Notes Series, Summer Program on "The Mathematics of Image Processing", Park City Mathematics Institute, 19 (2010), 9-158.  doi: 10.1090/pcms/019/02.  Google Scholar [19] D. Donoho, Compressed sensing, IEEE Transactions on Information Theory, 52 (2006), 1289-1306.   Google Scholar [20] J. M. Ede, Adaptive partial scanning transmission electron microscopy with reinforcement learning, preprint, arXiv: 2004.02786. Google Scholar [21] M. Elad and M. Aharon, Image denoising via sparse and redundant representations over learned dictionaries, IEEE Transactions on Image processing, 15 (2006), 3736-3745.  doi: 10.1109/TIP.2006.881969.  Google Scholar [22] E. Esser and X. Zhang, A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science, SIAM Journal on Imaging Sciences, 3 (2010), 1015-1046.  doi: 10.1137/09076934X.  Google Scholar [23] M. Gies, W. A. Kalender, H. Wolf and C. Suess, Dose reduction in CT by anatomically adapted tube current modulation. i. Simulation studies, Medical Physics, 26 (1999), 2235-2247.   Google Scholar [24] G. D. Godaliyadda, M. A. Uchic, D. H. Ye, M. A. Groeber, G. T. Buzzard and C. A. Bouman, A supervised learning approach for dynamic sampling, S & T Imaging. International Society for Optics and Photonics, 2016. Google Scholar [25] G. M. D. P. Godaliyadda, D. H. Ye, M. D. Uchic, M. A. Groeber, G. T. Buzzard and C. A. Bouman, A framework for dynamic image sampling based on supervised learning (slads), IEEE Trans. Comput. Imaging, 4 (2018), 1-16.  doi: 10.1109/TCI.2017.2777482.  Google Scholar [26] T. Goldstein and S. Osher, The split bregman method for l1-regularized problems, SIAM Journal Imaging Sciences, 2 (2009), 323-343.  doi: 10.1137/080725891.  Google Scholar [27] T. Goldstein and S. Osher, The split bregman method for $l_1$-regularized problems, SIAM Journal on Imaging Sciences, 2 (2009), 323-343.  doi: 10.1137/080725891.  Google Scholar [28] R. Gordon, R. Benderab and G. T. Herman, Algebraic Reconstruction Techniques (ART) for Three-Dimensional Electron Microscopy and X-ray Photography, Journal of Theoretical Biology, 1970. Google Scholar [29] S. Gu, L. Zhang, W. Zuo and X. Feng, Weighted nuclear norm minimization with application to image denoising, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2014), 2862–2869. Google Scholar [30] A. Halimi, P. Ciuciu, A. Mccarthy, S. Mclaughlin and G. Buller, Fast adaptive scene sampling for single-photon 3d lidar images, IEEE CAMSAP 2019 - International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, 2019. Google Scholar [31] S. Ji, Y. Xue and L. Carin, Bayesian compressive sensing, IEEE Transactions on Signal Processing, 56 (2008), 2346-2356.  doi: 10.1109/TSP.2007.914345.  Google Scholar [32] K. H. Jin, M. T. McCann, E. Froustey and M. Unser, Deep convolutional neural network for inverse problems in imaging, IEEE Transactions on Image Processing, 26 (2017), 4509-4522.  doi: 10.1109/TIP.2017.2713099.  Google Scholar [33] W. A. Kalender, H. Wolf and C. Suess, Dose reduction in CT by anatomically adapted tube current modulation. ii. Phantom measurements, Medical Physics, 26 (1999), 2248-2253.   Google Scholar [34] E. Kang, J. Min and J. C. Ye, A deep conversational neural network using directional wavelets for low-dose x-ray ct reconstruction, Medical Physics, 44 (2017), e360–e375. doi: 10.1002/mp.12344.  Google Scholar [35] A. Katsevich, Theoretically exact filtered backprojection-type inversion algorithm for spiral CT, SIAM Journal on Applied Mathematics, 62 (2002), 2012-2026.  doi: 10.1137/S0036139901387186.  Google Scholar [36] D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, arXiv: 1412.6980, 2014. Google Scholar [37] W. Kool, H. V. Hoof and M. Welling, Attention, learn to solve routing problems!, preprint, arXiv: 1803.08475, 2018. Google Scholar [38] Y. Li, Deep Reinforcement Learning: An overview, arXiv: 1701.07274, 2017. Google Scholar [39] L. Ly and Y.-H. R. Tsai, Autonomous exploration, reconstruction, and surveillance of 3d environments aided by deep learning, arXiv: 1809.06025, 2018. Google Scholar [40] S. Mallat, A Wavelet Tour of Signal Processing, Academic Press, Inc., San Diego, CA, 1998.   Google Scholar [41] M. T. McCann, K. H. Jin and M. Unser, Convolutional neural networks for inverse problems in imaging: A review, IEEE Signal Processing Magazine, 34 (2017), 85-95.   Google Scholar [42] C. McCollough, Tu-fg-207a-04: Overview of the low dose ct grand challenge, Medical Physics, 43 (2016), 3760-3760.  doi: 10.1118/1.4957556.  Google Scholar [43] A. Mittal, A. Dhawan, S. Manchanda, S. Medya, S. Ranu and A. Singh, Learning heuristics over large graphs via deep reinforcement learning, preprint, arXiv: 1903.03332, 2019. Google Scholar [44] V. Mnih, K. Kavukcuoglu and D. Silver, Human-level control through deep reinforcement learning, Nature, 518 (2015). Google Scholar [45] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland and G. Ostrovski, Human-level control through deep reinforcement learning, Nature, 518 (2015), 529-533.   Google Scholar [46] K. A. Mohan, S. V. Venkatakrishnan, E. B. Gulsoy, J. W. Gibbs, X. Xiao, M. D. Graef, P. W. Voorhees and C. A. Bouman, Timbir: A method for time-space reconstruction from interlaced views, IEEE Transactions on Computational Imaging, 1 (2015), 96-111.  doi: 10.1109/TCI.2015.2431913.  Google Scholar [47] E. Monier, N. Brun, T. Oberlin, X. Li, M. Tenc and N. Dobigeon, Fast reconstruction of atomic-scale stem-eels images from sparse sampling, Ultramicroscopy, 2020. Google Scholar [48] K. Mueller, Selection of optimal views for computed tomography reconstruction, Patent WO, Jan, 28 (2011). Google Scholar [49] K. Mueller, R. Yagel and J. J. Wheller, Anti-aliased three-dimensional cone-beam reconstruction of low-contrast objects with algebraic methods, IEEE Transactions On Medical Imaging, 6 (1999), 519-537.   Google Scholar [50] R. Ohbuchi and M. Aono, Quasi-Monte Carlo Rendering With Adaptive Sampling, 1996. Google Scholar [51] S. Osher, Z. Shi and W. Zhu, Low dimensional manifold model for image processing, SIAM Journal on Imaging Sciences, 10 (2017), 1669-1690.  doi: 10.1137/16M1058686.  Google Scholar [52] J. Park, J. Jung, A. P. Gupta, J. Soh, C. Jeong, J. Ahn, S. Cho, K. -H. Yoon, D. Kim, M. Mativenga, et al., Multi-beam x-ray source based on carbon nanotube emitters for tomosynthesis system, in Medical Imaging 2020: Physics of Medical Imaging, International Society for Optics and Photonics, 11312 (2020), 113122. Google Scholar [53] G. Placidi, M. Alecci and A. Sotgiu, Theory of adaptive acquisition method for image reconstruction from projections and application to epr image, Journal of Magnetic Resonance, (1995), 50–57. Google Scholar [54] S. RL, Fast calculation of the exact radiological path for a three-dimensional CT array, Medical Physics, 2 (1985), 252-5.   Google Scholar [55] A. Ron and Z. Shen, Affine systems in $L_{2}(\mathbb{R}^{d})$: The analysis of the analysis operator, Journal of Functional Analysis, 148 (1997), 408-447.  doi: 10.1006/jfan.1996.3079.  Google Scholar [56] L. I. Rudin, S. Osher and E. Fatemi, Nonlinear total variation based noise removal algorithms, Physical D: Nonlinear Phenomena, 60 (1992), 259-268.  doi: 10.1016/0167-2789(92)90242-F.  Google Scholar [57] J. Schulman and F. Wolski, Proximal policy optimization algorithms, arXiv: 1707.06347v2, 2017. Google Scholar [58] M. W. Seeger and H. Nickisch, Compressed sensing and bayesian experimental design, in Proceedings of the 25th International Conference on Machine Learning, (2008), 912–919. Google Scholar [59] B. Settles, Active Iearning Literature Survey, Technical report, University of Wisconsin-Madison Department of Computer Sciences, 2009. Google Scholar [60] C. Shen, Y. Gonzalez, L. Chen, S. B. Jiang and X. Jia, Intelligent parameter tuning in optimization-based iterative CT reconstruction via deep reinforcement learning, IEEE Transactions on Medical Imaging, 37 (2018), 1430-1439.   Google Scholar [61] E. Y. Sidky and X. Pan, Image reconstruction in circular cone-beam computed tomography by constrained, total-variation minimization, Physics in medicine and biology, 4777, 2008. Google Scholar [62] D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra and M. Riedmiller, Deterministic policy gradient algorithms, In International Conference on Machine Learning, (2014), 387–395. Google Scholar [63] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, MIT Press, Cambridge, MA, 2018.   Google Scholar [64] R. S. Sutton, D. A. McAllester, S. P. Singh and Y. Mansour, Policy gradient methods for reinforcement learning with function approximation, In Advances in Neural Information Processing Systems, (2000), 1057–1063. Google Scholar [65] C. Tai and E. Weinan, Multiscale adaptive representation of signals: I. The basic framework, The Journal of Machine Learning Research, 17 (2016), 4875-4912.   Google Scholar [66] G. Wang, A perspective on deep imaging, IEEE Access, 4 (2016), 8914-8924.   Google Scholar [67] Z. Wang and G. R. Arce, Variable density compressed image sampling, Image Processing, IEEE Transactions, 19 (2010), 264-270.  doi: 10.1109/TIP.2009.2032889.  Google Scholar [68] G. Wang, M. Kalra and C. G.Orton, Machine learning will transform radiology significantly within the next 5 years, Medical Physics, 44 (2017), 2041-2044.   Google Scholar [69] G. Wang, J. Chu Ye, K. Mueller and J. A Fessler, Image reconstruction is a new frontier of machine learning, IEEE Transactions on Medical Imaging, 37 (2018), 1289-1296.   Google Scholar [70] G. Wang and H. Yu, A scheme for multisource interior tomography, Medical physics, 36 (2009), 3575-3581.   Google Scholar [71] C. J. Watkins and P. Dayan, Q-learning, Machine learning, 8 (1992), 279-292.   Google Scholar [72] Q. Yang, P. Yan, Y. Zhang, H. Yu, Y. Shi, X. Mou, M. K. Kalra, Y. Zhang, L. Sun and G. Wang, Low-dose CT image denoising using a generative adversarial network with wasserstein distance and perceptual loss, IEEE Transactions on Medical Imaging, 37 (2018) 1348–1357. Google Scholar [73] L. Yu, M. Shiung, D. Jondal and C. H. McCollough, Development and validation of a practical lower-dose-simulation tool for optimizing computed tomography scan protocols, Journal of Computer Assisted Tomography, 36 (20124), 477-487.   Google Scholar [74] J. Zhang, G. Yang, Y. Lee, Y. Cheng, B. Gao, Q. Qiu, J. Lu and O. Zhou, A multi-beam x-ray imaging system based on carbon nanotube field emitters, in Medical Imaging 2006: Physics of Medical Imaging, International Society for Optics and Photonics, 6142 (2006), 614204. Google Scholar [75] S. Zhang, Z. Song, G. D. P. Godaliyadda, D. H. Ye, A. U. Chowdhury, A. Sengupta, G. T. Buzzard, C. A. Bouman and G. J. Simpson, Dynamic sparse sampling for confocal raman microscopy, Analytical Chemistry, 90 (2018), 4461-4469.   Google Scholar [76] Y. Zhang, G. M. D. Godaliyadda, N. Ferrier, E. B. Gulsoy, C. A. Bouman and C. Phatak, Slads-Net: Supervised Learning Approach for Dynamic Sampling Using Deep Neural Networks, Electronic Imaging, Computational Imaging XVI, 2018. Google Scholar [77] H. -M. Zhang and B. Dong, A review on deep learning in medical image reconstruction, Journal of the Operations Research Society of China, 8 (2020) 311–340. doi: 10.1007/s40305-019-00287-4.  Google Scholar [78] Z. Zhang, X. Liang, X. Dong, Y. Xie and G. Cao, A sparse-view CT reconstruction method based on combination of DenseNet and deconvolution, IEEE Transactions on Medical Imaging, 37 (2018), 1407-1417.   Google Scholar [79] M. Zhu and T. Chan, An efficient primal-dual hybrid gradient algorithm for total variation image restoration, UCLA CAM Report, 34 (2008), 8-34.   Google Scholar
Policy network architecture. Each MLP contains two hidden layers with 512 neurons. We use a multi-layer GRU which contains 3 recurrent layers and each layer has 256 neurons. The Angle MLP has one hidden layer of 512 neurons, and the Dose MLP has 2 hidden layers with 512 neurons
. Figures in (a), (b) and (c) correspond to the three different noise levels. For each (a), (b) and (c), the first row is PSNR and the second is SSIM. Figures from left to right are results from reconstruction methods SART, TV, WF and PD-net respectively. Every sub-figure contains histograms of three scanning strategies, i.e., RL-AD, DS-ED and UF-AEC">Figure 2.  Histograms of PSNR and SSIM from all the 350 test images as shown in Table 1. Figures in (a), (b) and (c) correspond to the three different noise levels. For each (a), (b) and (c), the first row is PSNR and the second is SSIM. Figures from left to right are results from reconstruction methods SART, TV, WF and PD-net respectively. Every sub-figure contains histograms of three scanning strategies, i.e., RL-AD, DS-ED and UF-AEC
Two examples of the reconstructed images. The top row contains the ground truth images and their zoom-in views. The second through the fourth row contain results from UF-AEC, DS-ED and RL-AD respectively, and combined with PD-net's reconstruction. Note that RL-AD selects 65 measurement angles for the subject in (a) and 54 measurement angles for the subject in (b)
(a) Distribution of number of measurements of the learned policy (RL-AD) on all the 350 testing CT images. (b) Dose usage distribution of 8 images that use around 54 measurements. (c) Dose usage distribution of 8 images that use around 65 measurements
(a): an example image that takes 54 measurements. (b): an example image that takes 64 measurements. We can see that the images for which RL selects more measurement angles contains more structures
. Top row: RL-AD, bottom row: DS-ED. The lines show the selected angles">Figure 6.  The angle selection of the CT image in Figure 5. Top row: RL-AD, bottom row: DS-ED. The lines show the selected angles
This table presents comparisons of different scanning strategies (1-3rd column for RL-AD, DS-ED and UF-AEC respectively) combined with different image reconstruction methods (1-4th row for SART, TV, WF and PD-net respectively). Last row presents the inference times of angle selection (in seconds) of the three compared scanning strategies. The mean (std) of the PSNR and SSIM of the reconstructed images and the inference times are computed among all 350 testing CT images. The best results among the compared algorithms are shown in bold numbers
 Reconstruction Method RL-AD DS-ED UF-AEC Noise 1 SART PSNR 23.48(0.47) 23.30(0.64) 23.01(0.64) SSIM 0.424(0.020) 0.403(0.023) 0.391(0.022) TV PSNR 23.85(0.42) 23.75(0.41) 23.63(0.38) SSIM 0.582(0.030) 0.579(0.030) 0.578(0.028) WF PSNR 25.14(0.40) 25.05(0.42) 24.91(0.39) SSIM 0.659(0.027) 0.652(0.027) 0.649(0.026) PD-net PSNR 30.87(0.64) 30.44(0.51) 30.23(0.46) SSIM 0.776(0.036) 0.771(0.029) 0.773(0.028) Noise 2 SART PSNR 23.15(0.48) 22.91(0.53) 22.60(0.64) SSIM 0.413(0.020) 0.390(0.024) 0.378(0.024) TV PSNR 23.74(0.40) 23.50(0.36) 23.27(0.40) SSIM 0.580(0.030) 0.576(0.030) 0.573(0.028) WF PSNR 24.98(0.29) 24.84(0.41) 24.68(0.39) SSIM 0.657(0.027) 0.649(0.026) 0.646(0.026) PD-net PSNR 30.78(0.64) 30.35(0.51) 30.15(0.77) SSIM 0.774(0.037) 0.769(0.030) 0.771(0.029) Noise 3 SART PSNR 20.71(0.55) 20.26(0.72) 19.83(0.66) SSIM 0.334(0.026) 0.304(0.030) 0.291(0.029) TV PSNR 21.73(0.57) 21.43(0.48) 21.08(0.47) SSIM 0.568(0.027) 0.555(0.026) 0.545(0.026) WF PSNR 23.35(0.48) 23.05(0.51) 22.72(0.55) SSIM 0.636(0.0326) 0.616(0.027) 0.605(0.028) PD-net PSNR 29.97(0.66) 29.56(0.51) 29.36(0.47) SSIM 0.753(0.038) 0.746(0.032) 0.747(0.031) Inference Time (s) 0.46(0.02) 0.21(0.008) 0.20(0.001)
 Reconstruction Method RL-AD DS-ED UF-AEC Noise 1 SART PSNR 23.48(0.47) 23.30(0.64) 23.01(0.64) SSIM 0.424(0.020) 0.403(0.023) 0.391(0.022) TV PSNR 23.85(0.42) 23.75(0.41) 23.63(0.38) SSIM 0.582(0.030) 0.579(0.030) 0.578(0.028) WF PSNR 25.14(0.40) 25.05(0.42) 24.91(0.39) SSIM 0.659(0.027) 0.652(0.027) 0.649(0.026) PD-net PSNR 30.87(0.64) 30.44(0.51) 30.23(0.46) SSIM 0.776(0.036) 0.771(0.029) 0.773(0.028) Noise 2 SART PSNR 23.15(0.48) 22.91(0.53) 22.60(0.64) SSIM 0.413(0.020) 0.390(0.024) 0.378(0.024) TV PSNR 23.74(0.40) 23.50(0.36) 23.27(0.40) SSIM 0.580(0.030) 0.576(0.030) 0.573(0.028) WF PSNR 24.98(0.29) 24.84(0.41) 24.68(0.39) SSIM 0.657(0.027) 0.649(0.026) 0.646(0.026) PD-net PSNR 30.78(0.64) 30.35(0.51) 30.15(0.77) SSIM 0.774(0.037) 0.769(0.030) 0.771(0.029) Noise 3 SART PSNR 20.71(0.55) 20.26(0.72) 19.83(0.66) SSIM 0.334(0.026) 0.304(0.030) 0.291(0.029) TV PSNR 21.73(0.57) 21.43(0.48) 21.08(0.47) SSIM 0.568(0.027) 0.555(0.026) 0.545(0.026) WF PSNR 23.35(0.48) 23.05(0.51) 22.72(0.55) SSIM 0.636(0.0326) 0.616(0.027) 0.605(0.028) PD-net PSNR 29.97(0.66) 29.56(0.51) 29.36(0.47) SSIM 0.753(0.038) 0.746(0.032) 0.747(0.031) Inference Time (s) 0.46(0.02) 0.21(0.008) 0.20(0.001)
 [1] Shousheng Luo, Tie Zhou. Superiorization of EM algorithm and its application in Single-Photon Emission Computed Tomography(SPECT). Inverse Problems & Imaging, 2014, 8 (1) : 223-246. doi: 10.3934/ipi.2014.8.223 [2] Dieudonné Nijimbere, Songzheng Zhao, Xunhao Gu, Moses Olabhele Esangbedo, Nyiribakwe Dominique. Tabu search guided by reinforcement learning for the max-mean dispersion problem. Journal of Industrial & Management Optimization, 2021, 17 (6) : 3223-3246. doi: 10.3934/jimo.2020115 [3] Jiaxin Zhang, Hoang Tran, Guannan Zhang. Accelerating reinforcement learning with a Directional-Gaussian-Smoothing evolution strategy. Electronic Research Archive, , () : -. doi: 10.3934/era.2021075 [4] Jianfeng Feng, Mariya Shcherbina, Brunello Tirozzi. Stability of the dynamics of an asymmetric neural network. Communications on Pure & Applied Analysis, 2009, 8 (2) : 655-671. doi: 10.3934/cpaa.2009.8.655 [5] Yuan Xu, Xin Jin, Saiwei Wang, Yang Tang. Optimal synchronization control of multiple euler-lagrange systems via event-triggered reinforcement learning. Discrete & Continuous Dynamical Systems - S, 2021, 14 (4) : 1495-1518. doi: 10.3934/dcdss.2020377 [6] Ndolane Sene. Fractional input stability and its application to neural network. Discrete & Continuous Dynamical Systems - S, 2020, 13 (3) : 853-865. doi: 10.3934/dcdss.2020049 [7] Ying Sue Huang, Chai Wah Wu. Stability of cellular neural network with small delays. Conference Publications, 2005, 2005 (Special) : 420-426. doi: 10.3934/proc.2005.2005.420 [8] King Hann Lim, Hong Hui Tan, Hendra G. Harno. Approximate greatest descent in neural network optimization. Numerical Algebra, Control & Optimization, 2018, 8 (3) : 327-336. doi: 10.3934/naco.2018021 [9] Shyan-Shiou Chen, Chih-Wen Shih. Asymptotic behaviors in a transiently chaotic neural network. Discrete & Continuous Dynamical Systems, 2004, 10 (3) : 805-826. doi: 10.3934/dcds.2004.10.805 [10] Rui Hu, Yuan Yuan. Stability, bifurcation analysis in a neural network model with delay and diffusion. Conference Publications, 2009, 2009 (Special) : 367-376. doi: 10.3934/proc.2009.2009.367 [11] Hui-Qiang Ma, Nan-Jing Huang. Neural network smoothing approximation method for stochastic variational inequality problems. Journal of Industrial & Management Optimization, 2015, 11 (2) : 645-660. doi: 10.3934/jimo.2015.11.645 [12] Yixin Guo, Aijun Zhang. Existence and nonexistence of traveling pulses in a lateral inhibition neural network. Discrete & Continuous Dynamical Systems - B, 2016, 21 (6) : 1729-1755. doi: 10.3934/dcdsb.2016020 [13] Jianhong Wu, Ruyuan Zhang. A simple delayed neural network with large capacity for associative memory. Discrete & Continuous Dynamical Systems - B, 2004, 4 (3) : 851-863. doi: 10.3934/dcdsb.2004.4.851 [14] Weishi Yin, Jiawei Ge, Pinchao Meng, Fuheng Qu. A neural network method for the inverse scattering problem of impenetrable cavities. Electronic Research Archive, 2020, 28 (2) : 1123-1142. doi: 10.3934/era.2020062 [15] Sanjay K. Mazumdar, Cheng-Chew Lim. A neural network based anti-skid brake system. Discrete & Continuous Dynamical Systems, 1999, 5 (2) : 321-338. doi: 10.3934/dcds.1999.5.321 [16] Hiroaki Uchida, Yuya Oishi, Toshimichi Saito. A simple digital spiking neural network: Synchronization and spike-train approximation. Discrete & Continuous Dynamical Systems - S, 2021, 14 (4) : 1479-1494. doi: 10.3934/dcdss.2020374 [17] Lidong Liu, Fajie Wei, Shenghan Zhou. Major project risk assessment method based on BP neural network. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 1053-1064. doi: 10.3934/dcdss.2019072 [18] K. L. Mak, J. G. Peng, Z. B. Xu, K. F. C. Yiu. A novel neural network for associative memory via dynamical systems. Discrete & Continuous Dynamical Systems - B, 2006, 6 (3) : 573-590. doi: 10.3934/dcdsb.2006.6.573 [19] Hyeontae Jo, Hwijae Son, Hyung Ju Hwang, Eun Heui Kim. Deep neural network approach to forward-inverse problems. Networks & Heterogeneous Media, 2020, 15 (2) : 247-259. doi: 10.3934/nhm.2020011 [20] Danilo Costarelli, Gianluca Vinti. Asymptotic expansions and Voronovskaja type theorems for the multivariate neural network operators. Mathematical Foundations of Computing, 2020, 3 (1) : 41-50. doi: 10.3934/mfc.2020004

2020 Impact Factor: 1.639

## Tools

Article outline

Figures and Tables