American Institute of Mathematical Sciences

March  2020, 2(1): 1-17. doi: 10.3934/fods.2020001

Stochastic gradient descent algorithm for stochastic optimization in solving analytic continuation problems

 1 Department of Mathematics, Florida State University, Tallahassee, Florida, USA 2 Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, Oak Ridge, Tennessee, USA

* Corresponding author: Feng Bao

Published  February 2020

Fund Project: The first author is supported by NSF grant DMS-1720222

We propose a stochastic gradient descent based optimization algorithm to solve the analytic continuation problem in which we extract real frequency spectra from imaginary time Quantum Monte Carlo data. The procedure of analytic continuation is an ill-posed inverse problem which is usually solved by regularized optimization methods, such like the Maximum Entropy method, or stochastic optimization methods. The main contribution of this work is to improve the performance of stochastic optimization approaches by introducing a supervised stochastic gradient descent algorithm to solve a flipped inverse system which processes the random solutions obtained by a type of Fast and Efficient Stochastic Optimization Method.

Citation: Feng Bao, Thomas Maier. Stochastic gradient descent algorithm for stochastic optimization in solving analytic continuation problems. Foundations of Data Science, 2020, 2 (1) : 1-17. doi: 10.3934/fods.2020001
References:

show all references

References:
Example 1. True spectrum
Example 1. (a) FESOM samples; (b) FESOM estimation
Example 1. Estimated spectrum learned from FESOM samples
Example 2. True spectrum
Example 2. (a) FESOM estimation; (b) Estimated spectrum learned from FESOM samples
Example 2. Comparison between SGD and MaxEnt
True spectrum
Example 3. Estimations for the spectrum
Example 3. Spectrum with fine feature in positive frequency region
Example 3. (a) MaxEnt estimation for $A_2$; (b) Comparison of MaxEnt in estimating $A_1$ (red) and $A_2$ (blue)
Example 3. (a) SGD estimation for $A_1$; (b) SGD estimation for $A_2$
 [1] Wataru Nakamura, Yasushi Narushima, Hiroshi Yabe. Nonlinear conjugate gradient methods with sufficient descent properties for unconstrained optimization. Journal of Industrial & Management Optimization, 2013, 9 (3) : 595-619. doi: 10.3934/jimo.2013.9.595 [2] Zhongqi Yin. A quantitative internal unique continuation for stochastic parabolic equations. Mathematical Control & Related Fields, 2015, 5 (1) : 165-176. doi: 10.3934/mcrf.2015.5.165 [3] David W. Pravica, Michael J. Spurr. Analytic continuation into the future. Conference Publications, 2003, 2003 (Special) : 709-716. doi: 10.3934/proc.2003.2003.709 [4] Zhenhuan Yang, Wei Shen, Yiming Ying, Xiaoming Yuan. Stochastic AUC optimization with general loss. Communications on Pure & Applied Analysis, 2020, 19 (8) : 4191-4212. doi: 10.3934/cpaa.2020188 [5] Peng Gao. Unique continuation property for stochastic nonclassical diffusion equations and stochastic linearized Benjamin-Bona-Mahony equations. Discrete & Continuous Dynamical Systems - B, 2019, 24 (6) : 2493-2510. doi: 10.3934/dcdsb.2018262 [6] Mingshang Hu. Stochastic global maximum principle for optimization with recursive utilities. Probability, Uncertainty and Quantitative Risk, 2017, 2 (0) : 1-. doi: 10.1186/s41546-017-0014-7 [7] Martin Heida, Stefan Neukamm, Mario Varga. Stochastic homogenization of $\Lambda$-convex gradient flows. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020328 [8] King Hann Lim, Hong Hui Tan, Hendra G. Harno. Approximate greatest descent in neural network optimization. Numerical Algebra, Control & Optimization, 2018, 8 (3) : 327-336. doi: 10.3934/naco.2018021 [9] Yuk L. Yung, Cameron Taketa, Ross Cheung, Run-Lie Shia. Infinite sum of the product of exponential and logarithmic functions, its analytic continuation, and application. Discrete & Continuous Dynamical Systems - B, 2010, 13 (1) : 229-248. doi: 10.3934/dcdsb.2010.13.229 [10] Jan Boman. Unique continuation of microlocally analytic distributions and injectivity theorems for the ray transform. Inverse Problems & Imaging, 2010, 4 (4) : 619-630. doi: 10.3934/ipi.2010.4.619 [11] Ardeshir Ahmadi, Hamed Davari-Ardakani. A multistage stochastic programming framework for cardinality constrained portfolio optimization. Numerical Algebra, Control & Optimization, 2017, 7 (3) : 359-377. doi: 10.3934/naco.2017023 [12] Haodong Yu, Jie Sun. Robust stochastic optimization with convex risk measures: A discretized subgradient scheme. Journal of Industrial & Management Optimization, 2019  doi: 10.3934/jimo.2019100 [13] Xiaming Chen. Kernel-based online gradient descent using distributed approach. Mathematical Foundations of Computing, 2019, 2 (1) : 1-9. doi: 10.3934/mfc.2019001 [14] Ting Hu. Kernel-based maximum correntropy criterion with gradient descent method. Communications on Pure & Applied Analysis, 2020, 19 (8) : 4159-4177. doi: 10.3934/cpaa.2020186 [15] Shishun Li, Zhengda Huang. Guaranteed descent conjugate gradient methods with modified secant condition. Journal of Industrial & Management Optimization, 2008, 4 (4) : 739-755. doi: 10.3934/jimo.2008.4.739 [16] Jean-Paul Arnaout, Georges Arnaout, John El Khoury. Simulation and optimization of ant colony optimization algorithm for the stochastic uncapacitated location-allocation problem. Journal of Industrial & Management Optimization, 2016, 12 (4) : 1215-1225. doi: 10.3934/jimo.2016.12.1215 [17] Peter Takáč. Stabilization of positive solutions for analytic gradient-like systems. Discrete & Continuous Dynamical Systems - A, 2000, 6 (4) : 947-973. doi: 10.3934/dcds.2000.6.947 [18] M. M. Ali, L. Masinga. A nonlinear optimization model for optimal order quantities with stochastic demand rate and price change. Journal of Industrial & Management Optimization, 2007, 3 (1) : 139-154. doi: 10.3934/jimo.2007.3.139 [19] Wenjuan Zhao, Shunfu Jin, Wuyi Yue. A stochastic model and social optimization of a blockchain system based on a general limited batch service queue. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020049 [20] Meng Xue, Yun Shi, Hailin Sun. Portfolio optimization with relaxation of stochastic second order dominance constraints via conditional value at risk. Journal of Industrial & Management Optimization, 2019  doi: 10.3934/jimo.2019071

Impact Factor: