March  2015, 5(1): 97-139. doi: 10.3934/mcrf.2015.5.97

A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon

1. 

Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China, China

2. 

Department of Mathematics, University of Central Florida, Orlando, FL 32816

Received  July 2013 Revised  November 2013 Published  January 2015

A linear-quadratic (LQ, for short) optimal control problem is considered for mean-field stochastic differential equations with constant coefficients in an infinite horizon. The stabilizability of the control system is studied followed by the discussion of the well-posedness of the LQ problem. The optimal control can be expressed as a linear state feedback involving the state and its mean, through the solutions of two algebraic Riccati equations. The solvability of such kind of Riccati equations is investigated by means of semi-definite programming method.
Citation: Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control & Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97
References:
[1]

N. U. Ahmed and X. Ding, A semilinear McKean-Vlasov stochastic evolution equation in Hilbert space,, Stoch. Proc. Appl., 60 (1995), 65.  doi: 10.1016/0304-4149(95)00050-X.  Google Scholar

[2]

N. U. Ahmed and X. Ding, Controlled McKean-Vlasov equations,, Comm. Appl. Anal., 5 (2001), 183.   Google Scholar

[3]

N. U. Ahmed, Nonlinear diffusion governed by McKean-Vlasov equation on Hilbert space and optimal control,, SIAM J. Control Optim., 46 (2007), 356.  doi: 10.1137/050645944.  Google Scholar

[4]

M. Ait Rami and X. Y. Zhou, Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls,, IEEE Transactions on Automatic Control, 45 (2000), 1131.  doi: 10.1109/9.863597.  Google Scholar

[5]

A. Albert, Conditions for positive and nonnegative definiteness in terms of pseudo-inverses,, SIAM J. Appl. Math., 17 (1969), 434.  doi: 10.1137/0117041.  Google Scholar

[6]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type,, Appl. Math. Optim., 63 (2011), 341.  doi: 10.1007/s00245-010-9123-8.  Google Scholar

[7]

A. Ben-Israel and T. N. E. Greville, Generalized Inverses,, Springer-Verlag, (2003).   Google Scholar

[8]

A. Bensoussan, G. Da Prato, M. C. Delfour and S. K. Mitter, Representation and Control of Infinite Dimensional Systems,, 2nd edition, (2007).  doi: 10.1007/978-0-8176-4581-6.  Google Scholar

[9]

V. S. Borkar and K. S. Kumar, McKean-Vlasov limit in portfolio optimization,, Stoch. Anal. Appl., 28 (2010), 884.  doi: 10.1080/07362994.2010.482836.  Google Scholar

[10]

S. Boyd, L. El Ghaoul, E. Feron and V. Balakrishnan, Linear Matrix Inequality in Systems and Control Theory,, SIAM, (1994).  doi: 10.1137/1.9781611970777.  Google Scholar

[11]

R. Buckdahn, B. Djehiche and J. Li, A general maximum principle for SDEs of mean-field type,, Applied Mathematics & Optimization, 64 (2011), 197.  doi: 10.1007/s00245-011-9136-y.  Google Scholar

[12]

R. Buckdahn, B. Djehiche, J. Li and S. Peng, Mean-field backward stochastic differential equations: A limit approach,, Ann. Probab., 37 (2009), 1524.  doi: 10.1214/08-AOP442.  Google Scholar

[13]

R. Buckdahn, J. Li and S. Peng, Mean-field backward stochastic differential equations and related partial differential equations,, Stoch. Process. Appl., 119 (2009), 3133.  doi: 10.1016/j.spa.2009.05.002.  Google Scholar

[14]

T. Chan, Dynamics of the McKean-Vlasov equation,, Ann. Probab., 22 (1994), 431.  doi: 10.1214/aop/1176988866.  Google Scholar

[15]

T. Chiang, McKean-Vlasov equations with discontinuous coefficients,, Soochow J. Math., 20 (1994), 507.   Google Scholar

[16]

D. Crisan and J. Xiong, Approximate McKean-Vlasov representations for a class of SPDEs,, Stochastics, 82 (2010), 53.  doi: 10.1080/17442500902723575.  Google Scholar

[17]

D. A. Dawson, Critical dynamics and fluctuations for a mean-field model of cooperative behavior,, J. Statist. Phys., 31 (1983), 29.  doi: 10.1007/BF01010922.  Google Scholar

[18]

D. A. Dawson and J. Gärtner, Large deviations from the McKean-Vlasov limit for weakly interacting diffusions,, Stochastics, 20 (1987), 247.  doi: 10.1080/17442508708833446.  Google Scholar

[19]

L. El Ghaoui and M. Ait Rami, Robust state-feedback stabilization of jump linear systems via LIMs,, Int. J. Robust and Nonlinear Contr., 6 (1996), 1015.  doi: 10.1002/(SICI)1099-1239(199611)6:9/10<1015::AID-RNC266>3.0.CO;2-0.  Google Scholar

[20]

J. Gärtner, On the Mckean-Vlasov limit for interacting diffusions,, Math. Nachr., 137 (1988), 197.  doi: 10.1002/mana.19881370116.  Google Scholar

[21]

C. Graham, McKean-Vlasov Ito-Skorohod equations, and nonlinear diffusions with discrete jump sets,, Stoch. Proc. Appl., 40 (1992), 69.  doi: 10.1016/0304-4149(92)90138-G.  Google Scholar

[22]

M. Grant and S. Boyd, Graph implementations for nonsmooth convex programs,, Recent Advances in Learning and Control (a tribute to M. Vidyasagar), 371 (2008), 95.  doi: 10.1007/978-1-84800-155-8_7.  Google Scholar

[23]

M. Huang, R. P. Malhamé, and P. E. Caines, Large population stochastic dynamic games: Closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle,, Comm. Inform. Systems, 6 (2006), 221.  doi: 10.4310/CIS.2006.v6.n3.a5.  Google Scholar

[24]

M. Kac, Foundations of kinetic theory,, Proc. 3rd Berkeley Sympos. Math. Statist. Prob., 3 (1956), 171.   Google Scholar

[25]

P. E. Kloeden and T. Lorenz, Stochastic differential equations with nonlocal sample dependence,, Stoch. Anal. Appl., 28 (2010), 937.  doi: 10.1080/07362994.2010.515194.  Google Scholar

[26]

P. M. Kotelenez and T. G. Kurtz, Macroscopic limit for stochastic partial differential equations of McKean-Vlasov type,, Prob. Theory Rel. Fields, 146 (2010), 189.  doi: 10.1007/s00440-008-0188-0.  Google Scholar

[27]

J. M. Lasry and P. L. Lions, Mean field games,, Japan J. Math., 2 (2007), 229.  doi: 10.1007/s11537-007-0657-8.  Google Scholar

[28]

X. Li, X.Y. Zhou and M. Ait Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon,, Journal of Global Optimization, 27 (2003), 149.  doi: 10.1023/A:1024887007165.  Google Scholar

[29]

H. P. McKean, A class of Markov processes associated with nonlinear parabolic equations,, Proc. Natl. Acad. Sci. USA, 56 (1966), 1907.  doi: 10.1073/pnas.56.6.1907.  Google Scholar

[30]

T. Meyer-Brandis, B. /'Oksendal and X. Y. Zhou, A mean-field stochastic maximum principle via Malliavin calculus,, Stochastics, 84 (2012), 643.  doi: 10.1080/17442508.2011.651619.  Google Scholar

[31]

J. Y. Park, P. Balasubramaniam and Y. H. Kang, Controllability of McKean-Vlasov stochastic integrodifferential evolution equation in Hilbert spaces,, Numer. Funct. Anal. Optim., 29 (2008), 1328.  doi: 10.1080/01630560802580679.  Google Scholar

[32]

R. Penrose, A generalized inverse of matrices,, Proc. Cambridge Philos. Soc., 51 (1955), 406.  doi: 10.1017/S0305004100030401.  Google Scholar

[33]

M. Scheutzow, Uniqueness and non-uniqueness of solutions of Vlasov-McKean equations,, J. Austral. Math. Soc., 43 (1987), 246.  doi: 10.1017/S1446788700029384.  Google Scholar

[34]

R. H Tutuncu, K. C. Toh, and M. J. Todd, Solving semidefinite-quadratic-linear programs using SDPT3,, Mathematical Programming Ser. B, 95 (2003), 189.  doi: 10.1007/s10107-002-0347-5.  Google Scholar

[35]

L. Vandenerghe and S. Boyd, Semidefinite programming,, SIAM Rev., 38 (1996), 49.  doi: 10.1137/1038003.  Google Scholar

[36]

A. Yu. Veretennikov, On ergodic measures for McKean-Vlasov stochastic equations,, in Monte Carlo and quasi-Monte Carlo methods 2004, (2004), 471.  doi: 10.1007/3-540-31186-6_29.  Google Scholar

[37]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations,, SIAM J. Control Optim., 51 (2013), 2809.  doi: 10.1137/120892477.  Google Scholar

[38]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations,, Applications of Mathematics (New York), (1999).  doi: 10.1007/978-1-4612-1466-3.  Google Scholar

show all references

References:
[1]

N. U. Ahmed and X. Ding, A semilinear McKean-Vlasov stochastic evolution equation in Hilbert space,, Stoch. Proc. Appl., 60 (1995), 65.  doi: 10.1016/0304-4149(95)00050-X.  Google Scholar

[2]

N. U. Ahmed and X. Ding, Controlled McKean-Vlasov equations,, Comm. Appl. Anal., 5 (2001), 183.   Google Scholar

[3]

N. U. Ahmed, Nonlinear diffusion governed by McKean-Vlasov equation on Hilbert space and optimal control,, SIAM J. Control Optim., 46 (2007), 356.  doi: 10.1137/050645944.  Google Scholar

[4]

M. Ait Rami and X. Y. Zhou, Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls,, IEEE Transactions on Automatic Control, 45 (2000), 1131.  doi: 10.1109/9.863597.  Google Scholar

[5]

A. Albert, Conditions for positive and nonnegative definiteness in terms of pseudo-inverses,, SIAM J. Appl. Math., 17 (1969), 434.  doi: 10.1137/0117041.  Google Scholar

[6]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type,, Appl. Math. Optim., 63 (2011), 341.  doi: 10.1007/s00245-010-9123-8.  Google Scholar

[7]

A. Ben-Israel and T. N. E. Greville, Generalized Inverses,, Springer-Verlag, (2003).   Google Scholar

[8]

A. Bensoussan, G. Da Prato, M. C. Delfour and S. K. Mitter, Representation and Control of Infinite Dimensional Systems,, 2nd edition, (2007).  doi: 10.1007/978-0-8176-4581-6.  Google Scholar

[9]

V. S. Borkar and K. S. Kumar, McKean-Vlasov limit in portfolio optimization,, Stoch. Anal. Appl., 28 (2010), 884.  doi: 10.1080/07362994.2010.482836.  Google Scholar

[10]

S. Boyd, L. El Ghaoul, E. Feron and V. Balakrishnan, Linear Matrix Inequality in Systems and Control Theory,, SIAM, (1994).  doi: 10.1137/1.9781611970777.  Google Scholar

[11]

R. Buckdahn, B. Djehiche and J. Li, A general maximum principle for SDEs of mean-field type,, Applied Mathematics & Optimization, 64 (2011), 197.  doi: 10.1007/s00245-011-9136-y.  Google Scholar

[12]

R. Buckdahn, B. Djehiche, J. Li and S. Peng, Mean-field backward stochastic differential equations: A limit approach,, Ann. Probab., 37 (2009), 1524.  doi: 10.1214/08-AOP442.  Google Scholar

[13]

R. Buckdahn, J. Li and S. Peng, Mean-field backward stochastic differential equations and related partial differential equations,, Stoch. Process. Appl., 119 (2009), 3133.  doi: 10.1016/j.spa.2009.05.002.  Google Scholar

[14]

T. Chan, Dynamics of the McKean-Vlasov equation,, Ann. Probab., 22 (1994), 431.  doi: 10.1214/aop/1176988866.  Google Scholar

[15]

T. Chiang, McKean-Vlasov equations with discontinuous coefficients,, Soochow J. Math., 20 (1994), 507.   Google Scholar

[16]

D. Crisan and J. Xiong, Approximate McKean-Vlasov representations for a class of SPDEs,, Stochastics, 82 (2010), 53.  doi: 10.1080/17442500902723575.  Google Scholar

[17]

D. A. Dawson, Critical dynamics and fluctuations for a mean-field model of cooperative behavior,, J. Statist. Phys., 31 (1983), 29.  doi: 10.1007/BF01010922.  Google Scholar

[18]

D. A. Dawson and J. Gärtner, Large deviations from the McKean-Vlasov limit for weakly interacting diffusions,, Stochastics, 20 (1987), 247.  doi: 10.1080/17442508708833446.  Google Scholar

[19]

L. El Ghaoui and M. Ait Rami, Robust state-feedback stabilization of jump linear systems via LIMs,, Int. J. Robust and Nonlinear Contr., 6 (1996), 1015.  doi: 10.1002/(SICI)1099-1239(199611)6:9/10<1015::AID-RNC266>3.0.CO;2-0.  Google Scholar

[20]

J. Gärtner, On the Mckean-Vlasov limit for interacting diffusions,, Math. Nachr., 137 (1988), 197.  doi: 10.1002/mana.19881370116.  Google Scholar

[21]

C. Graham, McKean-Vlasov Ito-Skorohod equations, and nonlinear diffusions with discrete jump sets,, Stoch. Proc. Appl., 40 (1992), 69.  doi: 10.1016/0304-4149(92)90138-G.  Google Scholar

[22]

M. Grant and S. Boyd, Graph implementations for nonsmooth convex programs,, Recent Advances in Learning and Control (a tribute to M. Vidyasagar), 371 (2008), 95.  doi: 10.1007/978-1-84800-155-8_7.  Google Scholar

[23]

M. Huang, R. P. Malhamé, and P. E. Caines, Large population stochastic dynamic games: Closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle,, Comm. Inform. Systems, 6 (2006), 221.  doi: 10.4310/CIS.2006.v6.n3.a5.  Google Scholar

[24]

M. Kac, Foundations of kinetic theory,, Proc. 3rd Berkeley Sympos. Math. Statist. Prob., 3 (1956), 171.   Google Scholar

[25]

P. E. Kloeden and T. Lorenz, Stochastic differential equations with nonlocal sample dependence,, Stoch. Anal. Appl., 28 (2010), 937.  doi: 10.1080/07362994.2010.515194.  Google Scholar

[26]

P. M. Kotelenez and T. G. Kurtz, Macroscopic limit for stochastic partial differential equations of McKean-Vlasov type,, Prob. Theory Rel. Fields, 146 (2010), 189.  doi: 10.1007/s00440-008-0188-0.  Google Scholar

[27]

J. M. Lasry and P. L. Lions, Mean field games,, Japan J. Math., 2 (2007), 229.  doi: 10.1007/s11537-007-0657-8.  Google Scholar

[28]

X. Li, X.Y. Zhou and M. Ait Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon,, Journal of Global Optimization, 27 (2003), 149.  doi: 10.1023/A:1024887007165.  Google Scholar

[29]

H. P. McKean, A class of Markov processes associated with nonlinear parabolic equations,, Proc. Natl. Acad. Sci. USA, 56 (1966), 1907.  doi: 10.1073/pnas.56.6.1907.  Google Scholar

[30]

T. Meyer-Brandis, B. /'Oksendal and X. Y. Zhou, A mean-field stochastic maximum principle via Malliavin calculus,, Stochastics, 84 (2012), 643.  doi: 10.1080/17442508.2011.651619.  Google Scholar

[31]

J. Y. Park, P. Balasubramaniam and Y. H. Kang, Controllability of McKean-Vlasov stochastic integrodifferential evolution equation in Hilbert spaces,, Numer. Funct. Anal. Optim., 29 (2008), 1328.  doi: 10.1080/01630560802580679.  Google Scholar

[32]

R. Penrose, A generalized inverse of matrices,, Proc. Cambridge Philos. Soc., 51 (1955), 406.  doi: 10.1017/S0305004100030401.  Google Scholar

[33]

M. Scheutzow, Uniqueness and non-uniqueness of solutions of Vlasov-McKean equations,, J. Austral. Math. Soc., 43 (1987), 246.  doi: 10.1017/S1446788700029384.  Google Scholar

[34]

R. H Tutuncu, K. C. Toh, and M. J. Todd, Solving semidefinite-quadratic-linear programs using SDPT3,, Mathematical Programming Ser. B, 95 (2003), 189.  doi: 10.1007/s10107-002-0347-5.  Google Scholar

[35]

L. Vandenerghe and S. Boyd, Semidefinite programming,, SIAM Rev., 38 (1996), 49.  doi: 10.1137/1038003.  Google Scholar

[36]

A. Yu. Veretennikov, On ergodic measures for McKean-Vlasov stochastic equations,, in Monte Carlo and quasi-Monte Carlo methods 2004, (2004), 471.  doi: 10.1007/3-540-31186-6_29.  Google Scholar

[37]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations,, SIAM J. Control Optim., 51 (2013), 2809.  doi: 10.1137/120892477.  Google Scholar

[38]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations,, Applications of Mathematics (New York), (1999).  doi: 10.1007/978-1-4612-1466-3.  Google Scholar

[1]

Siyang Cai, Yongmei Cai, Xuerong Mao. A stochastic differential equation SIS epidemic model with regime switching. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020317

[2]

Claudianor O. Alves, Rodrigo C. M. Nemer, Sergio H. Monari Soares. The use of the Morse theory to estimate the number of nontrivial solutions of a nonlinear Schrödinger equation with a magnetic field. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2020276

[3]

Reza Chaharpashlou, Abdon Atangana, Reza Saadati. On the fuzzy stability results for fractional stochastic Volterra integral equation. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020432

[4]

Hong Niu, Zhijiang Feng, Qijin Xiao, Yajun Zhang. A PID control method based on optimal control strategy. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 117-126. doi: 10.3934/naco.2020019

[5]

Awais Younus, Zoubia Dastgeer, Nudrat Ishaq, Abdul Ghaffar, Kottakkaran Sooppy Nisar, Devendra Kumar. On the observability of conformable linear time-invariant control systems. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020444

[6]

Lorenzo Zambotti. A brief and personal history of stochastic partial differential equations. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 471-487. doi: 10.3934/dcds.2020264

[7]

Yongge Tian, Pengyang Xie. Simultaneous optimal predictions under two seemingly unrelated linear random-effects models. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020168

[8]

Lars Grüne, Matthias A. Müller, Christopher M. Kellett, Steven R. Weller. Strict dissipativity for discrete time discounted optimal control problems. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020046

[9]

Yueyang Zheng, Jingtao Shi. A stackelberg game of backward stochastic differential equations with partial information. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020047

[10]

Youming Guo, Tingting Li. Optimal control strategies for an online game addiction model with low and high risk exposure. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020347

[11]

Fathalla A. Rihan, Hebatallah J. Alsakaji. Stochastic delay differential equations of three-species prey-predator system with cooperation among prey species. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020468

[12]

Jie Li, Xiangdong Ye, Tao Yu. Mean equicontinuity, complexity and applications. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 359-393. doi: 10.3934/dcds.2020167

[13]

Peter Poláčik, Pavol Quittner. Entire and ancient solutions of a supercritical semilinear heat equation. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 413-438. doi: 10.3934/dcds.2020136

[14]

Jianhua Huang, Yanbin Tang, Ming Wang. Singular support of the global attractor for a damped BBM equation. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020345

[15]

Stefano Bianchini, Paolo Bonicatto. Forward untangling and applications to the uniqueness problem for the continuity equation. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020384

[16]

Bernard Bonnard, Jérémy Rouot. Geometric optimal techniques to control the muscular force response to functional electrical stimulation using a non-isometric force-fatigue model. Journal of Geometric Mechanics, 2020  doi: 10.3934/jgm.2020032

[17]

Xuefei He, Kun Wang, Liwei Xu. Efficient finite difference methods for the nonlinear Helmholtz equation in Kerr medium. Electronic Research Archive, 2020, 28 (4) : 1503-1528. doi: 10.3934/era.2020079

[18]

Cheng He, Changzheng Qu. Global weak solutions for the two-component Novikov equation. Electronic Research Archive, 2020, 28 (4) : 1545-1562. doi: 10.3934/era.2020081

[19]

Hirokazu Ninomiya. Entire solutions of the Allen–Cahn–Nagumo equation in a multi-dimensional space. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 395-412. doi: 10.3934/dcds.2020364

[20]

Jiaquan Liu, Xiangqing Liu, Zhi-Qiang Wang. Sign-changing solutions for a parameter-dependent quasilinear equation. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020454

2019 Impact Factor: 0.857

Metrics

  • PDF downloads (141)
  • HTML views (0)
  • Cited by (36)

Other articles
by authors

[Back to Top]