2015, 5(1): 97-139. doi: 10.3934/mcrf.2015.5.97

A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon

1. 

Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China, China

2. 

Department of Mathematics, University of Central Florida, Orlando, FL 32816

Received  July 2013 Revised  November 2013 Published  January 2015

A linear-quadratic (LQ, for short) optimal control problem is considered for mean-field stochastic differential equations with constant coefficients in an infinite horizon. The stabilizability of the control system is studied followed by the discussion of the well-posedness of the LQ problem. The optimal control can be expressed as a linear state feedback involving the state and its mean, through the solutions of two algebraic Riccati equations. The solvability of such kind of Riccati equations is investigated by means of semi-definite programming method.
Citation: Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control & Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97
References:
[1]

N. U. Ahmed and X. Ding, A semilinear McKean-Vlasov stochastic evolution equation in Hilbert space,, Stoch. Proc. Appl., 60 (1995), 65. doi: 10.1016/0304-4149(95)00050-X.

[2]

N. U. Ahmed and X. Ding, Controlled McKean-Vlasov equations,, Comm. Appl. Anal., 5 (2001), 183.

[3]

N. U. Ahmed, Nonlinear diffusion governed by McKean-Vlasov equation on Hilbert space and optimal control,, SIAM J. Control Optim., 46 (2007), 356. doi: 10.1137/050645944.

[4]

M. Ait Rami and X. Y. Zhou, Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls,, IEEE Transactions on Automatic Control, 45 (2000), 1131. doi: 10.1109/9.863597.

[5]

A. Albert, Conditions for positive and nonnegative definiteness in terms of pseudo-inverses,, SIAM J. Appl. Math., 17 (1969), 434. doi: 10.1137/0117041.

[6]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type,, Appl. Math. Optim., 63 (2011), 341. doi: 10.1007/s00245-010-9123-8.

[7]

A. Ben-Israel and T. N. E. Greville, Generalized Inverses,, Springer-Verlag, (2003).

[8]

A. Bensoussan, G. Da Prato, M. C. Delfour and S. K. Mitter, Representation and Control of Infinite Dimensional Systems,, 2nd edition, (2007). doi: 10.1007/978-0-8176-4581-6.

[9]

V. S. Borkar and K. S. Kumar, McKean-Vlasov limit in portfolio optimization,, Stoch. Anal. Appl., 28 (2010), 884. doi: 10.1080/07362994.2010.482836.

[10]

S. Boyd, L. El Ghaoul, E. Feron and V. Balakrishnan, Linear Matrix Inequality in Systems and Control Theory,, SIAM, (1994). doi: 10.1137/1.9781611970777.

[11]

R. Buckdahn, B. Djehiche and J. Li, A general maximum principle for SDEs of mean-field type,, Applied Mathematics & Optimization, 64 (2011), 197. doi: 10.1007/s00245-011-9136-y.

[12]

R. Buckdahn, B. Djehiche, J. Li and S. Peng, Mean-field backward stochastic differential equations: A limit approach,, Ann. Probab., 37 (2009), 1524. doi: 10.1214/08-AOP442.

[13]

R. Buckdahn, J. Li and S. Peng, Mean-field backward stochastic differential equations and related partial differential equations,, Stoch. Process. Appl., 119 (2009), 3133. doi: 10.1016/j.spa.2009.05.002.

[14]

T. Chan, Dynamics of the McKean-Vlasov equation,, Ann. Probab., 22 (1994), 431. doi: 10.1214/aop/1176988866.

[15]

T. Chiang, McKean-Vlasov equations with discontinuous coefficients,, Soochow J. Math., 20 (1994), 507.

[16]

D. Crisan and J. Xiong, Approximate McKean-Vlasov representations for a class of SPDEs,, Stochastics, 82 (2010), 53. doi: 10.1080/17442500902723575.

[17]

D. A. Dawson, Critical dynamics and fluctuations for a mean-field model of cooperative behavior,, J. Statist. Phys., 31 (1983), 29. doi: 10.1007/BF01010922.

[18]

D. A. Dawson and J. Gärtner, Large deviations from the McKean-Vlasov limit for weakly interacting diffusions,, Stochastics, 20 (1987), 247. doi: 10.1080/17442508708833446.

[19]

L. El Ghaoui and M. Ait Rami, Robust state-feedback stabilization of jump linear systems via LIMs,, Int. J. Robust and Nonlinear Contr., 6 (1996), 1015. doi: 10.1002/(SICI)1099-1239(199611)6:9/10<1015::AID-RNC266>3.0.CO;2-0.

[20]

J. Gärtner, On the Mckean-Vlasov limit for interacting diffusions,, Math. Nachr., 137 (1988), 197. doi: 10.1002/mana.19881370116.

[21]

C. Graham, McKean-Vlasov Ito-Skorohod equations, and nonlinear diffusions with discrete jump sets,, Stoch. Proc. Appl., 40 (1992), 69. doi: 10.1016/0304-4149(92)90138-G.

[22]

M. Grant and S. Boyd, Graph implementations for nonsmooth convex programs,, Recent Advances in Learning and Control (a tribute to M. Vidyasagar), 371 (2008), 95. doi: 10.1007/978-1-84800-155-8_7.

[23]

M. Huang, R. P. Malhamé, and P. E. Caines, Large population stochastic dynamic games: Closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle,, Comm. Inform. Systems, 6 (2006), 221. doi: 10.4310/CIS.2006.v6.n3.a5.

[24]

M. Kac, Foundations of kinetic theory,, Proc. 3rd Berkeley Sympos. Math. Statist. Prob., 3 (1956), 171.

[25]

P. E. Kloeden and T. Lorenz, Stochastic differential equations with nonlocal sample dependence,, Stoch. Anal. Appl., 28 (2010), 937. doi: 10.1080/07362994.2010.515194.

[26]

P. M. Kotelenez and T. G. Kurtz, Macroscopic limit for stochastic partial differential equations of McKean-Vlasov type,, Prob. Theory Rel. Fields, 146 (2010), 189. doi: 10.1007/s00440-008-0188-0.

[27]

J. M. Lasry and P. L. Lions, Mean field games,, Japan J. Math., 2 (2007), 229. doi: 10.1007/s11537-007-0657-8.

[28]

X. Li, X.Y. Zhou and M. Ait Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon,, Journal of Global Optimization, 27 (2003), 149. doi: 10.1023/A:1024887007165.

[29]

H. P. McKean, A class of Markov processes associated with nonlinear parabolic equations,, Proc. Natl. Acad. Sci. USA, 56 (1966), 1907. doi: 10.1073/pnas.56.6.1907.

[30]

T. Meyer-Brandis, B. /'Oksendal and X. Y. Zhou, A mean-field stochastic maximum principle via Malliavin calculus,, Stochastics, 84 (2012), 643. doi: 10.1080/17442508.2011.651619.

[31]

J. Y. Park, P. Balasubramaniam and Y. H. Kang, Controllability of McKean-Vlasov stochastic integrodifferential evolution equation in Hilbert spaces,, Numer. Funct. Anal. Optim., 29 (2008), 1328. doi: 10.1080/01630560802580679.

[32]

R. Penrose, A generalized inverse of matrices,, Proc. Cambridge Philos. Soc., 51 (1955), 406. doi: 10.1017/S0305004100030401.

[33]

M. Scheutzow, Uniqueness and non-uniqueness of solutions of Vlasov-McKean equations,, J. Austral. Math. Soc., 43 (1987), 246. doi: 10.1017/S1446788700029384.

[34]

R. H Tutuncu, K. C. Toh, and M. J. Todd, Solving semidefinite-quadratic-linear programs using SDPT3,, Mathematical Programming Ser. B, 95 (2003), 189. doi: 10.1007/s10107-002-0347-5.

[35]

L. Vandenerghe and S. Boyd, Semidefinite programming,, SIAM Rev., 38 (1996), 49. doi: 10.1137/1038003.

[36]

A. Yu. Veretennikov, On ergodic measures for McKean-Vlasov stochastic equations,, in Monte Carlo and quasi-Monte Carlo methods 2004, (2004), 471. doi: 10.1007/3-540-31186-6_29.

[37]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations,, SIAM J. Control Optim., 51 (2013), 2809. doi: 10.1137/120892477.

[38]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations,, Applications of Mathematics (New York), (1999). doi: 10.1007/978-1-4612-1466-3.

show all references

References:
[1]

N. U. Ahmed and X. Ding, A semilinear McKean-Vlasov stochastic evolution equation in Hilbert space,, Stoch. Proc. Appl., 60 (1995), 65. doi: 10.1016/0304-4149(95)00050-X.

[2]

N. U. Ahmed and X. Ding, Controlled McKean-Vlasov equations,, Comm. Appl. Anal., 5 (2001), 183.

[3]

N. U. Ahmed, Nonlinear diffusion governed by McKean-Vlasov equation on Hilbert space and optimal control,, SIAM J. Control Optim., 46 (2007), 356. doi: 10.1137/050645944.

[4]

M. Ait Rami and X. Y. Zhou, Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls,, IEEE Transactions on Automatic Control, 45 (2000), 1131. doi: 10.1109/9.863597.

[5]

A. Albert, Conditions for positive and nonnegative definiteness in terms of pseudo-inverses,, SIAM J. Appl. Math., 17 (1969), 434. doi: 10.1137/0117041.

[6]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type,, Appl. Math. Optim., 63 (2011), 341. doi: 10.1007/s00245-010-9123-8.

[7]

A. Ben-Israel and T. N. E. Greville, Generalized Inverses,, Springer-Verlag, (2003).

[8]

A. Bensoussan, G. Da Prato, M. C. Delfour and S. K. Mitter, Representation and Control of Infinite Dimensional Systems,, 2nd edition, (2007). doi: 10.1007/978-0-8176-4581-6.

[9]

V. S. Borkar and K. S. Kumar, McKean-Vlasov limit in portfolio optimization,, Stoch. Anal. Appl., 28 (2010), 884. doi: 10.1080/07362994.2010.482836.

[10]

S. Boyd, L. El Ghaoul, E. Feron and V. Balakrishnan, Linear Matrix Inequality in Systems and Control Theory,, SIAM, (1994). doi: 10.1137/1.9781611970777.

[11]

R. Buckdahn, B. Djehiche and J. Li, A general maximum principle for SDEs of mean-field type,, Applied Mathematics & Optimization, 64 (2011), 197. doi: 10.1007/s00245-011-9136-y.

[12]

R. Buckdahn, B. Djehiche, J. Li and S. Peng, Mean-field backward stochastic differential equations: A limit approach,, Ann. Probab., 37 (2009), 1524. doi: 10.1214/08-AOP442.

[13]

R. Buckdahn, J. Li and S. Peng, Mean-field backward stochastic differential equations and related partial differential equations,, Stoch. Process. Appl., 119 (2009), 3133. doi: 10.1016/j.spa.2009.05.002.

[14]

T. Chan, Dynamics of the McKean-Vlasov equation,, Ann. Probab., 22 (1994), 431. doi: 10.1214/aop/1176988866.

[15]

T. Chiang, McKean-Vlasov equations with discontinuous coefficients,, Soochow J. Math., 20 (1994), 507.

[16]

D. Crisan and J. Xiong, Approximate McKean-Vlasov representations for a class of SPDEs,, Stochastics, 82 (2010), 53. doi: 10.1080/17442500902723575.

[17]

D. A. Dawson, Critical dynamics and fluctuations for a mean-field model of cooperative behavior,, J. Statist. Phys., 31 (1983), 29. doi: 10.1007/BF01010922.

[18]

D. A. Dawson and J. Gärtner, Large deviations from the McKean-Vlasov limit for weakly interacting diffusions,, Stochastics, 20 (1987), 247. doi: 10.1080/17442508708833446.

[19]

L. El Ghaoui and M. Ait Rami, Robust state-feedback stabilization of jump linear systems via LIMs,, Int. J. Robust and Nonlinear Contr., 6 (1996), 1015. doi: 10.1002/(SICI)1099-1239(199611)6:9/10<1015::AID-RNC266>3.0.CO;2-0.

[20]

J. Gärtner, On the Mckean-Vlasov limit for interacting diffusions,, Math. Nachr., 137 (1988), 197. doi: 10.1002/mana.19881370116.

[21]

C. Graham, McKean-Vlasov Ito-Skorohod equations, and nonlinear diffusions with discrete jump sets,, Stoch. Proc. Appl., 40 (1992), 69. doi: 10.1016/0304-4149(92)90138-G.

[22]

M. Grant and S. Boyd, Graph implementations for nonsmooth convex programs,, Recent Advances in Learning and Control (a tribute to M. Vidyasagar), 371 (2008), 95. doi: 10.1007/978-1-84800-155-8_7.

[23]

M. Huang, R. P. Malhamé, and P. E. Caines, Large population stochastic dynamic games: Closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle,, Comm. Inform. Systems, 6 (2006), 221. doi: 10.4310/CIS.2006.v6.n3.a5.

[24]

M. Kac, Foundations of kinetic theory,, Proc. 3rd Berkeley Sympos. Math. Statist. Prob., 3 (1956), 171.

[25]

P. E. Kloeden and T. Lorenz, Stochastic differential equations with nonlocal sample dependence,, Stoch. Anal. Appl., 28 (2010), 937. doi: 10.1080/07362994.2010.515194.

[26]

P. M. Kotelenez and T. G. Kurtz, Macroscopic limit for stochastic partial differential equations of McKean-Vlasov type,, Prob. Theory Rel. Fields, 146 (2010), 189. doi: 10.1007/s00440-008-0188-0.

[27]

J. M. Lasry and P. L. Lions, Mean field games,, Japan J. Math., 2 (2007), 229. doi: 10.1007/s11537-007-0657-8.

[28]

X. Li, X.Y. Zhou and M. Ait Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon,, Journal of Global Optimization, 27 (2003), 149. doi: 10.1023/A:1024887007165.

[29]

H. P. McKean, A class of Markov processes associated with nonlinear parabolic equations,, Proc. Natl. Acad. Sci. USA, 56 (1966), 1907. doi: 10.1073/pnas.56.6.1907.

[30]

T. Meyer-Brandis, B. /'Oksendal and X. Y. Zhou, A mean-field stochastic maximum principle via Malliavin calculus,, Stochastics, 84 (2012), 643. doi: 10.1080/17442508.2011.651619.

[31]

J. Y. Park, P. Balasubramaniam and Y. H. Kang, Controllability of McKean-Vlasov stochastic integrodifferential evolution equation in Hilbert spaces,, Numer. Funct. Anal. Optim., 29 (2008), 1328. doi: 10.1080/01630560802580679.

[32]

R. Penrose, A generalized inverse of matrices,, Proc. Cambridge Philos. Soc., 51 (1955), 406. doi: 10.1017/S0305004100030401.

[33]

M. Scheutzow, Uniqueness and non-uniqueness of solutions of Vlasov-McKean equations,, J. Austral. Math. Soc., 43 (1987), 246. doi: 10.1017/S1446788700029384.

[34]

R. H Tutuncu, K. C. Toh, and M. J. Todd, Solving semidefinite-quadratic-linear programs using SDPT3,, Mathematical Programming Ser. B, 95 (2003), 189. doi: 10.1007/s10107-002-0347-5.

[35]

L. Vandenerghe and S. Boyd, Semidefinite programming,, SIAM Rev., 38 (1996), 49. doi: 10.1137/1038003.

[36]

A. Yu. Veretennikov, On ergodic measures for McKean-Vlasov stochastic equations,, in Monte Carlo and quasi-Monte Carlo methods 2004, (2004), 471. doi: 10.1007/3-540-31186-6_29.

[37]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations,, SIAM J. Control Optim., 51 (2013), 2809. doi: 10.1137/120892477.

[38]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations,, Applications of Mathematics (New York), (1999). doi: 10.1007/978-1-4612-1466-3.

[1]

Kai Du, Jianhui Huang, Zhen Wu. Linear quadratic mean-field-game of backward stochastic differential systems. Mathematical Control & Related Fields, 2018, 8 (3&4) : 653-678. doi: 10.3934/mcrf.2018028

[2]

Martino Bardi. Explicit solutions of some linear-quadratic mean field games. Networks & Heterogeneous Media, 2012, 7 (2) : 243-261. doi: 10.3934/nhm.2012.7.243

[3]

Galina Kurina, Sahlar Meherrem. Decomposition of discrete linear-quadratic optimal control problems for switching systems. Conference Publications, 2015, 2015 (special) : 764-774. doi: 10.3934/proc.2015.0764

[4]

Thierry Horsin, Peter I. Kogut, Olivier Wilk. Optimal $L^2$-control problem in coefficients for a linear elliptic equation. II. Approximation of solutions and optimality conditions. Mathematical Control & Related Fields, 2016, 6 (4) : 595-628. doi: 10.3934/mcrf.2016017

[5]

Thierry Horsin, Peter I. Kogut. Optimal $L^2$-control problem in coefficients for a linear elliptic equation. I. Existence result. Mathematical Control & Related Fields, 2015, 5 (1) : 73-96. doi: 10.3934/mcrf.2015.5.73

[6]

Michael Herty, Lorenzo Pareschi, Sonja Steffensen. Mean--field control and Riccati equations. Networks & Heterogeneous Media, 2015, 10 (3) : 699-715. doi: 10.3934/nhm.2015.10.699

[7]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437

[8]

Haiyan Zhang. A necessary condition for mean-field type stochastic differential equations with correlated state and observation noises. Journal of Industrial & Management Optimization, 2016, 12 (4) : 1287-1301. doi: 10.3934/jimo.2016.12.1287

[9]

Juan Li, Wenqiang Li. Controlled reflected mean-field backward stochastic differential equations coupled with value function and related PDEs. Mathematical Control & Related Fields, 2015, 5 (3) : 501-516. doi: 10.3934/mcrf.2015.5.501

[10]

Hancheng Guo, Jie Xiong. A second-order stochastic maximum principle for generalized mean-field singular control problem. Mathematical Control & Related Fields, 2018, 8 (2) : 451-473. doi: 10.3934/mcrf.2018018

[11]

Shigeaki Koike, Hiroaki Morimoto, Shigeru Sakaguchi. A linear-quadratic control problem with discretionary stopping. Discrete & Continuous Dynamical Systems - B, 2007, 8 (2) : 261-277. doi: 10.3934/dcdsb.2007.8.261

[12]

Russell Johnson, Carmen Núñez. Remarks on linear-quadratic dissipative control systems. Discrete & Continuous Dynamical Systems - B, 2015, 20 (3) : 889-914. doi: 10.3934/dcdsb.2015.20.889

[13]

Tyrone E. Duncan. Some linear-quadratic stochastic differential games for equations in Hilbert spaces with fractional Brownian motions. Discrete & Continuous Dynamical Systems - A, 2015, 35 (11) : 5435-5445. doi: 10.3934/dcds.2015.35.5435

[14]

Georg Vossen, Stefan Volkwein. Model reduction techniques with a-posteriori error analysis for linear-quadratic optimal control problems. Numerical Algebra, Control & Optimization, 2012, 2 (3) : 465-485. doi: 10.3934/naco.2012.2.465

[15]

Rong Yang, Li Chen. Mean-field limit for a collision-avoiding flocking system and the time-asymptotic flocking dynamics for the kinetic equation. Kinetic & Related Models, 2014, 7 (2) : 381-400. doi: 10.3934/krm.2014.7.381

[16]

Yufeng Shi, Tianxiao Wang, Jiongmin Yong. Mean-field backward stochastic Volterra integral equations. Discrete & Continuous Dynamical Systems - B, 2013, 18 (7) : 1929-1967. doi: 10.3934/dcdsb.2013.18.1929

[17]

Roberta Fabbri, Russell Johnson, Sylvia Novo, Carmen Núñez. On linear-quadratic dissipative control processes with time-varying coefficients. Discrete & Continuous Dynamical Systems - A, 2013, 33 (1) : 193-210. doi: 10.3934/dcds.2013.33.193

[18]

Fulvia Confortola, Elisa Mastrogiacomo. Optimal control for stochastic heat equation with memory. Evolution Equations & Control Theory, 2014, 3 (1) : 35-58. doi: 10.3934/eect.2014.3.35

[19]

Patrick Gerard, Christophe Pallard. A mean-field toy model for resonant transport. Kinetic & Related Models, 2010, 3 (2) : 299-309. doi: 10.3934/krm.2010.3.299

[20]

Tijana Levajković, Hermann Mena, Amjad Tuffaha. The stochastic linear quadratic optimal control problem in Hilbert spaces: A polynomial chaos approach. Evolution Equations & Control Theory, 2016, 5 (1) : 105-134. doi: 10.3934/eect.2016.5.105

2017 Impact Factor: 0.631

Metrics

  • PDF downloads (19)
  • HTML views (0)
  • Cited by (13)

Other articles
by authors

[Back to Top]