March  2015, 5(1): 97-139. doi: 10.3934/mcrf.2015.5.97

A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon

1. 

Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China, China

2. 

Department of Mathematics, University of Central Florida, Orlando, FL 32816

Received  July 2013 Revised  November 2013 Published  January 2015

A linear-quadratic (LQ, for short) optimal control problem is considered for mean-field stochastic differential equations with constant coefficients in an infinite horizon. The stabilizability of the control system is studied followed by the discussion of the well-posedness of the LQ problem. The optimal control can be expressed as a linear state feedback involving the state and its mean, through the solutions of two algebraic Riccati equations. The solvability of such kind of Riccati equations is investigated by means of semi-definite programming method.
Citation: Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control & Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97
References:
[1]

N. U. Ahmed and X. Ding, A semilinear McKean-Vlasov stochastic evolution equation in Hilbert space,, Stoch. Proc. Appl., 60 (1995), 65.  doi: 10.1016/0304-4149(95)00050-X.  Google Scholar

[2]

N. U. Ahmed and X. Ding, Controlled McKean-Vlasov equations,, Comm. Appl. Anal., 5 (2001), 183.   Google Scholar

[3]

N. U. Ahmed, Nonlinear diffusion governed by McKean-Vlasov equation on Hilbert space and optimal control,, SIAM J. Control Optim., 46 (2007), 356.  doi: 10.1137/050645944.  Google Scholar

[4]

M. Ait Rami and X. Y. Zhou, Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls,, IEEE Transactions on Automatic Control, 45 (2000), 1131.  doi: 10.1109/9.863597.  Google Scholar

[5]

A. Albert, Conditions for positive and nonnegative definiteness in terms of pseudo-inverses,, SIAM J. Appl. Math., 17 (1969), 434.  doi: 10.1137/0117041.  Google Scholar

[6]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type,, Appl. Math. Optim., 63 (2011), 341.  doi: 10.1007/s00245-010-9123-8.  Google Scholar

[7]

A. Ben-Israel and T. N. E. Greville, Generalized Inverses,, Springer-Verlag, (2003).   Google Scholar

[8]

A. Bensoussan, G. Da Prato, M. C. Delfour and S. K. Mitter, Representation and Control of Infinite Dimensional Systems,, 2nd edition, (2007).  doi: 10.1007/978-0-8176-4581-6.  Google Scholar

[9]

V. S. Borkar and K. S. Kumar, McKean-Vlasov limit in portfolio optimization,, Stoch. Anal. Appl., 28 (2010), 884.  doi: 10.1080/07362994.2010.482836.  Google Scholar

[10]

S. Boyd, L. El Ghaoul, E. Feron and V. Balakrishnan, Linear Matrix Inequality in Systems and Control Theory,, SIAM, (1994).  doi: 10.1137/1.9781611970777.  Google Scholar

[11]

R. Buckdahn, B. Djehiche and J. Li, A general maximum principle for SDEs of mean-field type,, Applied Mathematics & Optimization, 64 (2011), 197.  doi: 10.1007/s00245-011-9136-y.  Google Scholar

[12]

R. Buckdahn, B. Djehiche, J. Li and S. Peng, Mean-field backward stochastic differential equations: A limit approach,, Ann. Probab., 37 (2009), 1524.  doi: 10.1214/08-AOP442.  Google Scholar

[13]

R. Buckdahn, J. Li and S. Peng, Mean-field backward stochastic differential equations and related partial differential equations,, Stoch. Process. Appl., 119 (2009), 3133.  doi: 10.1016/j.spa.2009.05.002.  Google Scholar

[14]

T. Chan, Dynamics of the McKean-Vlasov equation,, Ann. Probab., 22 (1994), 431.  doi: 10.1214/aop/1176988866.  Google Scholar

[15]

T. Chiang, McKean-Vlasov equations with discontinuous coefficients,, Soochow J. Math., 20 (1994), 507.   Google Scholar

[16]

D. Crisan and J. Xiong, Approximate McKean-Vlasov representations for a class of SPDEs,, Stochastics, 82 (2010), 53.  doi: 10.1080/17442500902723575.  Google Scholar

[17]

D. A. Dawson, Critical dynamics and fluctuations for a mean-field model of cooperative behavior,, J. Statist. Phys., 31 (1983), 29.  doi: 10.1007/BF01010922.  Google Scholar

[18]

D. A. Dawson and J. Gärtner, Large deviations from the McKean-Vlasov limit for weakly interacting diffusions,, Stochastics, 20 (1987), 247.  doi: 10.1080/17442508708833446.  Google Scholar

[19]

L. El Ghaoui and M. Ait Rami, Robust state-feedback stabilization of jump linear systems via LIMs,, Int. J. Robust and Nonlinear Contr., 6 (1996), 1015.  doi: 10.1002/(SICI)1099-1239(199611)6:9/10<1015::AID-RNC266>3.0.CO;2-0.  Google Scholar

[20]

J. Gärtner, On the Mckean-Vlasov limit for interacting diffusions,, Math. Nachr., 137 (1988), 197.  doi: 10.1002/mana.19881370116.  Google Scholar

[21]

C. Graham, McKean-Vlasov Ito-Skorohod equations, and nonlinear diffusions with discrete jump sets,, Stoch. Proc. Appl., 40 (1992), 69.  doi: 10.1016/0304-4149(92)90138-G.  Google Scholar

[22]

M. Grant and S. Boyd, Graph implementations for nonsmooth convex programs,, Recent Advances in Learning and Control (a tribute to M. Vidyasagar), 371 (2008), 95.  doi: 10.1007/978-1-84800-155-8_7.  Google Scholar

[23]

M. Huang, R. P. Malhamé, and P. E. Caines, Large population stochastic dynamic games: Closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle,, Comm. Inform. Systems, 6 (2006), 221.  doi: 10.4310/CIS.2006.v6.n3.a5.  Google Scholar

[24]

M. Kac, Foundations of kinetic theory,, Proc. 3rd Berkeley Sympos. Math. Statist. Prob., 3 (1956), 171.   Google Scholar

[25]

P. E. Kloeden and T. Lorenz, Stochastic differential equations with nonlocal sample dependence,, Stoch. Anal. Appl., 28 (2010), 937.  doi: 10.1080/07362994.2010.515194.  Google Scholar

[26]

P. M. Kotelenez and T. G. Kurtz, Macroscopic limit for stochastic partial differential equations of McKean-Vlasov type,, Prob. Theory Rel. Fields, 146 (2010), 189.  doi: 10.1007/s00440-008-0188-0.  Google Scholar

[27]

J. M. Lasry and P. L. Lions, Mean field games,, Japan J. Math., 2 (2007), 229.  doi: 10.1007/s11537-007-0657-8.  Google Scholar

[28]

X. Li, X.Y. Zhou and M. Ait Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon,, Journal of Global Optimization, 27 (2003), 149.  doi: 10.1023/A:1024887007165.  Google Scholar

[29]

H. P. McKean, A class of Markov processes associated with nonlinear parabolic equations,, Proc. Natl. Acad. Sci. USA, 56 (1966), 1907.  doi: 10.1073/pnas.56.6.1907.  Google Scholar

[30]

T. Meyer-Brandis, B. /'Oksendal and X. Y. Zhou, A mean-field stochastic maximum principle via Malliavin calculus,, Stochastics, 84 (2012), 643.  doi: 10.1080/17442508.2011.651619.  Google Scholar

[31]

J. Y. Park, P. Balasubramaniam and Y. H. Kang, Controllability of McKean-Vlasov stochastic integrodifferential evolution equation in Hilbert spaces,, Numer. Funct. Anal. Optim., 29 (2008), 1328.  doi: 10.1080/01630560802580679.  Google Scholar

[32]

R. Penrose, A generalized inverse of matrices,, Proc. Cambridge Philos. Soc., 51 (1955), 406.  doi: 10.1017/S0305004100030401.  Google Scholar

[33]

M. Scheutzow, Uniqueness and non-uniqueness of solutions of Vlasov-McKean equations,, J. Austral. Math. Soc., 43 (1987), 246.  doi: 10.1017/S1446788700029384.  Google Scholar

[34]

R. H Tutuncu, K. C. Toh, and M. J. Todd, Solving semidefinite-quadratic-linear programs using SDPT3,, Mathematical Programming Ser. B, 95 (2003), 189.  doi: 10.1007/s10107-002-0347-5.  Google Scholar

[35]

L. Vandenerghe and S. Boyd, Semidefinite programming,, SIAM Rev., 38 (1996), 49.  doi: 10.1137/1038003.  Google Scholar

[36]

A. Yu. Veretennikov, On ergodic measures for McKean-Vlasov stochastic equations,, in Monte Carlo and quasi-Monte Carlo methods 2004, (2004), 471.  doi: 10.1007/3-540-31186-6_29.  Google Scholar

[37]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations,, SIAM J. Control Optim., 51 (2013), 2809.  doi: 10.1137/120892477.  Google Scholar

[38]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations,, Applications of Mathematics (New York), (1999).  doi: 10.1007/978-1-4612-1466-3.  Google Scholar

show all references

References:
[1]

N. U. Ahmed and X. Ding, A semilinear McKean-Vlasov stochastic evolution equation in Hilbert space,, Stoch. Proc. Appl., 60 (1995), 65.  doi: 10.1016/0304-4149(95)00050-X.  Google Scholar

[2]

N. U. Ahmed and X. Ding, Controlled McKean-Vlasov equations,, Comm. Appl. Anal., 5 (2001), 183.   Google Scholar

[3]

N. U. Ahmed, Nonlinear diffusion governed by McKean-Vlasov equation on Hilbert space and optimal control,, SIAM J. Control Optim., 46 (2007), 356.  doi: 10.1137/050645944.  Google Scholar

[4]

M. Ait Rami and X. Y. Zhou, Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls,, IEEE Transactions on Automatic Control, 45 (2000), 1131.  doi: 10.1109/9.863597.  Google Scholar

[5]

A. Albert, Conditions for positive and nonnegative definiteness in terms of pseudo-inverses,, SIAM J. Appl. Math., 17 (1969), 434.  doi: 10.1137/0117041.  Google Scholar

[6]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type,, Appl. Math. Optim., 63 (2011), 341.  doi: 10.1007/s00245-010-9123-8.  Google Scholar

[7]

A. Ben-Israel and T. N. E. Greville, Generalized Inverses,, Springer-Verlag, (2003).   Google Scholar

[8]

A. Bensoussan, G. Da Prato, M. C. Delfour and S. K. Mitter, Representation and Control of Infinite Dimensional Systems,, 2nd edition, (2007).  doi: 10.1007/978-0-8176-4581-6.  Google Scholar

[9]

V. S. Borkar and K. S. Kumar, McKean-Vlasov limit in portfolio optimization,, Stoch. Anal. Appl., 28 (2010), 884.  doi: 10.1080/07362994.2010.482836.  Google Scholar

[10]

S. Boyd, L. El Ghaoul, E. Feron and V. Balakrishnan, Linear Matrix Inequality in Systems and Control Theory,, SIAM, (1994).  doi: 10.1137/1.9781611970777.  Google Scholar

[11]

R. Buckdahn, B. Djehiche and J. Li, A general maximum principle for SDEs of mean-field type,, Applied Mathematics & Optimization, 64 (2011), 197.  doi: 10.1007/s00245-011-9136-y.  Google Scholar

[12]

R. Buckdahn, B. Djehiche, J. Li and S. Peng, Mean-field backward stochastic differential equations: A limit approach,, Ann. Probab., 37 (2009), 1524.  doi: 10.1214/08-AOP442.  Google Scholar

[13]

R. Buckdahn, J. Li and S. Peng, Mean-field backward stochastic differential equations and related partial differential equations,, Stoch. Process. Appl., 119 (2009), 3133.  doi: 10.1016/j.spa.2009.05.002.  Google Scholar

[14]

T. Chan, Dynamics of the McKean-Vlasov equation,, Ann. Probab., 22 (1994), 431.  doi: 10.1214/aop/1176988866.  Google Scholar

[15]

T. Chiang, McKean-Vlasov equations with discontinuous coefficients,, Soochow J. Math., 20 (1994), 507.   Google Scholar

[16]

D. Crisan and J. Xiong, Approximate McKean-Vlasov representations for a class of SPDEs,, Stochastics, 82 (2010), 53.  doi: 10.1080/17442500902723575.  Google Scholar

[17]

D. A. Dawson, Critical dynamics and fluctuations for a mean-field model of cooperative behavior,, J. Statist. Phys., 31 (1983), 29.  doi: 10.1007/BF01010922.  Google Scholar

[18]

D. A. Dawson and J. Gärtner, Large deviations from the McKean-Vlasov limit for weakly interacting diffusions,, Stochastics, 20 (1987), 247.  doi: 10.1080/17442508708833446.  Google Scholar

[19]

L. El Ghaoui and M. Ait Rami, Robust state-feedback stabilization of jump linear systems via LIMs,, Int. J. Robust and Nonlinear Contr., 6 (1996), 1015.  doi: 10.1002/(SICI)1099-1239(199611)6:9/10<1015::AID-RNC266>3.0.CO;2-0.  Google Scholar

[20]

J. Gärtner, On the Mckean-Vlasov limit for interacting diffusions,, Math. Nachr., 137 (1988), 197.  doi: 10.1002/mana.19881370116.  Google Scholar

[21]

C. Graham, McKean-Vlasov Ito-Skorohod equations, and nonlinear diffusions with discrete jump sets,, Stoch. Proc. Appl., 40 (1992), 69.  doi: 10.1016/0304-4149(92)90138-G.  Google Scholar

[22]

M. Grant and S. Boyd, Graph implementations for nonsmooth convex programs,, Recent Advances in Learning and Control (a tribute to M. Vidyasagar), 371 (2008), 95.  doi: 10.1007/978-1-84800-155-8_7.  Google Scholar

[23]

M. Huang, R. P. Malhamé, and P. E. Caines, Large population stochastic dynamic games: Closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle,, Comm. Inform. Systems, 6 (2006), 221.  doi: 10.4310/CIS.2006.v6.n3.a5.  Google Scholar

[24]

M. Kac, Foundations of kinetic theory,, Proc. 3rd Berkeley Sympos. Math. Statist. Prob., 3 (1956), 171.   Google Scholar

[25]

P. E. Kloeden and T. Lorenz, Stochastic differential equations with nonlocal sample dependence,, Stoch. Anal. Appl., 28 (2010), 937.  doi: 10.1080/07362994.2010.515194.  Google Scholar

[26]

P. M. Kotelenez and T. G. Kurtz, Macroscopic limit for stochastic partial differential equations of McKean-Vlasov type,, Prob. Theory Rel. Fields, 146 (2010), 189.  doi: 10.1007/s00440-008-0188-0.  Google Scholar

[27]

J. M. Lasry and P. L. Lions, Mean field games,, Japan J. Math., 2 (2007), 229.  doi: 10.1007/s11537-007-0657-8.  Google Scholar

[28]

X. Li, X.Y. Zhou and M. Ait Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon,, Journal of Global Optimization, 27 (2003), 149.  doi: 10.1023/A:1024887007165.  Google Scholar

[29]

H. P. McKean, A class of Markov processes associated with nonlinear parabolic equations,, Proc. Natl. Acad. Sci. USA, 56 (1966), 1907.  doi: 10.1073/pnas.56.6.1907.  Google Scholar

[30]

T. Meyer-Brandis, B. /'Oksendal and X. Y. Zhou, A mean-field stochastic maximum principle via Malliavin calculus,, Stochastics, 84 (2012), 643.  doi: 10.1080/17442508.2011.651619.  Google Scholar

[31]

J. Y. Park, P. Balasubramaniam and Y. H. Kang, Controllability of McKean-Vlasov stochastic integrodifferential evolution equation in Hilbert spaces,, Numer. Funct. Anal. Optim., 29 (2008), 1328.  doi: 10.1080/01630560802580679.  Google Scholar

[32]

R. Penrose, A generalized inverse of matrices,, Proc. Cambridge Philos. Soc., 51 (1955), 406.  doi: 10.1017/S0305004100030401.  Google Scholar

[33]

M. Scheutzow, Uniqueness and non-uniqueness of solutions of Vlasov-McKean equations,, J. Austral. Math. Soc., 43 (1987), 246.  doi: 10.1017/S1446788700029384.  Google Scholar

[34]

R. H Tutuncu, K. C. Toh, and M. J. Todd, Solving semidefinite-quadratic-linear programs using SDPT3,, Mathematical Programming Ser. B, 95 (2003), 189.  doi: 10.1007/s10107-002-0347-5.  Google Scholar

[35]

L. Vandenerghe and S. Boyd, Semidefinite programming,, SIAM Rev., 38 (1996), 49.  doi: 10.1137/1038003.  Google Scholar

[36]

A. Yu. Veretennikov, On ergodic measures for McKean-Vlasov stochastic equations,, in Monte Carlo and quasi-Monte Carlo methods 2004, (2004), 471.  doi: 10.1007/3-540-31186-6_29.  Google Scholar

[37]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations,, SIAM J. Control Optim., 51 (2013), 2809.  doi: 10.1137/120892477.  Google Scholar

[38]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations,, Applications of Mathematics (New York), (1999).  doi: 10.1007/978-1-4612-1466-3.  Google Scholar

[1]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437

[2]

Marco Cirant, Diogo A. Gomes, Edgard A. Pimentel, Héctor Sánchez-Morgado. On some singular mean-field games. Journal of Dynamics & Games, 2021  doi: 10.3934/jdg.2021006

[3]

Seung-Yeal Ha, Jinwook Jung, Jeongho Kim, Jinyeong Park, Xiongtao Zhang. A mean-field limit of the particle swarmalator model. Kinetic & Related Models, , () : -. doi: 10.3934/krm.2021011

[4]

Nhu N. Nguyen, George Yin. Stochastic partial differential equation models for spatially dependent predator-prey equations. Discrete & Continuous Dynamical Systems - B, 2020, 25 (1) : 117-139. doi: 10.3934/dcdsb.2019175

[5]

Bin Pei, Yong Xu, Yuzhen Bai. Convergence of p-th mean in an averaging principle for stochastic partial differential equations driven by fractional Brownian motion. Discrete & Continuous Dynamical Systems - B, 2020, 25 (3) : 1141-1158. doi: 10.3934/dcdsb.2019213

[6]

Luke Finlay, Vladimir Gaitsgory, Ivan Lebedev. Linear programming solutions of periodic optimization problems: approximation of the optimal control. Journal of Industrial & Management Optimization, 2007, 3 (2) : 399-413. doi: 10.3934/jimo.2007.3.399

[7]

Shanjian Tang, Fu Zhang. Path-dependent optimal stochastic control and viscosity solution of associated Bellman equations. Discrete & Continuous Dynamical Systems - A, 2015, 35 (11) : 5521-5553. doi: 10.3934/dcds.2015.35.5521

[8]

Abdulrazzaq T. Abed, Azzam S. Y. Aladool. Applying particle swarm optimization based on Padé approximant to solve ordinary differential equation. Numerical Algebra, Control & Optimization, 2021  doi: 10.3934/naco.2021008

[9]

Changpin Li, Zhiqiang Li. Asymptotic behaviors of solution to partial differential equation with Caputo–Hadamard derivative and fractional Laplacian: Hyperbolic case. Discrete & Continuous Dynamical Systems - S, 2021  doi: 10.3934/dcdss.2021023

[10]

Christophe Zhang. Internal rapid stabilization of a 1-D linear transport equation with a scalar feedback. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021006

[11]

Marita Holtmannspötter, Arnd Rösch, Boris Vexler. A priori error estimates for the space-time finite element discretization of an optimal control problem governed by a coupled linear PDE-ODE system. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021014

[12]

Zhiming Guo, Zhi-Chun Yang, Xingfu Zou. Existence and uniqueness of positive solution to a non-local differential equation with homogeneous Dirichlet boundary condition---A non-monotone case. Communications on Pure & Applied Analysis, 2012, 11 (5) : 1825-1838. doi: 10.3934/cpaa.2012.11.1825

[13]

Tobias Geiger, Daniel Wachsmuth, Gerd Wachsmuth. Optimal control of ODEs with state suprema. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021012

[14]

Nizami A. Gasilov. Solving a system of linear differential equations with interval coefficients. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2739-2747. doi: 10.3934/dcdsb.2020203

[15]

J. Frédéric Bonnans, Justina Gianatti, Francisco J. Silva. On the convergence of the Sakawa-Shindo algorithm in stochastic control. Mathematical Control & Related Fields, 2016, 6 (3) : 391-406. doi: 10.3934/mcrf.2016008

[16]

Lorenzo Freddi. Optimal control of the transmission rate in compartmental epidemics. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021007

[17]

Vladimir Georgiev, Sandra Lucente. Focusing nlkg equation with singular potential. Communications on Pure & Applied Analysis, 2018, 17 (4) : 1387-1406. doi: 10.3934/cpaa.2018068

[18]

Daoyin He, Ingo Witt, Huicheng Yin. On the strauss index of semilinear tricomi equation. Communications on Pure & Applied Analysis, 2020, 19 (10) : 4817-4838. doi: 10.3934/cpaa.2020213

[19]

Carmen Cortázar, M. García-Huidobro, Pilar Herreros, Satoshi Tanaka. On the uniqueness of solutions of a semilinear equation in an annulus. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2021029

[20]

Xianming Liu, Guangyue Han. A Wong-Zakai approximation of stochastic differential equations driven by a general semimartingale. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2499-2508. doi: 10.3934/dcdsb.2020192

2019 Impact Factor: 0.857

Metrics

  • PDF downloads (165)
  • HTML views (0)
  • Cited by (39)

Other articles
by authors

[Back to Top]