American Institute of Mathematical Sciences

Advanced
doi: 10.3934/jimo.2021074

Open-loop solvability for mean-field stochastic linear quadratic optimal control problems of Markov regime-switching system

Kehan Si 1, , Zhenda Xu 1, , Ka Fai Cedric Yiu 2,, and  Xun Li 2,

1. 

School of Mathematics, Shandong University, Jinan, Shandong Province, 250100, China
2. 

Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China

* Corresponding author: Ka Fai Cedric Yiu

Received  September 2020 Revised  January 2021 Published  April 2021

This paper investigates the mean-field stochastic linear quadratic optimal control problem of Markov regime switching system (M-MF-SLQ, for short). The representation of the cost functional for the M-MF-SLQ is derived using the technique of operators. It is shown that the convexity of the cost functional is necessary for the finiteness of the M-MF-SLQ problem, whereas uniform convexity of the cost functional is sufficient for the open-loop solvability of the problem. By considering a family of uniformly convex cost functionals, a characterization of the finiteness of the problem is derived and a minimizing sequence, whose convergence is equivalent to the open-loop solvability of the problem, is constructed. We demonstrate with a few examples that our results can be employed for tackling some financial problems such as mean-variance portfolio selection problem.

Keywords: Mean-field, linear quadratic optimal control, Markov regime switching, open-loop solvability.
Mathematics Subject Classification: Primary: 49N10, 49N35; Secondary: 93E20.
Citation: Kehan Si, Zhenda Xu, Ka Fai Cedric Yiu, Xun Li. Open-loop solvability for mean-field stochastic linear quadratic optimal control problems of Markov regime-switching system. Journal of Industrial & Management Optimization, doi: 10.3934/jimo.2021074
References:
[1]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type, Applied Mathematics and Optimization, 63 (2011), 341-356.  doi: 10.1007/s00245-010-9123-8.  Google Scholar

[2]

J.-M. Bismut, Linear quadratic optimal stochastic control with random coefficients, SIAM Journal on Control and Optimization, 14 (1976), 419–444. doi: 10.1137/0314028.  Google Scholar

[3]

R. Buckdahn, B. Djehiche and J. Li, A general stochastic maximum principle for SDEs of mean-field type, Applied Mathematics and Optimization, 64 (2011), 197-216.  doi: 10.1007/s00245-011-9136-y.  Google Scholar

[4]

R. Buckdahn, B. Djehiche, J. Li and S. Peng, Mean-field backward stochastic differential equations: A limit approach, Annals of Probability, 37 (2009), 1524-1565.  doi: 10.1214/08-AOP442.  Google Scholar

[5]

X. Cui, X. Li and D. Li, Unified framework of mean-field formulations for optimal multi-period mean-variance portfolio selection, IEEE Transactions on Automatic Control, 59 (2014), 1833–1844. doi: 10.1109/TAC.2014.2311875.  Google Scholar

[6]

C. Donnelly, Suffcient stochastic maximum principle in a regime-switching diffusion model, Applied Mathematics & Optimization, 64 (2011), 155-169.  doi: 10.1007/s00245-010-9130-9.  Google Scholar

[7]

C. Donnelly and A. J. Heunis, Quadratic Risk Minimization in a Regime-Switching Model with Portfolio Constraints, SIAM Journal on Control and Optimization, 50 (2012), 2431–2461. doi: 10.1137/100809271.  Google Scholar

[8]

R. J. Elliott, L. Aggoun and J. B. Moore, Hidden Markov Models: Estimation and Control, Applications of Mathematics, New York: Springer-Verlag, 1995.  Google Scholar

[9]

R. Elliott, X. Li and Y.-H. Ni, Discrete time mean-field stochastic linear-quadratic optimal control problems, Automatica, 49 (2013), 3222-3233.  doi: 10.1016/j.automatica.2013.08.017.  Google Scholar

[10]

J. Huang, X. Li and J. Yong, A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon, Mathematical Control & Related Fields, 5 (2015), 97-139.  doi: 10.3934/mcrf.2015.5.97.  Google Scholar

[11]

M. Kac, Foundations of kinetic theory, Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 3 (1956), 171-197.   Google Scholar

[12]

H. Kushner, Optimal stochastic control, IRE Transactions on Automatic Control, 7 (1962), 120–122. doi: 10.1109/TAC.1962.1105490.  Google Scholar

[13]

X. Li and X. Y. Zhou, Indefinite stochastic LQ controls with Markovian jumps in a finite time horizon, Communications in Information and Systems, 2 (2002), 265-282.  doi: 10.4310/CIS.2002.v2.n3.a4.  Google Scholar

[14]

X. Li, J. Sun and J. Yong, Mean-field stochastic linear quadratic optimal control problems: Closed-loop solvability, Probability, Uncertainty and Quantitative Risk, 1 (2016), Paper No. 2, 24 pp. doi: 10.1186/s41546-016-0002-3.  Google Scholar

[15]

X. Li, X. Y. Zhou and M. Ait Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon, Journal of Global Optimization, 27 (2003), 149-175.  doi: 10.1023/A:1024887007165.  Google Scholar

[16]

X. Li and X. Y. Zhou, Continuous-time mean-variance efficiency: The 80% rule, The Annals of Applied Probability, 16 (2006), 1751-1763.  doi: 10.1214/105051606000000349.  Google Scholar

[17]

Y. Liu, G. Yin and X. Y. Zhou, Near-optimal controls of random-switching LQ problems with indefinite control weight costs, Automatica, 41 (2005), 1063-1070.  doi: 10.1016/j.automatica.2005.01.002.  Google Scholar

[18]

R. Penrose, A generalized inverse for matrices, Mathematical Proceedings of the Cambridge Philosophical Society, 51 (1955), 406-413.  doi: 10.1017/S0305004100030401.  Google Scholar

[19]

J. Sun and J. Yong, Linear quadratic stochastic differential games: open-loop and closed-loop saddle points, SIAM Journal on Control and Optimization, 52 (2014), 4082-4121.  doi: 10.1137/140953642.  Google Scholar

[20]

J. Sun, X. Li and J. Yong, Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM Journal on Control and Optimization, 54 (2016), 2274–2308. doi: 10.1137/15M103532X.  Google Scholar

[21]

J. Sun, Mean-Field Stochastic Linear Quadratic Optimal Control Problems: Open-Loop Solvabilities, ESAIM: Control, Optimisation and Calculus of Variations, 23 (2017), 1099-1127.  doi: 10.1051/cocv/2016023.  Google Scholar

[22]

R. Tao and Z. Wu, Maximum principle for optimal control problems of forward-backward regime-switching system and applications, Systems & Control Letters, 61 (2012), 911-917.  doi: 10.1016/j.sysconle.2012.06.006.  Google Scholar

[23]

W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM Journal on Control, 6 (1968), 681–697. doi: 10.1137/0306044.  Google Scholar

[24]

Z. Wu and X.-R. Wang, FBSDE with Poisson process and its application to linear quadratic stochastic optimal control problem with random jumps, Acta Automatica Sinica, 29 (2003), 821-826.   Google Scholar

[25]

G. G. Yin and Q. Zhang, Continuous-Time Markov Chains and Applications: A Two-Time-Scale Approach, 2nd Edition, Springer, New York, 2013. doi: 10.1007/978-1-4614-4346-9.  Google Scholar

[26]

K.-F. C. Yiu, J. Liu, T. K. Siu and W.-K. Ching, Optimal portfolios with regime switching and value-at-risk constraint, Automatica, 46 (2010), 979-989.  doi: 10.1016/j.automatica.2010.02.027.  Google Scholar

[27]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

[28]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations, SIAM Journal on Control and Optimization, 51 (2013), 2809-2838.  doi: 10.1137/120892477.  Google Scholar

[29]

Z. Yu, Infinite horizon jump-diffusion forward-backward stochastic differential equations and their application to backward linear-quadratic problems, ESAIM: Control, Optimisation and Calculus of Variations, 23 (2017), 1331–1359. doi: 10.1051/cocv/2016055.  Google Scholar

[30]

X. Zhang, R. J. Elliott, T. K. Siu and J. Guo, Markovian regime-switching market completion using additional markov jump assets, IMA Journal of Management Mathematics, 23 (2012), 283–305. doi: 10.1093/imaman/dpr018.  Google Scholar

[31]

X. Zhang and X. Li, Open-Loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems of Markov regime-switching system, arXiv: 1809.01891, 2018. Google Scholar

[32]

X. Zhang, Z. Sun and J. Xiong, A general stochastic maximum principle for a Markov regime switching jump-diffusion model of mean-field type, SIAM Journal on Control and Optimization, 56 (2018), 2563–2592. doi: 10.1137/17M112395X.  Google Scholar

[33]

X. Y. Zhou and G. Yin, Markowitz's mean-variance portfolio selection with regime switching: A continuous-time model, SIAM Journal on Control and Optimization, 42 (2003), 1466-1482.  doi: 10.1137/S0363012902405583.  Google Scholar

show all references

References:
[1]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type, Applied Mathematics and Optimization, 63 (2011), 341-356.  doi: 10.1007/s00245-010-9123-8.  Google Scholar

[2]

J.-M. Bismut, Linear quadratic optimal stochastic control with random coefficients, SIAM Journal on Control and Optimization, 14 (1976), 419–444. doi: 10.1137/0314028.  Google Scholar

[3]

R. Buckdahn, B. Djehiche and J. Li, A general stochastic maximum principle for SDEs of mean-field type, Applied Mathematics and Optimization, 64 (2011), 197-216.  doi: 10.1007/s00245-011-9136-y.  Google Scholar

[4]

R. Buckdahn, B. Djehiche, J. Li and S. Peng, Mean-field backward stochastic differential equations: A limit approach, Annals of Probability, 37 (2009), 1524-1565.  doi: 10.1214/08-AOP442.  Google Scholar

[5]

X. Cui, X. Li and D. Li, Unified framework of mean-field formulations for optimal multi-period mean-variance portfolio selection, IEEE Transactions on Automatic Control, 59 (2014), 1833–1844. doi: 10.1109/TAC.2014.2311875.  Google Scholar

[6]

C. Donnelly, Suffcient stochastic maximum principle in a regime-switching diffusion model, Applied Mathematics & Optimization, 64 (2011), 155-169.  doi: 10.1007/s00245-010-9130-9.  Google Scholar

[7]

C. Donnelly and A. J. Heunis, Quadratic Risk Minimization in a Regime-Switching Model with Portfolio Constraints, SIAM Journal on Control and Optimization, 50 (2012), 2431–2461. doi: 10.1137/100809271.  Google Scholar

[8]

R. J. Elliott, L. Aggoun and J. B. Moore, Hidden Markov Models: Estimation and Control, Applications of Mathematics, New York: Springer-Verlag, 1995.  Google Scholar

[9]

R. Elliott, X. Li and Y.-H. Ni, Discrete time mean-field stochastic linear-quadratic optimal control problems, Automatica, 49 (2013), 3222-3233.  doi: 10.1016/j.automatica.2013.08.017.  Google Scholar

[10]

J. Huang, X. Li and J. Yong, A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon, Mathematical Control & Related Fields, 5 (2015), 97-139.  doi: 10.3934/mcrf.2015.5.97.  Google Scholar

[11]

M. Kac, Foundations of kinetic theory, Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 3 (1956), 171-197.   Google Scholar

[12]

H. Kushner, Optimal stochastic control, IRE Transactions on Automatic Control, 7 (1962), 120–122. doi: 10.1109/TAC.1962.1105490.  Google Scholar

[13]

X. Li and X. Y. Zhou, Indefinite stochastic LQ controls with Markovian jumps in a finite time horizon, Communications in Information and Systems, 2 (2002), 265-282.  doi: 10.4310/CIS.2002.v2.n3.a4.  Google Scholar

[14]

X. Li, J. Sun and J. Yong, Mean-field stochastic linear quadratic optimal control problems: Closed-loop solvability, Probability, Uncertainty and Quantitative Risk, 1 (2016), Paper No. 2, 24 pp. doi: 10.1186/s41546-016-0002-3.  Google Scholar

[15]

X. Li, X. Y. Zhou and M. Ait Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon, Journal of Global Optimization, 27 (2003), 149-175.  doi: 10.1023/A:1024887007165.  Google Scholar

[16]

X. Li and X. Y. Zhou, Continuous-time mean-variance efficiency: The 80% rule, The Annals of Applied Probability, 16 (2006), 1751-1763.  doi: 10.1214/105051606000000349.  Google Scholar

[17]

Y. Liu, G. Yin and X. Y. Zhou, Near-optimal controls of random-switching LQ problems with indefinite control weight costs, Automatica, 41 (2005), 1063-1070.  doi: 10.1016/j.automatica.2005.01.002.  Google Scholar

[18]

R. Penrose, A generalized inverse for matrices, Mathematical Proceedings of the Cambridge Philosophical Society, 51 (1955), 406-413.  doi: 10.1017/S0305004100030401.  Google Scholar

[19]

J. Sun and J. Yong, Linear quadratic stochastic differential games: open-loop and closed-loop saddle points, SIAM Journal on Control and Optimization, 52 (2014), 4082-4121.  doi: 10.1137/140953642.  Google Scholar

[20]

J. Sun, X. Li and J. Yong, Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM Journal on Control and Optimization, 54 (2016), 2274–2308. doi: 10.1137/15M103532X.  Google Scholar

[21]

J. Sun, Mean-Field Stochastic Linear Quadratic Optimal Control Problems: Open-Loop Solvabilities, ESAIM: Control, Optimisation and Calculus of Variations, 23 (2017), 1099-1127.  doi: 10.1051/cocv/2016023.  Google Scholar

[22]

R. Tao and Z. Wu, Maximum principle for optimal control problems of forward-backward regime-switching system and applications, Systems & Control Letters, 61 (2012), 911-917.  doi: 10.1016/j.sysconle.2012.06.006.  Google Scholar

[23]

W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM Journal on Control, 6 (1968), 681–697. doi: 10.1137/0306044.  Google Scholar

[24]

Z. Wu and X.-R. Wang, FBSDE with Poisson process and its application to linear quadratic stochastic optimal control problem with random jumps, Acta Automatica Sinica, 29 (2003), 821-826.   Google Scholar

[25]

G. G. Yin and Q. Zhang, Continuous-Time Markov Chains and Applications: A Two-Time-Scale Approach, 2nd Edition, Springer, New York, 2013. doi: 10.1007/978-1-4614-4346-9.  Google Scholar

[26]

K.-F. C. Yiu, J. Liu, T. K. Siu and W.-K. Ching, Optimal portfolios with regime switching and value-at-risk constraint, Automatica, 46 (2010), 979-989.  doi: 10.1016/j.automatica.2010.02.027.  Google Scholar

[27]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

[28]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations, SIAM Journal on Control and Optimization, 51 (2013), 2809-2838.  doi: 10.1137/120892477.  Google Scholar

[29]

Z. Yu, Infinite horizon jump-diffusion forward-backward stochastic differential equations and their application to backward linear-quadratic problems, ESAIM: Control, Optimisation and Calculus of Variations, 23 (2017), 1331–1359. doi: 10.1051/cocv/2016055.  Google Scholar

[30]

X. Zhang, R. J. Elliott, T. K. Siu and J. Guo, Markovian regime-switching market completion using additional markov jump assets, IMA Journal of Management Mathematics, 23 (2012), 283–305. doi: 10.1093/imaman/dpr018.  Google Scholar

[31]

X. Zhang and X. Li, Open-Loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems of Markov regime-switching system, arXiv: 1809.01891, 2018. Google Scholar

[32]

X. Zhang, Z. Sun and J. Xiong, A general stochastic maximum principle for a Markov regime switching jump-diffusion model of mean-field type, SIAM Journal on Control and Optimization, 56 (2018), 2563–2592. doi: 10.1137/17M112395X.  Google Scholar

[33]

X. Y. Zhou and G. Yin, Markowitz's mean-variance portfolio selection with regime switching: A continuous-time model, SIAM Journal on Control and Optimization, 42 (2003), 1466-1482.  doi: 10.1137/S0363012902405583.  Google Scholar

[1]

Jun Moon. Linear-quadratic mean-field type stackelberg differential games for stochastic jump-diffusion systems. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021026
[2]

Marco Cirant, Diogo A. Gomes, Edgard A. Pimentel, Héctor Sánchez-Morgado. On some singular mean-field games. Journal of Dynamics & Games, 2021  doi: 10.3934/jdg.2021006
[3]

Seung-Yeal Ha, Jinwook Jung, Jeongho Kim, Jinyeong Park, Xiongtao Zhang. A mean-field limit of the particle swarmalator model. Kinetic & Related Models, , () : -. doi: 10.3934/krm.2021011
[4]

René Aïd, Roxana Dumitrescu, Peter Tankov. The entry and exit game in the electricity markets: A mean-field game approach. Journal of Dynamics & Games, 2021  doi: 10.3934/jdg.2021012
[5]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437
[6]

Christoforidou Amalia, Christian-Oliver Ewald. A lattice method for option evaluation with regime-switching asset correlation structure. Journal of Industrial & Management Optimization, 2021, 17 (4) : 1729-1752. doi: 10.3934/jimo.2020042
[7]

Luke Finlay, Vladimir Gaitsgory, Ivan Lebedev. Linear programming solutions of periodic optimization problems: approximation of the optimal control. Journal of Industrial & Management Optimization, 2007, 3 (2) : 399-413. doi: 10.3934/jimo.2007.3.399
[8]

Vladimir Gaitsgory, Ilya Shvartsman. Linear programming estimates for Cesàro and Abel limits of optimal values in optimal control problems. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021102
[9]

Qing Liu, Bingo Wing-Kuen Ling, Qingyun Dai, Qing Miao, Caixia Liu. Optimal maximally decimated M-channel mirrored paraunitary linear phase FIR filter bank design via norm relaxed sequential quadratic programming. Journal of Industrial & Management Optimization, 2021, 17 (4) : 1993-2011. doi: 10.3934/jimo.2020055
[10]

Marita Holtmannspötter, Arnd Rösch, Boris Vexler. A priori error estimates for the space-time finite element discretization of an optimal control problem governed by a coupled linear PDE-ODE system. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021014
[11]

Siting Liu, Levon Nurbekyan. Splitting methods for a class of non-potential mean field games. Journal of Dynamics & Games, 2021  doi: 10.3934/jdg.2021014
[12]

Tobias Geiger, Daniel Wachsmuth, Gerd Wachsmuth. Optimal control of ODEs with state suprema. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021012
[13]

Lorenzo Freddi. Optimal control of the transmission rate in compartmental epidemics. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021007
[14]

Marzia Bisi, Maria Groppi, Giorgio Martalò, Romina Travaglini. Optimal control of leachate recirculation for anaerobic processes in landfills. Discrete & Continuous Dynamical Systems - B, 2021, 26 (6) : 2957-2976. doi: 10.3934/dcdsb.2020215
[15]

Yingxu Tian, Junyi Guo, Zhongyang Sun. Optimal mean-variance reinsurance in a financial market with stochastic rate of return. Journal of Industrial & Management Optimization, 2021, 17 (4) : 1887-1912. doi: 10.3934/jimo.2020051
[16]

Paula A. González-Parra, Sunmi Lee, Leticia Velázquez, Carlos Castillo-Chavez. A note on the use of optimal control on a discrete time model of influenza dynamics. Mathematical Biosciences & Engineering, 2011, 8 (1) : 183-197. doi: 10.3934/mbe.2011.8.183
[17]

Xiaohong Li, Mingxin Sun, Zhaohua Gong, Enmin Feng. Multistage optimal control for microbial fed-batch fermentation process. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021040
[18]

John T. Betts, Stephen Campbell, Claire Digirolamo. Examination of solving optimal control problems with delays using GPOPS-Ⅱ. Numerical Algebra, Control & Optimization, 2021, 11 (2) : 283-305. doi: 10.3934/naco.2020026
[19]

Livia Betz, Irwin Yousept. Optimal control of elliptic variational inequalities with bounded and unbounded operators. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021009
[20]

Christian Meyer, Stephan Walther. Optimal control of perfect plasticity part I: Stress tracking. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021022

2019 Impact Factor: 1.366

Tools

Article outline

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences