American Institute of Mathematical Sciences

Advanced
July  2022, 18(4): 2415-2433. doi: 10.3934/jimo.2021074

Open-loop solvability for mean-field stochastic linear quadratic optimal control problems of Markov regime-switching system

Kehan Si 1, , Zhenda Xu 1, , Ka Fai Cedric Yiu 2,, and  Xun Li 2,

1. 

School of Mathematics, Shandong University, Jinan, Shandong Province, 250100, China
2. 

Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China

* Corresponding author: Ka Fai Cedric Yiu

Received  September 2020 Revised  January 2021 Published  July 2022 Early access  April 2021

This paper investigates the mean-field stochastic linear quadratic optimal control problem of Markov regime switching system (M-MF-SLQ, for short). The representation of the cost functional for the M-MF-SLQ is derived using the technique of operators. It is shown that the convexity of the cost functional is necessary for the finiteness of the M-MF-SLQ problem, whereas uniform convexity of the cost functional is sufficient for the open-loop solvability of the problem. By considering a family of uniformly convex cost functionals, a characterization of the finiteness of the problem is derived and a minimizing sequence, whose convergence is equivalent to the open-loop solvability of the problem, is constructed. We demonstrate with a few examples that our results can be employed for tackling some financial problems such as mean-variance portfolio selection problem.

Keywords: Mean-field, linear quadratic optimal control, Markov regime switching, open-loop solvability.
Mathematics Subject Classification: Primary: 49N10, 49N35; Secondary: 93E20.
Citation: Kehan Si, Zhenda Xu, Ka Fai Cedric Yiu, Xun Li. Open-loop solvability for mean-field stochastic linear quadratic optimal control problems of Markov regime-switching system. Journal of Industrial and Management Optimization, 2022, 18 (4) : 2415-2433. doi: 10.3934/jimo.2021074
References:
[1]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type, Applied Mathematics and Optimization, 63 (2011), 341-356.  doi: 10.1007/s00245-010-9123-8.

[2]

J.-M. Bismut, Linear quadratic optimal stochastic control with random coefficients, SIAM Journal on Control and Optimization, 14 (1976), 419–444. doi: 10.1137/0314028.

[3]

R. Buckdahn, B. Djehiche and J. Li, A general stochastic maximum principle for SDEs of mean-field type, Applied Mathematics and Optimization, 64 (2011), 197-216.  doi: 10.1007/s00245-011-9136-y.

[4]

R. Buckdahn, B. Djehiche, J. Li and S. Peng, Mean-field backward stochastic differential equations: A limit approach, Annals of Probability, 37 (2009), 1524-1565.  doi: 10.1214/08-AOP442.

[5]

X. Cui, X. Li and D. Li, Unified framework of mean-field formulations for optimal multi-period mean-variance portfolio selection, IEEE Transactions on Automatic Control, 59 (2014), 1833–1844. doi: 10.1109/TAC.2014.2311875.

[6]

C. Donnelly, Suffcient stochastic maximum principle in a regime-switching diffusion model, Applied Mathematics & Optimization, 64 (2011), 155-169.  doi: 10.1007/s00245-010-9130-9.

[7]

C. Donnelly and A. J. Heunis, Quadratic Risk Minimization in a Regime-Switching Model with Portfolio Constraints, SIAM Journal on Control and Optimization, 50 (2012), 2431–2461. doi: 10.1137/100809271.

[8]

R. J. Elliott, L. Aggoun and J. B. Moore, Hidden Markov Models: Estimation and Control, Applications of Mathematics, New York: Springer-Verlag, 1995.

[9]

R. Elliott, X. Li and Y.-H. Ni, Discrete time mean-field stochastic linear-quadratic optimal control problems, Automatica, 49 (2013), 3222-3233.  doi: 10.1016/j.automatica.2013.08.017.

[10]

J. Huang, X. Li and J. Yong, A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon, Mathematical Control & Related Fields, 5 (2015), 97-139.  doi: 10.3934/mcrf.2015.5.97.

[11]

M. Kac, Foundations of kinetic theory, Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 3 (1956), 171-197. 

[12]

H. Kushner, Optimal stochastic control, IRE Transactions on Automatic Control, 7 (1962), 120–122. doi: 10.1109/TAC.1962.1105490.

[13]

X. Li and X. Y. Zhou, Indefinite stochastic LQ controls with Markovian jumps in a finite time horizon, Communications in Information and Systems, 2 (2002), 265-282.  doi: 10.4310/CIS.2002.v2.n3.a4.

[14]

X. Li, J. Sun and J. Yong, Mean-field stochastic linear quadratic optimal control problems: Closed-loop solvability, Probability, Uncertainty and Quantitative Risk, 1 (2016), Paper No. 2, 24 pp. doi: 10.1186/s41546-016-0002-3.

[15]

X. Li, X. Y. Zhou and M. Ait Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon, Journal of Global Optimization, 27 (2003), 149-175.  doi: 10.1023/A:1024887007165.

[16]

X. Li and X. Y. Zhou, Continuous-time mean-variance efficiency: The 80% rule, The Annals of Applied Probability, 16 (2006), 1751-1763.  doi: 10.1214/105051606000000349.

[17]

Y. Liu, G. Yin and X. Y. Zhou, Near-optimal controls of random-switching LQ problems with indefinite control weight costs, Automatica, 41 (2005), 1063-1070.  doi: 10.1016/j.automatica.2005.01.002.

[18]

R. Penrose, A generalized inverse for matrices, Mathematical Proceedings of the Cambridge Philosophical Society, 51 (1955), 406-413.  doi: 10.1017/S0305004100030401.

[19]

J. Sun and J. Yong, Linear quadratic stochastic differential games: open-loop and closed-loop saddle points, SIAM Journal on Control and Optimization, 52 (2014), 4082-4121.  doi: 10.1137/140953642.

[20]

J. Sun, X. Li and J. Yong, Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM Journal on Control and Optimization, 54 (2016), 2274–2308. doi: 10.1137/15M103532X.

[21]

J. Sun, Mean-Field Stochastic Linear Quadratic Optimal Control Problems: Open-Loop Solvabilities, ESAIM: Control, Optimisation and Calculus of Variations, 23 (2017), 1099-1127.  doi: 10.1051/cocv/2016023.

[22]

R. Tao and Z. Wu, Maximum principle for optimal control problems of forward-backward regime-switching system and applications, Systems & Control Letters, 61 (2012), 911-917.  doi: 10.1016/j.sysconle.2012.06.006.

[23]

W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM Journal on Control, 6 (1968), 681–697. doi: 10.1137/0306044.

[24]

Z. Wu and X.-R. Wang, FBSDE with Poisson process and its application to linear quadratic stochastic optimal control problem with random jumps, Acta Automatica Sinica, 29 (2003), 821-826. 

[25]

G. G. Yin and Q. Zhang, Continuous-Time Markov Chains and Applications: A Two-Time-Scale Approach, 2nd Edition, Springer, New York, 2013. doi: 10.1007/978-1-4614-4346-9.

[26]

K.-F. C. Yiu, J. Liu, T. K. Siu and W.-K. Ching, Optimal portfolios with regime switching and value-at-risk constraint, Automatica, 46 (2010), 979-989.  doi: 10.1016/j.automatica.2010.02.027.

[27]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer New York, 1999. doi: 10.1007/978-1-4612-1466-3.

[28]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations, SIAM Journal on Control and Optimization, 51 (2013), 2809-2838.  doi: 10.1137/120892477.

[29]

Z. Yu, Infinite horizon jump-diffusion forward-backward stochastic differential equations and their application to backward linear-quadratic problems, ESAIM: Control, Optimisation and Calculus of Variations, 23 (2017), 1331–1359. doi: 10.1051/cocv/2016055.

[30]

X. Zhang, R. J. Elliott, T. K. Siu and J. Guo, Markovian regime-switching market completion using additional markov jump assets, IMA Journal of Management Mathematics, 23 (2012), 283–305. doi: 10.1093/imaman/dpr018.

[31]

X. Zhang and X. Li, Open-Loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems of Markov regime-switching system, arXiv: 1809.01891, 2018.

[32]

X. Zhang, Z. Sun and J. Xiong, A general stochastic maximum principle for a Markov regime switching jump-diffusion model of mean-field type, SIAM Journal on Control and Optimization, 56 (2018), 2563–2592. doi: 10.1137/17M112395X.

[33]

X. Y. Zhou and G. Yin, Markowitz's mean-variance portfolio selection with regime switching: A continuous-time model, SIAM Journal on Control and Optimization, 42 (2003), 1466-1482.  doi: 10.1137/S0363012902405583.

show all references

References:
[1]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type, Applied Mathematics and Optimization, 63 (2011), 341-356.  doi: 10.1007/s00245-010-9123-8.

[2]

J.-M. Bismut, Linear quadratic optimal stochastic control with random coefficients, SIAM Journal on Control and Optimization, 14 (1976), 419–444. doi: 10.1137/0314028.

[3]

R. Buckdahn, B. Djehiche and J. Li, A general stochastic maximum principle for SDEs of mean-field type, Applied Mathematics and Optimization, 64 (2011), 197-216.  doi: 10.1007/s00245-011-9136-y.

[4]

R. Buckdahn, B. Djehiche, J. Li and S. Peng, Mean-field backward stochastic differential equations: A limit approach, Annals of Probability, 37 (2009), 1524-1565.  doi: 10.1214/08-AOP442.

[5]

X. Cui, X. Li and D. Li, Unified framework of mean-field formulations for optimal multi-period mean-variance portfolio selection, IEEE Transactions on Automatic Control, 59 (2014), 1833–1844. doi: 10.1109/TAC.2014.2311875.

[6]

C. Donnelly, Suffcient stochastic maximum principle in a regime-switching diffusion model, Applied Mathematics & Optimization, 64 (2011), 155-169.  doi: 10.1007/s00245-010-9130-9.

[7]

C. Donnelly and A. J. Heunis, Quadratic Risk Minimization in a Regime-Switching Model with Portfolio Constraints, SIAM Journal on Control and Optimization, 50 (2012), 2431–2461. doi: 10.1137/100809271.

[8]

R. J. Elliott, L. Aggoun and J. B. Moore, Hidden Markov Models: Estimation and Control, Applications of Mathematics, New York: Springer-Verlag, 1995.

[9]

R. Elliott, X. Li and Y.-H. Ni, Discrete time mean-field stochastic linear-quadratic optimal control problems, Automatica, 49 (2013), 3222-3233.  doi: 10.1016/j.automatica.2013.08.017.

[10]

J. Huang, X. Li and J. Yong, A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon, Mathematical Control & Related Fields, 5 (2015), 97-139.  doi: 10.3934/mcrf.2015.5.97.

[11]

M. Kac, Foundations of kinetic theory, Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 3 (1956), 171-197. 

[12]

H. Kushner, Optimal stochastic control, IRE Transactions on Automatic Control, 7 (1962), 120–122. doi: 10.1109/TAC.1962.1105490.

[13]

X. Li and X. Y. Zhou, Indefinite stochastic LQ controls with Markovian jumps in a finite time horizon, Communications in Information and Systems, 2 (2002), 265-282.  doi: 10.4310/CIS.2002.v2.n3.a4.

[14]

X. Li, J. Sun and J. Yong, Mean-field stochastic linear quadratic optimal control problems: Closed-loop solvability, Probability, Uncertainty and Quantitative Risk, 1 (2016), Paper No. 2, 24 pp. doi: 10.1186/s41546-016-0002-3.

[15]

X. Li, X. Y. Zhou and M. Ait Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon, Journal of Global Optimization, 27 (2003), 149-175.  doi: 10.1023/A:1024887007165.

[16]

X. Li and X. Y. Zhou, Continuous-time mean-variance efficiency: The 80% rule, The Annals of Applied Probability, 16 (2006), 1751-1763.  doi: 10.1214/105051606000000349.

[17]

Y. Liu, G. Yin and X. Y. Zhou, Near-optimal controls of random-switching LQ problems with indefinite control weight costs, Automatica, 41 (2005), 1063-1070.  doi: 10.1016/j.automatica.2005.01.002.

[18]

R. Penrose, A generalized inverse for matrices, Mathematical Proceedings of the Cambridge Philosophical Society, 51 (1955), 406-413.  doi: 10.1017/S0305004100030401.

[19]

J. Sun and J. Yong, Linear quadratic stochastic differential games: open-loop and closed-loop saddle points, SIAM Journal on Control and Optimization, 52 (2014), 4082-4121.  doi: 10.1137/140953642.

[20]

J. Sun, X. Li and J. Yong, Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM Journal on Control and Optimization, 54 (2016), 2274–2308. doi: 10.1137/15M103532X.

[21]

J. Sun, Mean-Field Stochastic Linear Quadratic Optimal Control Problems: Open-Loop Solvabilities, ESAIM: Control, Optimisation and Calculus of Variations, 23 (2017), 1099-1127.  doi: 10.1051/cocv/2016023.

[22]

R. Tao and Z. Wu, Maximum principle for optimal control problems of forward-backward regime-switching system and applications, Systems & Control Letters, 61 (2012), 911-917.  doi: 10.1016/j.sysconle.2012.06.006.

[23]

W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM Journal on Control, 6 (1968), 681–697. doi: 10.1137/0306044.

[24]

Z. Wu and X.-R. Wang, FBSDE with Poisson process and its application to linear quadratic stochastic optimal control problem with random jumps, Acta Automatica Sinica, 29 (2003), 821-826. 

[25]

G. G. Yin and Q. Zhang, Continuous-Time Markov Chains and Applications: A Two-Time-Scale Approach, 2nd Edition, Springer, New York, 2013. doi: 10.1007/978-1-4614-4346-9.

[26]

K.-F. C. Yiu, J. Liu, T. K. Siu and W.-K. Ching, Optimal portfolios with regime switching and value-at-risk constraint, Automatica, 46 (2010), 979-989.  doi: 10.1016/j.automatica.2010.02.027.

[27]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer New York, 1999. doi: 10.1007/978-1-4612-1466-3.

[28]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations, SIAM Journal on Control and Optimization, 51 (2013), 2809-2838.  doi: 10.1137/120892477.

[29]

Z. Yu, Infinite horizon jump-diffusion forward-backward stochastic differential equations and their application to backward linear-quadratic problems, ESAIM: Control, Optimisation and Calculus of Variations, 23 (2017), 1331–1359. doi: 10.1051/cocv/2016055.

[30]

X. Zhang, R. J. Elliott, T. K. Siu and J. Guo, Markovian regime-switching market completion using additional markov jump assets, IMA Journal of Management Mathematics, 23 (2012), 283–305. doi: 10.1093/imaman/dpr018.

[31]

X. Zhang and X. Li, Open-Loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems of Markov regime-switching system, arXiv: 1809.01891, 2018.

[32]

X. Zhang, Z. Sun and J. Xiong, A general stochastic maximum principle for a Markov regime switching jump-diffusion model of mean-field type, SIAM Journal on Control and Optimization, 56 (2018), 2563–2592. doi: 10.1137/17M112395X.

[33]

X. Y. Zhou and G. Yin, Markowitz's mean-variance portfolio selection with regime switching: A continuous-time model, SIAM Journal on Control and Optimization, 42 (2003), 1466-1482.  doi: 10.1137/S0363012902405583.

[1]

Jingrui Sun, Hanxiao Wang. Mean-field stochastic linear-quadratic optimal control problems: Weak closed-loop solvability. Mathematical Control and Related Fields, 2021, 11 (1) : 47-71. doi: 10.3934/mcrf.2020026
[2]

Xun Li, Jingrui Sun, Jiongmin Yong. Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability. Probability, Uncertainty and Quantitative Risk, 2016, 1 (0) : 2-. doi: 10.1186/s41546-016-0002-3
[3]

Alain Bensoussan, Shaokuan Chen, Suresh P. Sethi. Linear quadratic differential games with mixed leadership: The open-loop solution. Numerical Algebra, Control and Optimization, 2013, 3 (1) : 95-108. doi: 10.3934/naco.2013.3.95
[4]

Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control and Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97
[5]

Ishak Alia, Mohamed Sofiane Alia. Open-loop equilibrium strategy for mean-variance Portfolio selection with investment constraints in a non-Markovian regime-switching jump-diffusion model. Journal of Industrial and Management Optimization, 2022  doi: 10.3934/jimo.2022048
[6]

Yu Li, Kok Lay Teo, Shuhua Zhang. A new feedback form of open-loop Stackelberg strategy in a general linear-quadratic differential game. Journal of Industrial and Management Optimization, 2022  doi: 10.3934/jimo.2022105
[7]

Salah Eddine Choutri, Boualem Djehiche, Hamidou Tembine. Optimal control and zero-sum games for Markov chains of mean-field type. Mathematical Control and Related Fields, 2019, 9 (3) : 571-605. doi: 10.3934/mcrf.2019026
[8]

Hanxiao Wang, Jingrui Sun, Jiongmin Yong. Weak closed-loop solvability of stochastic linear-quadratic optimal control problems. Discrete and Continuous Dynamical Systems, 2019, 39 (5) : 2785-2805. doi: 10.3934/dcds.2019117
[9]

Ishak Alia. Open-loop equilibriums for a general class of time-inconsistent stochastic optimal control problems. Mathematical Control and Related Fields, 2021  doi: 10.3934/mcrf.2021053
[10]

Zhenghong Qiu, Jianhui Huang, Tinghan Xie. Linear-Quadratic-Gaussian mean-field controls of social optima. Mathematical Control and Related Fields, 2021  doi: 10.3934/mcrf.2021047
[11]

Hélène Hibon, Ying Hu, Shanjian Tang. Mean-field type quadratic BSDEs. Numerical Algebra, Control and Optimization, 2022  doi: 10.3934/naco.2022009
[12]

Jiannan Zhang, Ping Chen, Zhuo Jin, Shuanming Li. Open-loop equilibrium strategy for mean-variance portfolio selection: A log-return model. Journal of Industrial and Management Optimization, 2021, 17 (2) : 765-777. doi: 10.3934/jimo.2019133
[13]

Liming Zhang, Rongming Wang, Jiaqin Wei. Open-loop equilibrium mean-variance reinsurance, new business and investment strategies with constraints. Journal of Industrial and Management Optimization, 2021  doi: 10.3934/jimo.2021140
[14]

Tianxiao Wang. Characterizations of equilibrium controls in time inconsistent mean-field stochastic linear quadratic problems. I. Mathematical Control and Related Fields, 2019, 9 (2) : 385-409. doi: 10.3934/mcrf.2019018
[15]

Jianhui Huang, Shujun Wang, Zhen Wu. Backward-forward linear-quadratic mean-field games with major and minor agents. Probability, Uncertainty and Quantitative Risk, 2016, 1 (0) : 8-. doi: 10.1186/s41546-016-0009-9
[16]

René Carmona, Kenza Hamidouche, Mathieu Laurière, Zongjun Tan. Linear-quadratic zero-sum mean-field type games: Optimality conditions and policy optimization. Journal of Dynamics and Games, 2021, 8 (4) : 403-443. doi: 10.3934/jdg.2021023
[17]

Jun Moon. Linear-quadratic mean-field type stackelberg differential games for stochastic jump-diffusion systems. Mathematical Control and Related Fields, 2022, 12 (2) : 371-404. doi: 10.3934/mcrf.2021026
[18]

Galina Kurina, Sahlar Meherrem. Decomposition of discrete linear-quadratic optimal control problems for switching systems. Conference Publications, 2015, 2015 (special) : 764-774. doi: 10.3934/proc.2015.0764
[19]

Lin Xu, Rongming Wang, Dingjun Yao. Optimal stochastic investment games under Markov regime switching market. Journal of Industrial and Management Optimization, 2014, 10 (3) : 795-815. doi: 10.3934/jimo.2014.10.795
[20]

Dean A. Carlson. Finding open-loop Nash equilibrium for variational games. Conference Publications, 2005, 2005 (Special) : 153-163. doi: 10.3934/proc.2005.2005.153

2021 Impact Factor: 1.411

Tools

Article outline

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy