American Institute of Mathematical Sciences

Advanced
doi: 10.3934/jimo.2021074

Open-loop solvability for mean-field stochastic linear quadratic optimal control problems of Markov regime-switching system

Kehan Si 1, , Zhenda Xu 1, , Ka Fai Cedric Yiu 2,, and  Xun Li 2,

1. 

School of Mathematics, Shandong University, Jinan, Shandong Province, 250100, China
2. 

Department of Applied Mathematics, The Hong Kong Polytechnic University, Hong Kong, China

* Corresponding author: Ka Fai Cedric Yiu

Received  September 2020 Revised  January 2021 Early access  April 2021

This paper investigates the mean-field stochastic linear quadratic optimal control problem of Markov regime switching system (M-MF-SLQ, for short). The representation of the cost functional for the M-MF-SLQ is derived using the technique of operators. It is shown that the convexity of the cost functional is necessary for the finiteness of the M-MF-SLQ problem, whereas uniform convexity of the cost functional is sufficient for the open-loop solvability of the problem. By considering a family of uniformly convex cost functionals, a characterization of the finiteness of the problem is derived and a minimizing sequence, whose convergence is equivalent to the open-loop solvability of the problem, is constructed. We demonstrate with a few examples that our results can be employed for tackling some financial problems such as mean-variance portfolio selection problem.

Keywords: Mean-field, linear quadratic optimal control, Markov regime switching, open-loop solvability.
Mathematics Subject Classification: Primary: 49N10, 49N35; Secondary: 93E20.
Citation: Kehan Si, Zhenda Xu, Ka Fai Cedric Yiu, Xun Li. Open-loop solvability for mean-field stochastic linear quadratic optimal control problems of Markov regime-switching system. Journal of Industrial & Management Optimization, doi: 10.3934/jimo.2021074
References:
[1]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type, Applied Mathematics and Optimization, 63 (2011), 341-356.  doi: 10.1007/s00245-010-9123-8.  Google Scholar

[2]

J.-M. Bismut, Linear quadratic optimal stochastic control with random coefficients, SIAM Journal on Control and Optimization, 14 (1976), 419–444. doi: 10.1137/0314028.  Google Scholar

[3]

R. Buckdahn, B. Djehiche and J. Li, A general stochastic maximum principle for SDEs of mean-field type, Applied Mathematics and Optimization, 64 (2011), 197-216.  doi: 10.1007/s00245-011-9136-y.  Google Scholar

[4]

R. Buckdahn, B. Djehiche, J. Li and S. Peng, Mean-field backward stochastic differential equations: A limit approach, Annals of Probability, 37 (2009), 1524-1565.  doi: 10.1214/08-AOP442.  Google Scholar

[5]

X. Cui, X. Li and D. Li, Unified framework of mean-field formulations for optimal multi-period mean-variance portfolio selection, IEEE Transactions on Automatic Control, 59 (2014), 1833–1844. doi: 10.1109/TAC.2014.2311875.  Google Scholar

[6]

C. Donnelly, Suffcient stochastic maximum principle in a regime-switching diffusion model, Applied Mathematics & Optimization, 64 (2011), 155-169.  doi: 10.1007/s00245-010-9130-9.  Google Scholar

[7]

C. Donnelly and A. J. Heunis, Quadratic Risk Minimization in a Regime-Switching Model with Portfolio Constraints, SIAM Journal on Control and Optimization, 50 (2012), 2431–2461. doi: 10.1137/100809271.  Google Scholar

[8]

R. J. Elliott, L. Aggoun and J. B. Moore, Hidden Markov Models: Estimation and Control, Applications of Mathematics, New York: Springer-Verlag, 1995.  Google Scholar

[9]

R. Elliott, X. Li and Y.-H. Ni, Discrete time mean-field stochastic linear-quadratic optimal control problems, Automatica, 49 (2013), 3222-3233.  doi: 10.1016/j.automatica.2013.08.017.  Google Scholar

[10]

J. Huang, X. Li and J. Yong, A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon, Mathematical Control & Related Fields, 5 (2015), 97-139.  doi: 10.3934/mcrf.2015.5.97.  Google Scholar

[11]

M. Kac, Foundations of kinetic theory, Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 3 (1956), 171-197.   Google Scholar

[12]

H. Kushner, Optimal stochastic control, IRE Transactions on Automatic Control, 7 (1962), 120–122. doi: 10.1109/TAC.1962.1105490.  Google Scholar

[13]

X. Li and X. Y. Zhou, Indefinite stochastic LQ controls with Markovian jumps in a finite time horizon, Communications in Information and Systems, 2 (2002), 265-282.  doi: 10.4310/CIS.2002.v2.n3.a4.  Google Scholar

[14]

X. Li, J. Sun and J. Yong, Mean-field stochastic linear quadratic optimal control problems: Closed-loop solvability, Probability, Uncertainty and Quantitative Risk, 1 (2016), Paper No. 2, 24 pp. doi: 10.1186/s41546-016-0002-3.  Google Scholar

[15]

X. Li, X. Y. Zhou and M. Ait Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon, Journal of Global Optimization, 27 (2003), 149-175.  doi: 10.1023/A:1024887007165.  Google Scholar

[16]

X. Li and X. Y. Zhou, Continuous-time mean-variance efficiency: The 80% rule, The Annals of Applied Probability, 16 (2006), 1751-1763.  doi: 10.1214/105051606000000349.  Google Scholar

[17]

Y. Liu, G. Yin and X. Y. Zhou, Near-optimal controls of random-switching LQ problems with indefinite control weight costs, Automatica, 41 (2005), 1063-1070.  doi: 10.1016/j.automatica.2005.01.002.  Google Scholar

[18]

R. Penrose, A generalized inverse for matrices, Mathematical Proceedings of the Cambridge Philosophical Society, 51 (1955), 406-413.  doi: 10.1017/S0305004100030401.  Google Scholar

[19]

J. Sun and J. Yong, Linear quadratic stochastic differential games: open-loop and closed-loop saddle points, SIAM Journal on Control and Optimization, 52 (2014), 4082-4121.  doi: 10.1137/140953642.  Google Scholar

[20]

J. Sun, X. Li and J. Yong, Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM Journal on Control and Optimization, 54 (2016), 2274–2308. doi: 10.1137/15M103532X.  Google Scholar

[21]

J. Sun, Mean-Field Stochastic Linear Quadratic Optimal Control Problems: Open-Loop Solvabilities, ESAIM: Control, Optimisation and Calculus of Variations, 23 (2017), 1099-1127.  doi: 10.1051/cocv/2016023.  Google Scholar

[22]

R. Tao and Z. Wu, Maximum principle for optimal control problems of forward-backward regime-switching system and applications, Systems & Control Letters, 61 (2012), 911-917.  doi: 10.1016/j.sysconle.2012.06.006.  Google Scholar

[23]

W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM Journal on Control, 6 (1968), 681–697. doi: 10.1137/0306044.  Google Scholar

[24]

Z. Wu and X.-R. Wang, FBSDE with Poisson process and its application to linear quadratic stochastic optimal control problem with random jumps, Acta Automatica Sinica, 29 (2003), 821-826.   Google Scholar

[25]

G. G. Yin and Q. Zhang, Continuous-Time Markov Chains and Applications: A Two-Time-Scale Approach, 2nd Edition, Springer, New York, 2013. doi: 10.1007/978-1-4614-4346-9.  Google Scholar

[26]

K.-F. C. Yiu, J. Liu, T. K. Siu and W.-K. Ching, Optimal portfolios with regime switching and value-at-risk constraint, Automatica, 46 (2010), 979-989.  doi: 10.1016/j.automatica.2010.02.027.  Google Scholar

[27]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

[28]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations, SIAM Journal on Control and Optimization, 51 (2013), 2809-2838.  doi: 10.1137/120892477.  Google Scholar

[29]

Z. Yu, Infinite horizon jump-diffusion forward-backward stochastic differential equations and their application to backward linear-quadratic problems, ESAIM: Control, Optimisation and Calculus of Variations, 23 (2017), 1331–1359. doi: 10.1051/cocv/2016055.  Google Scholar

[30]

X. Zhang, R. J. Elliott, T. K. Siu and J. Guo, Markovian regime-switching market completion using additional markov jump assets, IMA Journal of Management Mathematics, 23 (2012), 283–305. doi: 10.1093/imaman/dpr018.  Google Scholar

[31]

X. Zhang and X. Li, Open-Loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems of Markov regime-switching system, arXiv: 1809.01891, 2018. Google Scholar

[32]

X. Zhang, Z. Sun and J. Xiong, A general stochastic maximum principle for a Markov regime switching jump-diffusion model of mean-field type, SIAM Journal on Control and Optimization, 56 (2018), 2563–2592. doi: 10.1137/17M112395X.  Google Scholar

[33]

X. Y. Zhou and G. Yin, Markowitz's mean-variance portfolio selection with regime switching: A continuous-time model, SIAM Journal on Control and Optimization, 42 (2003), 1466-1482.  doi: 10.1137/S0363012902405583.  Google Scholar

show all references

References:
[1]

D. Andersson and B. Djehiche, A maximum principle for SDEs of mean-field type, Applied Mathematics and Optimization, 63 (2011), 341-356.  doi: 10.1007/s00245-010-9123-8.  Google Scholar

[2]

J.-M. Bismut, Linear quadratic optimal stochastic control with random coefficients, SIAM Journal on Control and Optimization, 14 (1976), 419–444. doi: 10.1137/0314028.  Google Scholar

[3]

R. Buckdahn, B. Djehiche and J. Li, A general stochastic maximum principle for SDEs of mean-field type, Applied Mathematics and Optimization, 64 (2011), 197-216.  doi: 10.1007/s00245-011-9136-y.  Google Scholar

[4]

R. Buckdahn, B. Djehiche, J. Li and S. Peng, Mean-field backward stochastic differential equations: A limit approach, Annals of Probability, 37 (2009), 1524-1565.  doi: 10.1214/08-AOP442.  Google Scholar

[5]

X. Cui, X. Li and D. Li, Unified framework of mean-field formulations for optimal multi-period mean-variance portfolio selection, IEEE Transactions on Automatic Control, 59 (2014), 1833–1844. doi: 10.1109/TAC.2014.2311875.  Google Scholar

[6]

C. Donnelly, Suffcient stochastic maximum principle in a regime-switching diffusion model, Applied Mathematics & Optimization, 64 (2011), 155-169.  doi: 10.1007/s00245-010-9130-9.  Google Scholar

[7]

C. Donnelly and A. J. Heunis, Quadratic Risk Minimization in a Regime-Switching Model with Portfolio Constraints, SIAM Journal on Control and Optimization, 50 (2012), 2431–2461. doi: 10.1137/100809271.  Google Scholar

[8]

R. J. Elliott, L. Aggoun and J. B. Moore, Hidden Markov Models: Estimation and Control, Applications of Mathematics, New York: Springer-Verlag, 1995.  Google Scholar

[9]

R. Elliott, X. Li and Y.-H. Ni, Discrete time mean-field stochastic linear-quadratic optimal control problems, Automatica, 49 (2013), 3222-3233.  doi: 10.1016/j.automatica.2013.08.017.  Google Scholar

[10]

J. Huang, X. Li and J. Yong, A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon, Mathematical Control & Related Fields, 5 (2015), 97-139.  doi: 10.3934/mcrf.2015.5.97.  Google Scholar

[11]

M. Kac, Foundations of kinetic theory, Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 3 (1956), 171-197.   Google Scholar

[12]

H. Kushner, Optimal stochastic control, IRE Transactions on Automatic Control, 7 (1962), 120–122. doi: 10.1109/TAC.1962.1105490.  Google Scholar

[13]

X. Li and X. Y. Zhou, Indefinite stochastic LQ controls with Markovian jumps in a finite time horizon, Communications in Information and Systems, 2 (2002), 265-282.  doi: 10.4310/CIS.2002.v2.n3.a4.  Google Scholar

[14]

X. Li, J. Sun and J. Yong, Mean-field stochastic linear quadratic optimal control problems: Closed-loop solvability, Probability, Uncertainty and Quantitative Risk, 1 (2016), Paper No. 2, 24 pp. doi: 10.1186/s41546-016-0002-3.  Google Scholar

[15]

X. Li, X. Y. Zhou and M. Ait Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon, Journal of Global Optimization, 27 (2003), 149-175.  doi: 10.1023/A:1024887007165.  Google Scholar

[16]

X. Li and X. Y. Zhou, Continuous-time mean-variance efficiency: The 80% rule, The Annals of Applied Probability, 16 (2006), 1751-1763.  doi: 10.1214/105051606000000349.  Google Scholar

[17]

Y. Liu, G. Yin and X. Y. Zhou, Near-optimal controls of random-switching LQ problems with indefinite control weight costs, Automatica, 41 (2005), 1063-1070.  doi: 10.1016/j.automatica.2005.01.002.  Google Scholar

[18]

R. Penrose, A generalized inverse for matrices, Mathematical Proceedings of the Cambridge Philosophical Society, 51 (1955), 406-413.  doi: 10.1017/S0305004100030401.  Google Scholar

[19]

J. Sun and J. Yong, Linear quadratic stochastic differential games: open-loop and closed-loop saddle points, SIAM Journal on Control and Optimization, 52 (2014), 4082-4121.  doi: 10.1137/140953642.  Google Scholar

[20]

J. Sun, X. Li and J. Yong, Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems, SIAM Journal on Control and Optimization, 54 (2016), 2274–2308. doi: 10.1137/15M103532X.  Google Scholar

[21]

J. Sun, Mean-Field Stochastic Linear Quadratic Optimal Control Problems: Open-Loop Solvabilities, ESAIM: Control, Optimisation and Calculus of Variations, 23 (2017), 1099-1127.  doi: 10.1051/cocv/2016023.  Google Scholar

[22]

R. Tao and Z. Wu, Maximum principle for optimal control problems of forward-backward regime-switching system and applications, Systems & Control Letters, 61 (2012), 911-917.  doi: 10.1016/j.sysconle.2012.06.006.  Google Scholar

[23]

W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM Journal on Control, 6 (1968), 681–697. doi: 10.1137/0306044.  Google Scholar

[24]

Z. Wu and X.-R. Wang, FBSDE with Poisson process and its application to linear quadratic stochastic optimal control problem with random jumps, Acta Automatica Sinica, 29 (2003), 821-826.   Google Scholar

[25]

G. G. Yin and Q. Zhang, Continuous-Time Markov Chains and Applications: A Two-Time-Scale Approach, 2nd Edition, Springer, New York, 2013. doi: 10.1007/978-1-4614-4346-9.  Google Scholar

[26]

K.-F. C. Yiu, J. Liu, T. K. Siu and W.-K. Ching, Optimal portfolios with regime switching and value-at-risk constraint, Automatica, 46 (2010), 979-989.  doi: 10.1016/j.automatica.2010.02.027.  Google Scholar

[27]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

[28]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations, SIAM Journal on Control and Optimization, 51 (2013), 2809-2838.  doi: 10.1137/120892477.  Google Scholar

[29]

Z. Yu, Infinite horizon jump-diffusion forward-backward stochastic differential equations and their application to backward linear-quadratic problems, ESAIM: Control, Optimisation and Calculus of Variations, 23 (2017), 1331–1359. doi: 10.1051/cocv/2016055.  Google Scholar

[30]

X. Zhang, R. J. Elliott, T. K. Siu and J. Guo, Markovian regime-switching market completion using additional markov jump assets, IMA Journal of Management Mathematics, 23 (2012), 283–305. doi: 10.1093/imaman/dpr018.  Google Scholar

[31]

X. Zhang and X. Li, Open-Loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems of Markov regime-switching system, arXiv: 1809.01891, 2018. Google Scholar

[32]

X. Zhang, Z. Sun and J. Xiong, A general stochastic maximum principle for a Markov regime switching jump-diffusion model of mean-field type, SIAM Journal on Control and Optimization, 56 (2018), 2563–2592. doi: 10.1137/17M112395X.  Google Scholar

[33]

X. Y. Zhou and G. Yin, Markowitz's mean-variance portfolio selection with regime switching: A continuous-time model, SIAM Journal on Control and Optimization, 42 (2003), 1466-1482.  doi: 10.1137/S0363012902405583.  Google Scholar

[1]

Jingrui Sun, Hanxiao Wang. Mean-field stochastic linear-quadratic optimal control problems: Weak closed-loop solvability. Mathematical Control & Related Fields, 2021, 11 (1) : 47-71. doi: 10.3934/mcrf.2020026
[2]

Xun Li, Jingrui Sun, Jiongmin Yong. Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability. Probability, Uncertainty and Quantitative Risk, 2016, 1 (0) : 2-. doi: 10.1186/s41546-016-0002-3
[3]

Alain Bensoussan, Shaokuan Chen, Suresh P. Sethi. Linear quadratic differential games with mixed leadership: The open-loop solution. Numerical Algebra, Control & Optimization, 2013, 3 (1) : 95-108. doi: 10.3934/naco.2013.3.95
[4]

Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control & Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97
[5]

Salah Eddine Choutri, Boualem Djehiche, Hamidou Tembine. Optimal control and zero-sum games for Markov chains of mean-field type. Mathematical Control & Related Fields, 2019, 9 (3) : 571-605. doi: 10.3934/mcrf.2019026
[6]

Hanxiao Wang, Jingrui Sun, Jiongmin Yong. Weak closed-loop solvability of stochastic linear-quadratic optimal control problems. Discrete & Continuous Dynamical Systems, 2019, 39 (5) : 2785-2805. doi: 10.3934/dcds.2019117
[7]

Jiannan Zhang, Ping Chen, Zhuo Jin, Shuanming Li. Open-loop equilibrium strategy for mean-variance portfolio selection: A log-return model. Journal of Industrial & Management Optimization, 2021, 17 (2) : 765-777. doi: 10.3934/jimo.2019133
[8]

Jun Moon. Linear-quadratic mean-field type stackelberg differential games for stochastic jump-diffusion systems. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021026
[9]

Tianxiao Wang. Characterizations of equilibrium controls in time inconsistent mean-field stochastic linear quadratic problems. I. Mathematical Control & Related Fields, 2019, 9 (2) : 385-409. doi: 10.3934/mcrf.2019018
[10]

Jianhui Huang, Shujun Wang, Zhen Wu. Backward-forward linear-quadratic mean-field games with major and minor agents. Probability, Uncertainty and Quantitative Risk, 2016, 1 (0) : 8-. doi: 10.1186/s41546-016-0009-9
[11]

Galina Kurina, Sahlar Meherrem. Decomposition of discrete linear-quadratic optimal control problems for switching systems. Conference Publications, 2015, 2015 (special) : 764-774. doi: 10.3934/proc.2015.0764
[12]

Lin Xu, Rongming Wang, Dingjun Yao. Optimal stochastic investment games under Markov regime switching market. Journal of Industrial & Management Optimization, 2014, 10 (3) : 795-815. doi: 10.3934/jimo.2014.10.795
[13]

Dean A. Carlson. Finding open-loop Nash equilibrium for variational games. Conference Publications, 2005, 2005 (Special) : 153-163. doi: 10.3934/proc.2005.2005.153
[14]

Jiaqin Wei. Time-inconsistent optimal control problems with regime-switching. Mathematical Control & Related Fields, 2017, 7 (4) : 585-622. doi: 10.3934/mcrf.2017022
[15]

Wensheng Yin, Jinde Cao, Yong Ren. Inverse optimal control of regime-switching jump diffusions. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021034
[16]

Lin Xu, Dingjun Yao, Gongpin Cheng. Optimal investment and dividend for an insurer under a Markov regime switching market with high gain tax. Journal of Industrial & Management Optimization, 2020, 16 (1) : 325-356. doi: 10.3934/jimo.2018154
[17]

Jiaqin Wei, Zhuo Jin, Hailiang Yang. Optimal dividend policy with liability constraint under a hidden Markov regime-switching model. Journal of Industrial & Management Optimization, 2019, 15 (4) : 1965-1993. doi: 10.3934/jimo.2018132
[18]

Robert S. Anderssen, Martin Kružík. Modelling of wheat-flour dough mixing as an open-loop hysteretic process. Discrete & Continuous Dynamical Systems - B, 2013, 18 (2) : 283-293. doi: 10.3934/dcdsb.2013.18.283
[19]

Martino Bardi. Explicit solutions of some linear-quadratic mean field games. Networks & Heterogeneous Media, 2012, 7 (2) : 243-261. doi: 10.3934/nhm.2012.7.243
[20]

Kai Du, Jianhui Huang, Zhen Wu. Linear quadratic mean-field-game of backward stochastic differential systems. Mathematical Control & Related Fields, 2018, 8 (3&4) : 653-678. doi: 10.3934/mcrf.2018028

2020 Impact Factor: 1.801

Tools

Article outline

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences