December  2017, 7(4): 585-622. doi: 10.3934/mcrf.2017022

Time-inconsistent optimal control problems with regime-switching

School of Statistics, East China Normal University, 500 Dongchuan Road, Shanghai 200241, China

Received  January 2016 Revised  April 2017 Published  September 2017

In this paper, a time-inconsistent optimal control problem is studied for diffusion processes modulated by a continuous-time Markov chain. In the performance functional, the running cost and terminal cost depend on not only the initial time, but also the initial state of the Markov chain. By modifying the method of multi-person game, we obtain an equilibrium Hamilton-Jacobi-Bellman equation under proper conditions. The well-posedness of this equilibrium HJB Equation is studied in the case where the diffusion term is independent of the control variable. Furthermore, a time-inconsistent linear-quadratic control problem is considered as a special case.

Citation: Jiaqin Wei. Time-inconsistent optimal control problems with regime-switching. Mathematical Control & Related Fields, 2017, 7 (4) : 585-622. doi: 10.3934/mcrf.2017022
References:
[1]

S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation, Review of Financial Studies, 23 (2010), 2970-3016.   Google Scholar

[2]

T. Björk, Finite dimensional optimal filters for a class of ltô-processes with jumping parameters, Stochastics, 4 (1980), 167-183.  doi: 10.1080/17442508008833160.  Google Scholar

[3]

T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochastic control problems, 2010, Working Paper, Stockholm School of Economics. Google Scholar

[4]

T. BjörkA. Murgoci and X. Zhou, Mean–variance portfolio optimization with state-dependent risk aversion, Mathematical Finance, 24 (2014), 1-24.  doi: 10.1111/j.1467-9965.2011.00515.x.  Google Scholar

[5]

J. Buffington and R. Elliott, American options with regime switching, International Journal of Theoretical and Applied Finance, 5 (2002), 497-514.  doi: 10.1142/S0219024902001523.  Google Scholar

[6]

E. Çanakoğlu and S. Özekici, HARA frontiers of optimal portfolios in stochastic markets, European Journal of Operational Research, 221 (2012), 129-137.  doi: 10.1016/j.ejor.2011.10.012.  Google Scholar

[7]

I. Ekeland and A. Lazrak, Being serious about non-commitment: Subgame perfect equilibrium in continuous time, 2006, Preprint. University of British Columbia. Google Scholar

[8]

I. Ekeland and T. Pirvu, Investment and consumption without commitment, Mathematics and Financial Economics, 2 (2008), 57-86.  doi: 10.1007/s11579-008-0014-6.  Google Scholar

[9]

W. Fleming and H. Soner, Controlled Markov Processes and Viscosity Solutions Springer, New York, 2006.  Google Scholar

[10]

A. Friedman, Partial Differential Equations of Parabolic Type Prentice Hall, Inc. , Englewood Cliffs, NJ, 1964.  Google Scholar

[11]

S. M. Goldman, Consistent plans, Rev. Financ. Stud., 47 (1980), 533-537.  doi: 10.2307/2297304.  Google Scholar

[12]

D. Laibson, Golden eggs and hyperbolic discounting, Quarterly Journal of Economics, 112 (1997), 443-478.  doi: 10.1162/003355397555253.  Google Scholar

[13]

B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing, Rev. Financ. Stud., 40 (1973), 391-401.  doi: 10.2307/2296458.  Google Scholar

[14]

R. A. Pollak, Consistent planning, Rev. Financ. Stud., 35 (1968), 201-208.  doi: 10.2307/2296548.  Google Scholar

[15]

P. Protter, Stochastic Integration and Differential Equations Springer, Berlin, Heidelberg, 2005. Google Scholar

[16]

L. Sotomayor, Stochastic Control with Regime Switching and its Applications to Financial Economics PhD thesis, University of Alberta, 2008.  Google Scholar

[17]

L. Sotomayor and A. Cadenillas, Explicit solutions of consumption-investment problems in financial markets with regime switching, Mathematical Finance, 19 (2009), 251-279.  doi: 10.1111/j.1467-9965.2009.00366.x.  Google Scholar

[18]

R. Strotz, Myopia and inconsistency in dynamic utility maximization, Readings in Welfare Economics, (1973), 128-143.  doi: 10.1007/978-1-349-15492-0_10.  Google Scholar

[19]

H. Wang and Z. Wu, Time-inconsistent optimal control problem with random coefficients and stochastic equilibrium hjb equation, Mathematical Control and Related Fields, 5 (2015), 651-678.  doi: 10.3934/mcrf.2015.5.651.  Google Scholar

[20]

J. WeiR. Wang and H. Yang, On the optimal dividend strategy in a regime-switching diffusion model, Advances in Applied Probability, 44 (2012), 886-906.  doi: 10.1017/S0001867800005929.  Google Scholar

[21]

J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem, Mathematical Control and Related Fields, 1 (2011), 83-118.  doi: 10.3934/mcrf.2011.1.83.  Google Scholar

[22]

J. Yong, Time-inconsistent optimal control problems and the equilibrium HJB equation, Mathematical Control and Related Fields, 2 (2012), 271-329.  doi: 10.3934/mcrf.2012.2.271.  Google Scholar

[23]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations-time-consistent solutions, SIAM J. Control Optim. , 51 (2013), 2809–2838, arXiv: 1304.3964 [math. OC]. doi: 10.1137/120892477.  Google Scholar

[24]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

[25]

Q. Zhang and G. Yin, Nearly-optimal asset allocation in hybrid stock investment models, Journal of Optimization Theory and Applications, 121 (2004), 419-444.  doi: 10.1023/B:JOTA.0000037412.23243.6c.  Google Scholar

[26]

Q. ZhaoY. Shen and J. Wei, Consumption–investment strategies with non-exponential discounting and logarithmic utility, European Journal of Operational Research, 238 (2014), 824-835.  doi: 10.1016/j.ejor.2014.04.034.  Google Scholar

[27]

X. Zhou and G. Yin, Markowitz's mean-variance portfolio selection with regime switching: A continuous-time model, SIAM Journal on Control and Optimization, 42 (2003), 1466-1482.  doi: 10.1137/S0363012902405583.  Google Scholar

show all references

References:
[1]

S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation, Review of Financial Studies, 23 (2010), 2970-3016.   Google Scholar

[2]

T. Björk, Finite dimensional optimal filters for a class of ltô-processes with jumping parameters, Stochastics, 4 (1980), 167-183.  doi: 10.1080/17442508008833160.  Google Scholar

[3]

T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochastic control problems, 2010, Working Paper, Stockholm School of Economics. Google Scholar

[4]

T. BjörkA. Murgoci and X. Zhou, Mean–variance portfolio optimization with state-dependent risk aversion, Mathematical Finance, 24 (2014), 1-24.  doi: 10.1111/j.1467-9965.2011.00515.x.  Google Scholar

[5]

J. Buffington and R. Elliott, American options with regime switching, International Journal of Theoretical and Applied Finance, 5 (2002), 497-514.  doi: 10.1142/S0219024902001523.  Google Scholar

[6]

E. Çanakoğlu and S. Özekici, HARA frontiers of optimal portfolios in stochastic markets, European Journal of Operational Research, 221 (2012), 129-137.  doi: 10.1016/j.ejor.2011.10.012.  Google Scholar

[7]

I. Ekeland and A. Lazrak, Being serious about non-commitment: Subgame perfect equilibrium in continuous time, 2006, Preprint. University of British Columbia. Google Scholar

[8]

I. Ekeland and T. Pirvu, Investment and consumption without commitment, Mathematics and Financial Economics, 2 (2008), 57-86.  doi: 10.1007/s11579-008-0014-6.  Google Scholar

[9]

W. Fleming and H. Soner, Controlled Markov Processes and Viscosity Solutions Springer, New York, 2006.  Google Scholar

[10]

A. Friedman, Partial Differential Equations of Parabolic Type Prentice Hall, Inc. , Englewood Cliffs, NJ, 1964.  Google Scholar

[11]

S. M. Goldman, Consistent plans, Rev. Financ. Stud., 47 (1980), 533-537.  doi: 10.2307/2297304.  Google Scholar

[12]

D. Laibson, Golden eggs and hyperbolic discounting, Quarterly Journal of Economics, 112 (1997), 443-478.  doi: 10.1162/003355397555253.  Google Scholar

[13]

B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing, Rev. Financ. Stud., 40 (1973), 391-401.  doi: 10.2307/2296458.  Google Scholar

[14]

R. A. Pollak, Consistent planning, Rev. Financ. Stud., 35 (1968), 201-208.  doi: 10.2307/2296548.  Google Scholar

[15]

P. Protter, Stochastic Integration and Differential Equations Springer, Berlin, Heidelberg, 2005. Google Scholar

[16]

L. Sotomayor, Stochastic Control with Regime Switching and its Applications to Financial Economics PhD thesis, University of Alberta, 2008.  Google Scholar

[17]

L. Sotomayor and A. Cadenillas, Explicit solutions of consumption-investment problems in financial markets with regime switching, Mathematical Finance, 19 (2009), 251-279.  doi: 10.1111/j.1467-9965.2009.00366.x.  Google Scholar

[18]

R. Strotz, Myopia and inconsistency in dynamic utility maximization, Readings in Welfare Economics, (1973), 128-143.  doi: 10.1007/978-1-349-15492-0_10.  Google Scholar

[19]

H. Wang and Z. Wu, Time-inconsistent optimal control problem with random coefficients and stochastic equilibrium hjb equation, Mathematical Control and Related Fields, 5 (2015), 651-678.  doi: 10.3934/mcrf.2015.5.651.  Google Scholar

[20]

J. WeiR. Wang and H. Yang, On the optimal dividend strategy in a regime-switching diffusion model, Advances in Applied Probability, 44 (2012), 886-906.  doi: 10.1017/S0001867800005929.  Google Scholar

[21]

J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem, Mathematical Control and Related Fields, 1 (2011), 83-118.  doi: 10.3934/mcrf.2011.1.83.  Google Scholar

[22]

J. Yong, Time-inconsistent optimal control problems and the equilibrium HJB equation, Mathematical Control and Related Fields, 2 (2012), 271-329.  doi: 10.3934/mcrf.2012.2.271.  Google Scholar

[23]

J. Yong, Linear-quadratic optimal control problems for mean-field stochastic differential equations-time-consistent solutions, SIAM J. Control Optim. , 51 (2013), 2809–2838, arXiv: 1304.3964 [math. OC]. doi: 10.1137/120892477.  Google Scholar

[24]

J. Yong and X. Y. Zhou, Stochastic Controls: Hamiltonian Systems and HJB Equations Springer-Verlag, New York, 1999. doi: 10.1007/978-1-4612-1466-3.  Google Scholar

[25]

Q. Zhang and G. Yin, Nearly-optimal asset allocation in hybrid stock investment models, Journal of Optimization Theory and Applications, 121 (2004), 419-444.  doi: 10.1023/B:JOTA.0000037412.23243.6c.  Google Scholar

[26]

Q. ZhaoY. Shen and J. Wei, Consumption–investment strategies with non-exponential discounting and logarithmic utility, European Journal of Operational Research, 238 (2014), 824-835.  doi: 10.1016/j.ejor.2014.04.034.  Google Scholar

[27]

X. Zhou and G. Yin, Markowitz's mean-variance portfolio selection with regime switching: A continuous-time model, SIAM Journal on Control and Optimization, 42 (2003), 1466-1482.  doi: 10.1137/S0363012902405583.  Google Scholar

Figure 1.  The solutions for $P(1,t,1)$ and $P(2,t,2)$
[1]

Siyang Cai, Yongmei Cai, Xuerong Mao. A stochastic differential equation SIS epidemic model with regime switching. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020317

[2]

Awais Younus, Zoubia Dastgeer, Nudrat Ishaq, Abdul Ghaffar, Kottakkaran Sooppy Nisar, Devendra Kumar. On the observability of conformable linear time-invariant control systems. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020444

[3]

Sergey Rashkovskiy. Hamilton-Jacobi theory for Hamiltonian and non-Hamiltonian systems. Journal of Geometric Mechanics, 2020, 12 (4) : 563-583. doi: 10.3934/jgm.2020024

[4]

Youming Guo, Tingting Li. Optimal control strategies for an online game addiction model with low and high risk exposure. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020347

[5]

Hirokazu Ninomiya. Entire solutions of the Allen–Cahn–Nagumo equation in a multi-dimensional space. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 395-412. doi: 10.3934/dcds.2020364

[6]

Lars Grüne, Matthias A. Müller, Christopher M. Kellett, Steven R. Weller. Strict dissipativity for discrete time discounted optimal control problems. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020046

[7]

Peng Luo. Comparison theorem for diagonally quadratic BSDEs. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020374

[8]

Huu-Quang Nguyen, Ya-Chi Chu, Ruey-Lin Sheu. On the convexity for the range set of two quadratic functions. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020169

[9]

Shasha Hu, Yihong Xu, Yuhan Zhang. Second-Order characterizations for set-valued equilibrium problems with variable ordering structures. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020164

[10]

Yueyang Zheng, Jingtao Shi. A stackelberg game of backward stochastic differential equations with partial information. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020047

[11]

Juan Pablo Pinasco, Mauro Rodriguez Cartabia, Nicolas Saintier. Evolutionary game theory in mixed strategies: From microscopic interactions to kinetic equations. Kinetic & Related Models, , () : -. doi: 10.3934/krm.2020051

[12]

Djamel Aaid, Amel Noui, Özen Özer. Piecewise quadratic bounding functions for finding real roots of polynomials. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 63-73. doi: 10.3934/naco.2020015

[13]

Hong Niu, Zhijiang Feng, Qijin Xiao, Yajun Zhang. A PID control method based on optimal control strategy. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 117-126. doi: 10.3934/naco.2020019

[14]

Shun Zhang, Jianlin Jiang, Su Zhang, Yibing Lv, Yuzhen Guo. ADMM-type methods for generalized multi-facility Weber problem. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020171

[15]

Touria Karite, Ali Boutoulout. Global and regional constrained controllability for distributed parabolic linear systems: RHUM approach. Numerical Algebra, Control & Optimization, 2020  doi: 10.3934/naco.2020055

[16]

Noufel Frikha, Valentin Konakov, Stéphane Menozzi. Well-posedness of some non-linear stable driven SDEs. Discrete & Continuous Dynamical Systems - A, 2021, 41 (2) : 849-898. doi: 10.3934/dcds.2020302

[17]

Ripeng Huang, Shaojian Qu, Xiaoguang Yang, Zhimin Liu. Multi-stage distributionally robust optimization with risk aversion. Journal of Industrial & Management Optimization, 2021, 17 (1) : 233-259. doi: 10.3934/jimo.2019109

[18]

Hongguang Ma, Xiang Li. Multi-period hazardous waste collection planning with consideration of risk stability. Journal of Industrial & Management Optimization, 2021, 17 (1) : 393-408. doi: 10.3934/jimo.2019117

[19]

Sören Bartels, Jakob Keck. Adaptive time stepping in elastoplasticity. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 71-88. doi: 10.3934/dcdss.2020323

[20]

Wenmeng Geng, Kai Tao. Large deviation theorems for dirichlet determinants of analytic quasi-periodic jacobi operators with Brjuno-Rüssmann frequency. Communications on Pure & Applied Analysis, 2020, 19 (12) : 5305-5335. doi: 10.3934/cpaa.2020240

2019 Impact Factor: 0.857

Metrics

  • PDF downloads (131)
  • HTML views (183)
  • Cited by (4)

Other articles
by authors

[Back to Top]