American Institute of Mathematical Sciences

Advanced
September  2012, 2(3): 271-329. doi: 10.3934/mcrf.2012.2.271

Time-inconsistent optimal control problems and the equilibrium HJB equation

Jiongmin Yong 1,

1. 

Department of Mathematics, University of Central Florida, Orlando, FL 32816

Received  April 2012 Revised  May 2012 Published  August 2012

A general time-inconsistent optimal control problem is considered for stochastic differential equations with deterministic coefficients. Under suitable conditions, a Hamilton-Jacobi-Bellman type equation is derived for the equilibrium value function of the problem. Well-posedness such an equation is studied, and time-consistent equilibrium strategies are constructed. As special cases, the linear-quadratic problem and a generalized Merton's portfolio problem are investigated.
Keywords: Time-inconsistent optimal control problem, forward-backward stochastic differential equation., equilibrium value function, equilibrium Hamilton-Jacobi-Bellman equation.
Mathematics Subject Classification: Primary: 93E20, 49L20, 49N10, 49N70; Secondary: 35Q93,91A23, 91A6.
Citation: Jiongmin Yong. Time-inconsistent optimal control problems and the equilibrium HJB equation. Mathematical Control & Related Fields, 2012, 2 (3) : 271-329. doi: 10.3934/mcrf.2012.2.271
References:
[1]

S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation,, Rev. Finan. Stud., 23 (2010), 2970.  doi: 10.1093/rfs/hhq028.  Google Scholar

[2]

T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochasitic control problem,, working paper., ().   Google Scholar

[3]

T. Björk, A. Murgoci and X. Y. Zhou, Mean varaiance portfolio optimization with state dependent risk aversion,, Math. Finance, ().   Google Scholar

[4]

E. V. Böhm-Bawerk, "The Positive Theory of Capital,'', Books for Libraries Press, (1891).   Google Scholar

[5]

A. Caplin and J. Leahy, The recursive approach to time inconsistency,, J. Econ. Theory, 131 (2006), 134.  doi: 10.1016/j.jet.2005.05.006.  Google Scholar

[6]

I. Ekeland and A. Lazrak, The golden rule when preferences are time inconsistent,, Math. Finan. Econ., 4 (2010), 29.   Google Scholar

[7]

I. Ekeland and T. A. Pirvu, Investment and consumption without commitment,, Math. Finan. Econ., 2 (2008), 57.   Google Scholar

[8]

I. Ekeland, O. Mbodji and T. A. Pirvu, Time-consistent portfolio management,, SIAM J. Financial Math., 3 (2012), 1.   Google Scholar

[9]

W. H. Fleming and H. M. Soner, "Controlled Markov Processes and Viscosity Solutions,'', 2nd edition, 25 (2006).   Google Scholar

[10]

A. Friedman, "Partial Differential Equations of Parabolic Type,'', Prentice Hall, (1964).   Google Scholar

[11]

S. M. Goldman, Consistent plans,, Review of Economic Studies, 47 (1980), 533.   Google Scholar

[12]

S. R. Grenadier and N. Wang, Investment under uncertainty and time-inconsistent preferences,, J. Finan. Econ., 84 (2007), 2.   Google Scholar

[13]

P. J.-J. Herings and K. I. M. Rohde, Time-inconsistent preferences in a general equilibrium model,, Econ. Theory, 29 (2006), 591.  doi: 10.1007/s00199-005-0020-3.  Google Scholar

[14]

Y. Hu, H. Jin and X. Y. Zhou, Time-inconsistent stochastic linear-quadratic control,, , (2011).   Google Scholar

[15]

D. Hume, "A Treatise of Human Nature,'', First edition, (1739).   Google Scholar

[16]

W. S. Jevons, "Theory of Political Economy,'', McMillan, (1871).   Google Scholar

[17]

L. Karp and I. H. Lee, Time-consistent policies,, J. Econ. Theory, 112 (2003), 353.  doi: 10.1016/S0022-0531(03)00067-X.  Google Scholar

[18]

D. Laibson, Golden eggs and hyperbolic discounting,, Quarterly J. Econ., 112 (1997), 443.  doi: 10.1162/003355397555253.  Google Scholar

[19]

J. Ma, P. Protter and J. Yong, Solving forward-backward stochastic differential equations explicitly-a four-step scheme,, Probab. Theory Related Fields, 98 (1994), 339.  doi: 10.1007/BF01192258.  Google Scholar

[20]

J. Ma and J. Yong, "Forward-Backward Stochastic Differential Equations and Their Applications,'', Lecture Notes in Math., 1702 (1999).   Google Scholar

[21]

A. Malthus, An essay on the principle of population,, in, (1986), 2.   Google Scholar

[22]

A. Marshall, "Principles of Economics,'', 1st ed., (1890).   Google Scholar

[23]

J. Marin-Solano and J. Navas, Consumption and portfolio rules for time-inconsistent investors,, European J. Operational Research, 201 (2010), 860.  doi: 10.1016/j.ejor.2009.04.005.  Google Scholar

[24]

J. Marin-Solano and E. V. Shevkoplyas, Non-constant discounting and differential games with random time horizon,, Automatica J. IFAC, 47 (2011), 2626.  doi: 10.1016/j.automatica.2011.09.010.  Google Scholar

[25]

M. Miller and M. Salmon, Dynamic games and the time inconsistency of optimal policy in open economics,, The Economic Journal, 95 (1985), 124.  doi: 10.2307/2232876.  Google Scholar

[26]

I. Palacios-Huerta, Time-inconsistent preferences in Adam Smith and Davis Hume,, History of Political Economy, 35 (2003), 241.  doi: 10.1215/00182702-35-2-241.  Google Scholar

[27]

V. Pareto, "Manuel d'Économie Politique,'', Girard and Brieve, (1909).   Google Scholar

[28]

B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing,, Review of Economic Studies, 40 (1973), 391.   Google Scholar

[29]

R. A. Pollak, Consistent planning,, Review of Economic Studies, 35 (1968), 185.  doi: 10.2307/2296547.  Google Scholar

[30]

A. Smith, "The Theory of Moral Sentiments,'', First edition, (1759).   Google Scholar

[31]

R. H. Strotz, Myopia and inconsistency indynamic utility maximization,, Review of Econ. Studies, 23 (1955), 165.  doi: 10.2307/2295722.  Google Scholar

[32]

L. Tesfatsion, Time inconsistency of benevolent government economics,, J. Public Economics, 31 (1986), 25.  doi: 10.1016/0047-2727(86)90070-8.  Google Scholar

[33]

J. Yong, Backward stochastic Volterra integral equations and some related problems,, Stoch. Proc. Appl., 116 (2006), 779.  doi: 10.1016/j.spa.2006.01.005.  Google Scholar

[34]

J. Yong, Well-posedness and regularity of backward stochastic Volterra integral equations,, Prob. Theory Rel. Fields, 142 (2008), 21.  doi: 10.1007/s00440-007-0098-6.  Google Scholar

[35]

J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem,, Math. Control Related Fields, 1 (2011), 83.   Google Scholar

[36]

J. Yong, Deterministic time-inconsistent optimal control problems-An essentially cooperative approach,, Acta Math. Appl. Sinica Engl. Ser., 28 (2012), 1.   Google Scholar

[37]

J. Yong and X. Y. Zhou, "Stochastic Controls. Hamiltonian Systems and HJB Equations,'', Applications of Mathematics (New York), 43 (1999).   Google Scholar

show all references

References:
[1]

S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation,, Rev. Finan. Stud., 23 (2010), 2970.  doi: 10.1093/rfs/hhq028.  Google Scholar

[2]

T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochasitic control problem,, working paper., ().   Google Scholar

[3]

T. Björk, A. Murgoci and X. Y. Zhou, Mean varaiance portfolio optimization with state dependent risk aversion,, Math. Finance, ().   Google Scholar

[4]

E. V. Böhm-Bawerk, "The Positive Theory of Capital,'', Books for Libraries Press, (1891).   Google Scholar

[5]

A. Caplin and J. Leahy, The recursive approach to time inconsistency,, J. Econ. Theory, 131 (2006), 134.  doi: 10.1016/j.jet.2005.05.006.  Google Scholar

[6]

I. Ekeland and A. Lazrak, The golden rule when preferences are time inconsistent,, Math. Finan. Econ., 4 (2010), 29.   Google Scholar

[7]

I. Ekeland and T. A. Pirvu, Investment and consumption without commitment,, Math. Finan. Econ., 2 (2008), 57.   Google Scholar

[8]

I. Ekeland, O. Mbodji and T. A. Pirvu, Time-consistent portfolio management,, SIAM J. Financial Math., 3 (2012), 1.   Google Scholar

[9]

W. H. Fleming and H. M. Soner, "Controlled Markov Processes and Viscosity Solutions,'', 2nd edition, 25 (2006).   Google Scholar

[10]

A. Friedman, "Partial Differential Equations of Parabolic Type,'', Prentice Hall, (1964).   Google Scholar

[11]

S. M. Goldman, Consistent plans,, Review of Economic Studies, 47 (1980), 533.   Google Scholar

[12]

S. R. Grenadier and N. Wang, Investment under uncertainty and time-inconsistent preferences,, J. Finan. Econ., 84 (2007), 2.   Google Scholar

[13]

P. J.-J. Herings and K. I. M. Rohde, Time-inconsistent preferences in a general equilibrium model,, Econ. Theory, 29 (2006), 591.  doi: 10.1007/s00199-005-0020-3.  Google Scholar

[14]

Y. Hu, H. Jin and X. Y. Zhou, Time-inconsistent stochastic linear-quadratic control,, , (2011).   Google Scholar

[15]

D. Hume, "A Treatise of Human Nature,'', First edition, (1739).   Google Scholar

[16]

W. S. Jevons, "Theory of Political Economy,'', McMillan, (1871).   Google Scholar

[17]

L. Karp and I. H. Lee, Time-consistent policies,, J. Econ. Theory, 112 (2003), 353.  doi: 10.1016/S0022-0531(03)00067-X.  Google Scholar

[18]

D. Laibson, Golden eggs and hyperbolic discounting,, Quarterly J. Econ., 112 (1997), 443.  doi: 10.1162/003355397555253.  Google Scholar

[19]

J. Ma, P. Protter and J. Yong, Solving forward-backward stochastic differential equations explicitly-a four-step scheme,, Probab. Theory Related Fields, 98 (1994), 339.  doi: 10.1007/BF01192258.  Google Scholar

[20]

J. Ma and J. Yong, "Forward-Backward Stochastic Differential Equations and Their Applications,'', Lecture Notes in Math., 1702 (1999).   Google Scholar

[21]

A. Malthus, An essay on the principle of population,, in, (1986), 2.   Google Scholar

[22]

A. Marshall, "Principles of Economics,'', 1st ed., (1890).   Google Scholar

[23]

J. Marin-Solano and J. Navas, Consumption and portfolio rules for time-inconsistent investors,, European J. Operational Research, 201 (2010), 860.  doi: 10.1016/j.ejor.2009.04.005.  Google Scholar

[24]

J. Marin-Solano and E. V. Shevkoplyas, Non-constant discounting and differential games with random time horizon,, Automatica J. IFAC, 47 (2011), 2626.  doi: 10.1016/j.automatica.2011.09.010.  Google Scholar

[25]

M. Miller and M. Salmon, Dynamic games and the time inconsistency of optimal policy in open economics,, The Economic Journal, 95 (1985), 124.  doi: 10.2307/2232876.  Google Scholar

[26]

I. Palacios-Huerta, Time-inconsistent preferences in Adam Smith and Davis Hume,, History of Political Economy, 35 (2003), 241.  doi: 10.1215/00182702-35-2-241.  Google Scholar

[27]

V. Pareto, "Manuel d'Économie Politique,'', Girard and Brieve, (1909).   Google Scholar

[28]

B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing,, Review of Economic Studies, 40 (1973), 391.   Google Scholar

[29]

R. A. Pollak, Consistent planning,, Review of Economic Studies, 35 (1968), 185.  doi: 10.2307/2296547.  Google Scholar

[30]

A. Smith, "The Theory of Moral Sentiments,'', First edition, (1759).   Google Scholar

[31]

R. H. Strotz, Myopia and inconsistency indynamic utility maximization,, Review of Econ. Studies, 23 (1955), 165.  doi: 10.2307/2295722.  Google Scholar

[32]

L. Tesfatsion, Time inconsistency of benevolent government economics,, J. Public Economics, 31 (1986), 25.  doi: 10.1016/0047-2727(86)90070-8.  Google Scholar

[33]

J. Yong, Backward stochastic Volterra integral equations and some related problems,, Stoch. Proc. Appl., 116 (2006), 779.  doi: 10.1016/j.spa.2006.01.005.  Google Scholar

[34]

J. Yong, Well-posedness and regularity of backward stochastic Volterra integral equations,, Prob. Theory Rel. Fields, 142 (2008), 21.  doi: 10.1007/s00440-007-0098-6.  Google Scholar

[35]

J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem,, Math. Control Related Fields, 1 (2011), 83.   Google Scholar

[36]

J. Yong, Deterministic time-inconsistent optimal control problems-An essentially cooperative approach,, Acta Math. Appl. Sinica Engl. Ser., 28 (2012), 1.   Google Scholar

[37]

J. Yong and X. Y. Zhou, "Stochastic Controls. Hamiltonian Systems and HJB Equations,'', Applications of Mathematics (New York), 43 (1999).   Google Scholar

[1]

Stefano Bianchini, Paolo Bonicatto. Forward untangling and applications to the uniqueness problem for the continuity equation. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020384
[2]

Editorial Office. Retraction: Xiao-Qian Jiang and Lun-Chuan Zhang, A pricing option approach based on backward stochastic differential equation theory. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 969-969. doi: 10.3934/dcdss.2019065
[3]

Yi Zhou, Jianli Liu. The initial-boundary value problem on a strip for the equation of time-like extremal surfaces. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 381-397. doi: 10.3934/dcds.2009.23.381
[4]

Vo Van Au, Hossein Jafari, Zakia Hammouch, Nguyen Huy Tuan. On a final value problem for a nonlinear fractional pseudo-parabolic equation. Electronic Research Archive, 2021, 29 (1) : 1709-1734. doi: 10.3934/era.2020088
[5]

Siyang Cai, Yongmei Cai, Xuerong Mao. A stochastic differential equation SIS epidemic model with regime switching. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020317
[6]

Tetsuya Ishiwata, Young Chol Yang. Numerical and mathematical analysis of blow-up problems for a stochastic differential equation. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 909-918. doi: 10.3934/dcdss.2020391
[7]

Yueyang Zheng, Jingtao Shi. A stackelberg game of backward stochastic differential equations with partial information. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020047
[8]

Christian Clason, Vu Huu Nhu, Arnd Rösch. Optimal control of a non-smooth quasilinear elliptic equation. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020052
[9]

Olivier Ley, Erwin Topp, Miguel Yangari. Some results for the large time behavior of Hamilton-Jacobi equations with Caputo time derivative. Discrete & Continuous Dynamical Systems - A, 2021  doi: 10.3934/dcds.2021007
[10]

Guangjun Shen, Xueying Wu, Xiuwei Yin. Stabilization of stochastic differential equations driven by G-Lévy process with discrete-time feedback control. Discrete & Continuous Dynamical Systems - B, 2021, 26 (2) : 755-774. doi: 10.3934/dcdsb.2020133
[11]

Manuel Friedrich, Martin Kružík, Ulisse Stefanelli. Equilibrium of immersed hyperelastic solids. Discrete & Continuous Dynamical Systems - S, 2021  doi: 10.3934/dcdss.2021003
[12]

Kerioui Nadjah, Abdelouahab Mohammed Salah. Stability and Hopf bifurcation of the coexistence equilibrium for a differential-algebraic biological economic system with predator harvesting. Electronic Research Archive, 2021, 29 (1) : 1641-1660. doi: 10.3934/era.2020084
[13]

Qingfeng Zhu, Yufeng Shi. Nonzero-sum differential game of backward doubly stochastic systems with delay and applications. Mathematical Control & Related Fields, 2021, 11 (1) : 73-94. doi: 10.3934/mcrf.2020028
[14]

Hai Huang, Xianlong Fu. Optimal control problems for a neutral integro-differential system with infinite delay. Evolution Equations & Control Theory, 2020  doi: 10.3934/eect.2020107
[15]

Yukihiko Nakata. Existence of a period two solution of a delay differential equation. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 1103-1110. doi: 10.3934/dcdss.2020392
[16]

Lars Grüne, Matthias A. Müller, Christopher M. Kellett, Steven R. Weller. Strict dissipativity for discrete time discounted optimal control problems. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020046
[17]

Sergey Rashkovskiy. Hamilton-Jacobi theory for Hamiltonian and non-Hamiltonian systems. Journal of Geometric Mechanics, 2020, 12 (4) : 563-583. doi: 10.3934/jgm.2020024
[18]

Yuan Tan, Qingyuan Cao, Lan Li, Tianshi Hu, Min Su. A chance-constrained stochastic model predictive control problem with disturbance feedback. Journal of Industrial & Management Optimization, 2021, 17 (1) : 67-79. doi: 10.3934/jimo.2019099
[19]

Elimhan N. Mahmudov. Infimal convolution and duality in convex optimal control problems with second order evolution differential inclusions. Evolution Equations & Control Theory, 2021, 10 (1) : 37-59. doi: 10.3934/eect.2020051
[20]

Evelyn Sander, Thomas Wanner. Equilibrium validation in models for pattern formation based on Sobolev embeddings. Discrete & Continuous Dynamical Systems - B, 2021, 26 (1) : 603-632. doi: 10.3934/dcdsb.2020260

2019 Impact Factor: 0.857

Tools

Metrics

  • PDF downloads (71)
  • HTML views (0)
  • Cited by (49)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences