September  2012, 2(3): 271-329. doi: 10.3934/mcrf.2012.2.271

Time-inconsistent optimal control problems and the equilibrium HJB equation

1. 

Department of Mathematics, University of Central Florida, Orlando, FL 32816

Received  April 2012 Revised  May 2012 Published  August 2012

A general time-inconsistent optimal control problem is considered for stochastic differential equations with deterministic coefficients. Under suitable conditions, a Hamilton-Jacobi-Bellman type equation is derived for the equilibrium value function of the problem. Well-posedness such an equation is studied, and time-consistent equilibrium strategies are constructed. As special cases, the linear-quadratic problem and a generalized Merton's portfolio problem are investigated.
Citation: Jiongmin Yong. Time-inconsistent optimal control problems and the equilibrium HJB equation. Mathematical Control & Related Fields, 2012, 2 (3) : 271-329. doi: 10.3934/mcrf.2012.2.271
References:
[1]

S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation,, Rev. Finan. Stud., 23 (2010), 2970.  doi: 10.1093/rfs/hhq028.  Google Scholar

[2]

T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochasitic control problem,, working paper., ().   Google Scholar

[3]

T. Björk, A. Murgoci and X. Y. Zhou, Mean varaiance portfolio optimization with state dependent risk aversion,, Math. Finance, ().   Google Scholar

[4]

E. V. Böhm-Bawerk, "The Positive Theory of Capital,'', Books for Libraries Press, (1891).   Google Scholar

[5]

A. Caplin and J. Leahy, The recursive approach to time inconsistency,, J. Econ. Theory, 131 (2006), 134.  doi: 10.1016/j.jet.2005.05.006.  Google Scholar

[6]

I. Ekeland and A. Lazrak, The golden rule when preferences are time inconsistent,, Math. Finan. Econ., 4 (2010), 29.   Google Scholar

[7]

I. Ekeland and T. A. Pirvu, Investment and consumption without commitment,, Math. Finan. Econ., 2 (2008), 57.   Google Scholar

[8]

I. Ekeland, O. Mbodji and T. A. Pirvu, Time-consistent portfolio management,, SIAM J. Financial Math., 3 (2012), 1.   Google Scholar

[9]

W. H. Fleming and H. M. Soner, "Controlled Markov Processes and Viscosity Solutions,'', 2nd edition, 25 (2006).   Google Scholar

[10]

A. Friedman, "Partial Differential Equations of Parabolic Type,'', Prentice Hall, (1964).   Google Scholar

[11]

S. M. Goldman, Consistent plans,, Review of Economic Studies, 47 (1980), 533.   Google Scholar

[12]

S. R. Grenadier and N. Wang, Investment under uncertainty and time-inconsistent preferences,, J. Finan. Econ., 84 (2007), 2.   Google Scholar

[13]

P. J.-J. Herings and K. I. M. Rohde, Time-inconsistent preferences in a general equilibrium model,, Econ. Theory, 29 (2006), 591.  doi: 10.1007/s00199-005-0020-3.  Google Scholar

[14]

Y. Hu, H. Jin and X. Y. Zhou, Time-inconsistent stochastic linear-quadratic control,, , (2011).   Google Scholar

[15]

D. Hume, "A Treatise of Human Nature,'', First edition, (1739).   Google Scholar

[16]

W. S. Jevons, "Theory of Political Economy,'', McMillan, (1871).   Google Scholar

[17]

L. Karp and I. H. Lee, Time-consistent policies,, J. Econ. Theory, 112 (2003), 353.  doi: 10.1016/S0022-0531(03)00067-X.  Google Scholar

[18]

D. Laibson, Golden eggs and hyperbolic discounting,, Quarterly J. Econ., 112 (1997), 443.  doi: 10.1162/003355397555253.  Google Scholar

[19]

J. Ma, P. Protter and J. Yong, Solving forward-backward stochastic differential equations explicitly-a four-step scheme,, Probab. Theory Related Fields, 98 (1994), 339.  doi: 10.1007/BF01192258.  Google Scholar

[20]

J. Ma and J. Yong, "Forward-Backward Stochastic Differential Equations and Their Applications,'', Lecture Notes in Math., 1702 (1999).   Google Scholar

[21]

A. Malthus, An essay on the principle of population,, in, (1986), 2.   Google Scholar

[22]

A. Marshall, "Principles of Economics,'', 1st ed., (1890).   Google Scholar

[23]

J. Marin-Solano and J. Navas, Consumption and portfolio rules for time-inconsistent investors,, European J. Operational Research, 201 (2010), 860.  doi: 10.1016/j.ejor.2009.04.005.  Google Scholar

[24]

J. Marin-Solano and E. V. Shevkoplyas, Non-constant discounting and differential games with random time horizon,, Automatica J. IFAC, 47 (2011), 2626.  doi: 10.1016/j.automatica.2011.09.010.  Google Scholar

[25]

M. Miller and M. Salmon, Dynamic games and the time inconsistency of optimal policy in open economics,, The Economic Journal, 95 (1985), 124.  doi: 10.2307/2232876.  Google Scholar

[26]

I. Palacios-Huerta, Time-inconsistent preferences in Adam Smith and Davis Hume,, History of Political Economy, 35 (2003), 241.  doi: 10.1215/00182702-35-2-241.  Google Scholar

[27]

V. Pareto, "Manuel d'Économie Politique,'', Girard and Brieve, (1909).   Google Scholar

[28]

B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing,, Review of Economic Studies, 40 (1973), 391.   Google Scholar

[29]

R. A. Pollak, Consistent planning,, Review of Economic Studies, 35 (1968), 185.  doi: 10.2307/2296547.  Google Scholar

[30]

A. Smith, "The Theory of Moral Sentiments,'', First edition, (1759).   Google Scholar

[31]

R. H. Strotz, Myopia and inconsistency indynamic utility maximization,, Review of Econ. Studies, 23 (1955), 165.  doi: 10.2307/2295722.  Google Scholar

[32]

L. Tesfatsion, Time inconsistency of benevolent government economics,, J. Public Economics, 31 (1986), 25.  doi: 10.1016/0047-2727(86)90070-8.  Google Scholar

[33]

J. Yong, Backward stochastic Volterra integral equations and some related problems,, Stoch. Proc. Appl., 116 (2006), 779.  doi: 10.1016/j.spa.2006.01.005.  Google Scholar

[34]

J. Yong, Well-posedness and regularity of backward stochastic Volterra integral equations,, Prob. Theory Rel. Fields, 142 (2008), 21.  doi: 10.1007/s00440-007-0098-6.  Google Scholar

[35]

J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem,, Math. Control Related Fields, 1 (2011), 83.   Google Scholar

[36]

J. Yong, Deterministic time-inconsistent optimal control problems-An essentially cooperative approach,, Acta Math. Appl. Sinica Engl. Ser., 28 (2012), 1.   Google Scholar

[37]

J. Yong and X. Y. Zhou, "Stochastic Controls. Hamiltonian Systems and HJB Equations,'', Applications of Mathematics (New York), 43 (1999).   Google Scholar

show all references

References:
[1]

S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation,, Rev. Finan. Stud., 23 (2010), 2970.  doi: 10.1093/rfs/hhq028.  Google Scholar

[2]

T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochasitic control problem,, working paper., ().   Google Scholar

[3]

T. Björk, A. Murgoci and X. Y. Zhou, Mean varaiance portfolio optimization with state dependent risk aversion,, Math. Finance, ().   Google Scholar

[4]

E. V. Böhm-Bawerk, "The Positive Theory of Capital,'', Books for Libraries Press, (1891).   Google Scholar

[5]

A. Caplin and J. Leahy, The recursive approach to time inconsistency,, J. Econ. Theory, 131 (2006), 134.  doi: 10.1016/j.jet.2005.05.006.  Google Scholar

[6]

I. Ekeland and A. Lazrak, The golden rule when preferences are time inconsistent,, Math. Finan. Econ., 4 (2010), 29.   Google Scholar

[7]

I. Ekeland and T. A. Pirvu, Investment and consumption without commitment,, Math. Finan. Econ., 2 (2008), 57.   Google Scholar

[8]

I. Ekeland, O. Mbodji and T. A. Pirvu, Time-consistent portfolio management,, SIAM J. Financial Math., 3 (2012), 1.   Google Scholar

[9]

W. H. Fleming and H. M. Soner, "Controlled Markov Processes and Viscosity Solutions,'', 2nd edition, 25 (2006).   Google Scholar

[10]

A. Friedman, "Partial Differential Equations of Parabolic Type,'', Prentice Hall, (1964).   Google Scholar

[11]

S. M. Goldman, Consistent plans,, Review of Economic Studies, 47 (1980), 533.   Google Scholar

[12]

S. R. Grenadier and N. Wang, Investment under uncertainty and time-inconsistent preferences,, J. Finan. Econ., 84 (2007), 2.   Google Scholar

[13]

P. J.-J. Herings and K. I. M. Rohde, Time-inconsistent preferences in a general equilibrium model,, Econ. Theory, 29 (2006), 591.  doi: 10.1007/s00199-005-0020-3.  Google Scholar

[14]

Y. Hu, H. Jin and X. Y. Zhou, Time-inconsistent stochastic linear-quadratic control,, , (2011).   Google Scholar

[15]

D. Hume, "A Treatise of Human Nature,'', First edition, (1739).   Google Scholar

[16]

W. S. Jevons, "Theory of Political Economy,'', McMillan, (1871).   Google Scholar

[17]

L. Karp and I. H. Lee, Time-consistent policies,, J. Econ. Theory, 112 (2003), 353.  doi: 10.1016/S0022-0531(03)00067-X.  Google Scholar

[18]

D. Laibson, Golden eggs and hyperbolic discounting,, Quarterly J. Econ., 112 (1997), 443.  doi: 10.1162/003355397555253.  Google Scholar

[19]

J. Ma, P. Protter and J. Yong, Solving forward-backward stochastic differential equations explicitly-a four-step scheme,, Probab. Theory Related Fields, 98 (1994), 339.  doi: 10.1007/BF01192258.  Google Scholar

[20]

J. Ma and J. Yong, "Forward-Backward Stochastic Differential Equations and Their Applications,'', Lecture Notes in Math., 1702 (1999).   Google Scholar

[21]

A. Malthus, An essay on the principle of population,, in, (1986), 2.   Google Scholar

[22]

A. Marshall, "Principles of Economics,'', 1st ed., (1890).   Google Scholar

[23]

J. Marin-Solano and J. Navas, Consumption and portfolio rules for time-inconsistent investors,, European J. Operational Research, 201 (2010), 860.  doi: 10.1016/j.ejor.2009.04.005.  Google Scholar

[24]

J. Marin-Solano and E. V. Shevkoplyas, Non-constant discounting and differential games with random time horizon,, Automatica J. IFAC, 47 (2011), 2626.  doi: 10.1016/j.automatica.2011.09.010.  Google Scholar

[25]

M. Miller and M. Salmon, Dynamic games and the time inconsistency of optimal policy in open economics,, The Economic Journal, 95 (1985), 124.  doi: 10.2307/2232876.  Google Scholar

[26]

I. Palacios-Huerta, Time-inconsistent preferences in Adam Smith and Davis Hume,, History of Political Economy, 35 (2003), 241.  doi: 10.1215/00182702-35-2-241.  Google Scholar

[27]

V. Pareto, "Manuel d'Économie Politique,'', Girard and Brieve, (1909).   Google Scholar

[28]

B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing,, Review of Economic Studies, 40 (1973), 391.   Google Scholar

[29]

R. A. Pollak, Consistent planning,, Review of Economic Studies, 35 (1968), 185.  doi: 10.2307/2296547.  Google Scholar

[30]

A. Smith, "The Theory of Moral Sentiments,'', First edition, (1759).   Google Scholar

[31]

R. H. Strotz, Myopia and inconsistency indynamic utility maximization,, Review of Econ. Studies, 23 (1955), 165.  doi: 10.2307/2295722.  Google Scholar

[32]

L. Tesfatsion, Time inconsistency of benevolent government economics,, J. Public Economics, 31 (1986), 25.  doi: 10.1016/0047-2727(86)90070-8.  Google Scholar

[33]

J. Yong, Backward stochastic Volterra integral equations and some related problems,, Stoch. Proc. Appl., 116 (2006), 779.  doi: 10.1016/j.spa.2006.01.005.  Google Scholar

[34]

J. Yong, Well-posedness and regularity of backward stochastic Volterra integral equations,, Prob. Theory Rel. Fields, 142 (2008), 21.  doi: 10.1007/s00440-007-0098-6.  Google Scholar

[35]

J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem,, Math. Control Related Fields, 1 (2011), 83.   Google Scholar

[36]

J. Yong, Deterministic time-inconsistent optimal control problems-An essentially cooperative approach,, Acta Math. Appl. Sinica Engl. Ser., 28 (2012), 1.   Google Scholar

[37]

J. Yong and X. Y. Zhou, "Stochastic Controls. Hamiltonian Systems and HJB Equations,'', Applications of Mathematics (New York), 43 (1999).   Google Scholar

[1]

Stefano Bianchini, Paolo Bonicatto. Forward untangling and applications to the uniqueness problem for the continuity equation. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020384

[2]

Editorial Office. Retraction: Xiao-Qian Jiang and Lun-Chuan Zhang, A pricing option approach based on backward stochastic differential equation theory. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 969-969. doi: 10.3934/dcdss.2019065

[3]

Yi Zhou, Jianli Liu. The initial-boundary value problem on a strip for the equation of time-like extremal surfaces. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 381-397. doi: 10.3934/dcds.2009.23.381

[4]

Vo Van Au, Hossein Jafari, Zakia Hammouch, Nguyen Huy Tuan. On a final value problem for a nonlinear fractional pseudo-parabolic equation. Electronic Research Archive, 2021, 29 (1) : 1709-1734. doi: 10.3934/era.2020088

[5]

Siyang Cai, Yongmei Cai, Xuerong Mao. A stochastic differential equation SIS epidemic model with regime switching. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020317

[6]

Tetsuya Ishiwata, Young Chol Yang. Numerical and mathematical analysis of blow-up problems for a stochastic differential equation. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 909-918. doi: 10.3934/dcdss.2020391

[7]

Yueyang Zheng, Jingtao Shi. A stackelberg game of backward stochastic differential equations with partial information. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020047

[8]

Christian Clason, Vu Huu Nhu, Arnd Rösch. Optimal control of a non-smooth quasilinear elliptic equation. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020052

[9]

Olivier Ley, Erwin Topp, Miguel Yangari. Some results for the large time behavior of Hamilton-Jacobi equations with Caputo time derivative. Discrete & Continuous Dynamical Systems - A, 2021  doi: 10.3934/dcds.2021007

[10]

Guangjun Shen, Xueying Wu, Xiuwei Yin. Stabilization of stochastic differential equations driven by G-Lévy process with discrete-time feedback control. Discrete & Continuous Dynamical Systems - B, 2021, 26 (2) : 755-774. doi: 10.3934/dcdsb.2020133

[11]

Manuel Friedrich, Martin Kružík, Ulisse Stefanelli. Equilibrium of immersed hyperelastic solids. Discrete & Continuous Dynamical Systems - S, 2021  doi: 10.3934/dcdss.2021003

[12]

Kerioui Nadjah, Abdelouahab Mohammed Salah. Stability and Hopf bifurcation of the coexistence equilibrium for a differential-algebraic biological economic system with predator harvesting. Electronic Research Archive, 2021, 29 (1) : 1641-1660. doi: 10.3934/era.2020084

[13]

Qingfeng Zhu, Yufeng Shi. Nonzero-sum differential game of backward doubly stochastic systems with delay and applications. Mathematical Control & Related Fields, 2021, 11 (1) : 73-94. doi: 10.3934/mcrf.2020028

[14]

Hai Huang, Xianlong Fu. Optimal control problems for a neutral integro-differential system with infinite delay. Evolution Equations & Control Theory, 2020  doi: 10.3934/eect.2020107

[15]

Yukihiko Nakata. Existence of a period two solution of a delay differential equation. Discrete & Continuous Dynamical Systems - S, 2021, 14 (3) : 1103-1110. doi: 10.3934/dcdss.2020392

[16]

Lars Grüne, Matthias A. Müller, Christopher M. Kellett, Steven R. Weller. Strict dissipativity for discrete time discounted optimal control problems. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020046

[17]

Sergey Rashkovskiy. Hamilton-Jacobi theory for Hamiltonian and non-Hamiltonian systems. Journal of Geometric Mechanics, 2020, 12 (4) : 563-583. doi: 10.3934/jgm.2020024

[18]

Yuan Tan, Qingyuan Cao, Lan Li, Tianshi Hu, Min Su. A chance-constrained stochastic model predictive control problem with disturbance feedback. Journal of Industrial & Management Optimization, 2021, 17 (1) : 67-79. doi: 10.3934/jimo.2019099

[19]

Elimhan N. Mahmudov. Infimal convolution and duality in convex optimal control problems with second order evolution differential inclusions. Evolution Equations & Control Theory, 2021, 10 (1) : 37-59. doi: 10.3934/eect.2020051

[20]

Evelyn Sander, Thomas Wanner. Equilibrium validation in models for pattern formation based on Sobolev embeddings. Discrete & Continuous Dynamical Systems - B, 2021, 26 (1) : 603-632. doi: 10.3934/dcdsb.2020260

2019 Impact Factor: 0.857

Metrics

  • PDF downloads (71)
  • HTML views (0)
  • Cited by (49)

Other articles
by authors

[Back to Top]