American Institute of Mathematical Sciences

Advanced
September  2012, 2(3): 271-329. doi: 10.3934/mcrf.2012.2.271

Time-inconsistent optimal control problems and the equilibrium HJB equation

Jiongmin Yong 1,

1. 

Department of Mathematics, University of Central Florida, Orlando, FL 32816

Received  April 2012 Revised  May 2012 Published  August 2012

A general time-inconsistent optimal control problem is considered for stochastic differential equations with deterministic coefficients. Under suitable conditions, a Hamilton-Jacobi-Bellman type equation is derived for the equilibrium value function of the problem. Well-posedness such an equation is studied, and time-consistent equilibrium strategies are constructed. As special cases, the linear-quadratic problem and a generalized Merton's portfolio problem are investigated.
Keywords: Time-inconsistent optimal control problem, forward-backward stochastic differential equation., equilibrium value function, equilibrium Hamilton-Jacobi-Bellman equation.
Mathematics Subject Classification: Primary: 93E20, 49L20, 49N10, 49N70; Secondary: 35Q93,91A23, 91A6.
Citation: Jiongmin Yong. Time-inconsistent optimal control problems and the equilibrium HJB equation. Mathematical Control & Related Fields, 2012, 2 (3) : 271-329. doi: 10.3934/mcrf.2012.2.271
References:
[1]

S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation, Rev. Finan. Stud., 23 (2010), 2970-3016. doi: 10.1093/rfs/hhq028.  Google Scholar

[2]

T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochasitic control problem,, working paper., ().   Google Scholar

[3]

T. Björk, A. Murgoci and X. Y. Zhou, Mean varaiance portfolio optimization with state dependent risk aversion,, Math. Finance, ().   Google Scholar

[4]

E. V. Böhm-Bawerk, "The Positive Theory of Capital,'' Books for Libraries Press, Freeport, New York, 1891. Google Scholar

[5]

A. Caplin and J. Leahy, The recursive approach to time inconsistency, J. Econ. Theory, 131 (2006), 134-156. doi: 10.1016/j.jet.2005.05.006.  Google Scholar

[6]

I. Ekeland and A. Lazrak, The golden rule when preferences are time inconsistent, Math. Finan. Econ., 4 (2010), 29-55.  Google Scholar

[7]

I. Ekeland and T. A. Pirvu, Investment and consumption without commitment, Math. Finan. Econ., 2 (2008), 57-86.  Google Scholar

[8]

I. Ekeland, O. Mbodji and T. A. Pirvu, Time-consistent portfolio management, SIAM J. Financial Math., 3 (2012), 1-32. Google Scholar

[9]

W. H. Fleming and H. M. Soner, "Controlled Markov Processes and Viscosity Solutions,'' 2nd edition, Stochastic Modelling and Applied Probability, 25, Springer, New York, 2006.  Google Scholar

[10]

A. Friedman, "Partial Differential Equations of Parabolic Type,'' Prentice Hall, Inc., Englewood Cliffs, NJ, 1964.  Google Scholar

[11]

S. M. Goldman, Consistent plans, Review of Economic Studies, 47 (1980), 533-537. Google Scholar

[12]

S. R. Grenadier and N. Wang, Investment under uncertainty and time-inconsistent preferences, J. Finan. Econ., 84 (2007), 2-39. Google Scholar

[13]

P. J.-J. Herings and K. I. M. Rohde, Time-inconsistent preferences in a general equilibrium model, Econ. Theory, 29 (2006), 591-619. doi: 10.1007/s00199-005-0020-3.  Google Scholar

[14]

Y. Hu, H. Jin and X. Y. Zhou, Time-inconsistent stochastic linear-quadratic control, arXiv:1111.0818, 2011. Google Scholar

[15]

D. Hume, "A Treatise of Human Nature,'' First edition, 1739; Reprint, Oxford Univ. Press, New York, 1978. Google Scholar

[16]

W. S. Jevons, "Theory of Political Economy,'' McMillan, London, 1871. Google Scholar

[17]

L. Karp and I. H. Lee, Time-consistent policies, J. Econ. Theory, 112 (2003), 353-364. doi: 10.1016/S0022-0531(03)00067-X.  Google Scholar

[18]

D. Laibson, Golden eggs and hyperbolic discounting, Quarterly J. Econ., 112 (1997), 443-477. doi: 10.1162/003355397555253.  Google Scholar

[19]

J. Ma, P. Protter and J. Yong, Solving forward-backward stochastic differential equations explicitly-a four-step scheme, Probab. Theory Related Fields, 98 (1994), 339-359. doi: 10.1007/BF01192258.  Google Scholar

[20]

J. Ma and J. Yong, "Forward-Backward Stochastic Differential Equations and Their Applications,'' Lecture Notes in Math., 1702, Springer-Verlag, Berlin, 1999.  Google Scholar

[21]

A. Malthus, An essay on the principle of population, in "The Works of Thomas Robert Malthus," Vols. 2-3 (eds. E. A. Wrigley and D. Souden), W. Pickering, London, 1986. Google Scholar

[22]

A. Marshall, "Principles of Economics,'' 1st ed., 1890; 8th ed., Macmillan, London, 1920. Google Scholar

[23]

J. Marin-Solano and J. Navas, Consumption and portfolio rules for time-inconsistent investors, European J. Operational Research, 201 (2010), 860-872. doi: 10.1016/j.ejor.2009.04.005.  Google Scholar

[24]

J. Marin-Solano and E. V. Shevkoplyas, Non-constant discounting and differential games with random time horizon, Automatica J. IFAC, 47 (2011), 2626-2638. doi: 10.1016/j.automatica.2011.09.010.  Google Scholar

[25]

M. Miller and M. Salmon, Dynamic games and the time inconsistency of optimal policy in open economics, The Economic Journal, 95 (1985), 124-137. doi: 10.2307/2232876.  Google Scholar

[26]

I. Palacios-Huerta, Time-inconsistent preferences in Adam Smith and Davis Hume, History of Political Economy, 35 (2003), 241-268. doi: 10.1215/00182702-35-2-241.  Google Scholar

[27]

V. Pareto, "Manuel d'Économie Politique,'' Girard and Brieve, Paris, 1909. Google Scholar

[28]

B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing, Review of Economic Studies, 40 (1973), 391-401. Google Scholar

[29]

R. A. Pollak, Consistent planning, Review of Economic Studies, 35 (1968), 185-199. doi: 10.2307/2296547.  Google Scholar

[30]

A. Smith, "The Theory of Moral Sentiments,'' First edition, 1759; Reprint, Oxford Univ. Press, 1976. Google Scholar

[31]

R. H. Strotz, Myopia and inconsistency indynamic utility maximization, Review of Econ. Studies, 23 (1955), 165-180. doi: 10.2307/2295722.  Google Scholar

[32]

L. Tesfatsion, Time inconsistency of benevolent government economics, J. Public Economics, 31 (1986), 25-52. doi: 10.1016/0047-2727(86)90070-8.  Google Scholar

[33]

J. Yong, Backward stochastic Volterra integral equations and some related problems, Stoch. Proc. Appl., 116 (2006), 779-795. doi: 10.1016/j.spa.2006.01.005.  Google Scholar

[34]

J. Yong, Well-posedness and regularity of backward stochastic Volterra integral equations, Prob. Theory Rel. Fields, 142 (2008), 21-77. doi: 10.1007/s00440-007-0098-6.  Google Scholar

[35]

J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem, Math. Control Related Fields, 1 (2011), 83-118.  Google Scholar

[36]

J. Yong, Deterministic time-inconsistent optimal control problems-An essentially cooperative approach, Acta Math. Appl. Sinica Engl. Ser., 28 (2012), 1-30.  Google Scholar

[37]

J. Yong and X. Y. Zhou, "Stochastic Controls. Hamiltonian Systems and HJB Equations,'' Applications of Mathematics (New York), 43, Springer-Verlag, New York, 1999.  Google Scholar

show all references

References:
[1]

S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation, Rev. Finan. Stud., 23 (2010), 2970-3016. doi: 10.1093/rfs/hhq028.  Google Scholar

[2]

T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochasitic control problem,, working paper., ().   Google Scholar

[3]

T. Björk, A. Murgoci and X. Y. Zhou, Mean varaiance portfolio optimization with state dependent risk aversion,, Math. Finance, ().   Google Scholar

[4]

E. V. Böhm-Bawerk, "The Positive Theory of Capital,'' Books for Libraries Press, Freeport, New York, 1891. Google Scholar

[5]

A. Caplin and J. Leahy, The recursive approach to time inconsistency, J. Econ. Theory, 131 (2006), 134-156. doi: 10.1016/j.jet.2005.05.006.  Google Scholar

[6]

I. Ekeland and A. Lazrak, The golden rule when preferences are time inconsistent, Math. Finan. Econ., 4 (2010), 29-55.  Google Scholar

[7]

I. Ekeland and T. A. Pirvu, Investment and consumption without commitment, Math. Finan. Econ., 2 (2008), 57-86.  Google Scholar

[8]

I. Ekeland, O. Mbodji and T. A. Pirvu, Time-consistent portfolio management, SIAM J. Financial Math., 3 (2012), 1-32. Google Scholar

[9]

W. H. Fleming and H. M. Soner, "Controlled Markov Processes and Viscosity Solutions,'' 2nd edition, Stochastic Modelling and Applied Probability, 25, Springer, New York, 2006.  Google Scholar

[10]

A. Friedman, "Partial Differential Equations of Parabolic Type,'' Prentice Hall, Inc., Englewood Cliffs, NJ, 1964.  Google Scholar

[11]

S. M. Goldman, Consistent plans, Review of Economic Studies, 47 (1980), 533-537. Google Scholar

[12]

S. R. Grenadier and N. Wang, Investment under uncertainty and time-inconsistent preferences, J. Finan. Econ., 84 (2007), 2-39. Google Scholar

[13]

P. J.-J. Herings and K. I. M. Rohde, Time-inconsistent preferences in a general equilibrium model, Econ. Theory, 29 (2006), 591-619. doi: 10.1007/s00199-005-0020-3.  Google Scholar

[14]

Y. Hu, H. Jin and X. Y. Zhou, Time-inconsistent stochastic linear-quadratic control, arXiv:1111.0818, 2011. Google Scholar

[15]

D. Hume, "A Treatise of Human Nature,'' First edition, 1739; Reprint, Oxford Univ. Press, New York, 1978. Google Scholar

[16]

W. S. Jevons, "Theory of Political Economy,'' McMillan, London, 1871. Google Scholar

[17]

L. Karp and I. H. Lee, Time-consistent policies, J. Econ. Theory, 112 (2003), 353-364. doi: 10.1016/S0022-0531(03)00067-X.  Google Scholar

[18]

D. Laibson, Golden eggs and hyperbolic discounting, Quarterly J. Econ., 112 (1997), 443-477. doi: 10.1162/003355397555253.  Google Scholar

[19]

J. Ma, P. Protter and J. Yong, Solving forward-backward stochastic differential equations explicitly-a four-step scheme, Probab. Theory Related Fields, 98 (1994), 339-359. doi: 10.1007/BF01192258.  Google Scholar

[20]

J. Ma and J. Yong, "Forward-Backward Stochastic Differential Equations and Their Applications,'' Lecture Notes in Math., 1702, Springer-Verlag, Berlin, 1999.  Google Scholar

[21]

A. Malthus, An essay on the principle of population, in "The Works of Thomas Robert Malthus," Vols. 2-3 (eds. E. A. Wrigley and D. Souden), W. Pickering, London, 1986. Google Scholar

[22]

A. Marshall, "Principles of Economics,'' 1st ed., 1890; 8th ed., Macmillan, London, 1920. Google Scholar

[23]

J. Marin-Solano and J. Navas, Consumption and portfolio rules for time-inconsistent investors, European J. Operational Research, 201 (2010), 860-872. doi: 10.1016/j.ejor.2009.04.005.  Google Scholar

[24]

J. Marin-Solano and E. V. Shevkoplyas, Non-constant discounting and differential games with random time horizon, Automatica J. IFAC, 47 (2011), 2626-2638. doi: 10.1016/j.automatica.2011.09.010.  Google Scholar

[25]

M. Miller and M. Salmon, Dynamic games and the time inconsistency of optimal policy in open economics, The Economic Journal, 95 (1985), 124-137. doi: 10.2307/2232876.  Google Scholar

[26]

I. Palacios-Huerta, Time-inconsistent preferences in Adam Smith and Davis Hume, History of Political Economy, 35 (2003), 241-268. doi: 10.1215/00182702-35-2-241.  Google Scholar

[27]

V. Pareto, "Manuel d'Économie Politique,'' Girard and Brieve, Paris, 1909. Google Scholar

[28]

B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing, Review of Economic Studies, 40 (1973), 391-401. Google Scholar

[29]

R. A. Pollak, Consistent planning, Review of Economic Studies, 35 (1968), 185-199. doi: 10.2307/2296547.  Google Scholar

[30]

A. Smith, "The Theory of Moral Sentiments,'' First edition, 1759; Reprint, Oxford Univ. Press, 1976. Google Scholar

[31]

R. H. Strotz, Myopia and inconsistency indynamic utility maximization, Review of Econ. Studies, 23 (1955), 165-180. doi: 10.2307/2295722.  Google Scholar

[32]

L. Tesfatsion, Time inconsistency of benevolent government economics, J. Public Economics, 31 (1986), 25-52. doi: 10.1016/0047-2727(86)90070-8.  Google Scholar

[33]

J. Yong, Backward stochastic Volterra integral equations and some related problems, Stoch. Proc. Appl., 116 (2006), 779-795. doi: 10.1016/j.spa.2006.01.005.  Google Scholar

[34]

J. Yong, Well-posedness and regularity of backward stochastic Volterra integral equations, Prob. Theory Rel. Fields, 142 (2008), 21-77. doi: 10.1007/s00440-007-0098-6.  Google Scholar

[35]

J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem, Math. Control Related Fields, 1 (2011), 83-118.  Google Scholar

[36]

J. Yong, Deterministic time-inconsistent optimal control problems-An essentially cooperative approach, Acta Math. Appl. Sinica Engl. Ser., 28 (2012), 1-30.  Google Scholar

[37]

J. Yong and X. Y. Zhou, "Stochastic Controls. Hamiltonian Systems and HJB Equations,'' Applications of Mathematics (New York), 43, Springer-Verlag, New York, 1999.  Google Scholar

[1]

Haiyang Wang, Zhen Wu. Time-inconsistent optimal control problem with random coefficients and stochastic equilibrium HJB equation. Mathematical Control & Related Fields, 2015, 5 (3) : 651-678. doi: 10.3934/mcrf.2015.5.651
[2]

Steven Richardson, Song Wang. The viscosity approximation to the Hamilton-Jacobi-Bellman equation in optimal feedback control: Upper bounds for extended domains. Journal of Industrial & Management Optimization, 2010, 6 (1) : 161-175. doi: 10.3934/jimo.2010.6.161
[3]

Zhen-Zhen Tao, Bing Sun. A feedback design for numerical solution to optimal control problems based on Hamilton-Jacobi-Bellman equation. Electronic Research Archive, , () : -. doi: 10.3934/era.2021046
[4]

Jean-Claude Zambrini. On the geometry of the Hamilton-Jacobi-Bellman equation. Journal of Geometric Mechanics, 2009, 1 (3) : 369-387. doi: 10.3934/jgm.2009.1.369
[5]

Bian-Xia Yang, Shanshan Gu, Guowei Dai. Existence and multiplicity for Hamilton-Jacobi-Bellman equation. Communications on Pure & Applied Analysis, , () : -. doi: 10.3934/cpaa.2021130
[6]

Ishak Alia. Time-inconsistent stochastic optimal control problems: a backward stochastic partial differential equations approach. Mathematical Control & Related Fields, 2020, 10 (4) : 785-826. doi: 10.3934/mcrf.2020020
[7]

Daniele Castorina, Annalisa Cesaroni, Luca Rossi. On a parabolic Hamilton-Jacobi-Bellman equation degenerating at the boundary. Communications on Pure & Applied Analysis, 2016, 15 (4) : 1251-1263. doi: 10.3934/cpaa.2016.15.1251
[8]

Ishak Alia. A non-exponential discounting time-inconsistent stochastic optimal control problem for jump-diffusion. Mathematical Control & Related Fields, 2019, 9 (3) : 541-570. doi: 10.3934/mcrf.2019025
[9]

Jiongmin Yong. A deterministic linear quadratic time-inconsistent optimal control problem. Mathematical Control & Related Fields, 2011, 1 (1) : 83-118. doi: 10.3934/mcrf.2011.1.83
[10]

Yufeng Shi, Tianxiao Wang, Jiongmin Yong. Optimal control problems of forward-backward stochastic Volterra integral equations. Mathematical Control & Related Fields, 2015, 5 (3) : 613-649. doi: 10.3934/mcrf.2015.5.613
[11]

Yushi Hamaguchi. Extended backward stochastic Volterra integral equations and their applications to time-Inconsistent stochastic recursive control problems. Mathematical Control & Related Fields, 2021, 11 (2) : 433-478. doi: 10.3934/mcrf.2020043
[12]

Xin Chen, Ana Bela Cruzeiro. Stochastic geodesics and forward-backward stochastic differential equations on Lie groups. Conference Publications, 2013, 2013 (special) : 115-121. doi: 10.3934/proc.2013.2013.115
[13]

Qingmeng Wei, Zhiyong Yu. Time-inconsistent recursive zero-sum stochastic differential games. Mathematical Control & Related Fields, 2018, 8 (3&4) : 1051-1079. doi: 10.3934/mcrf.2018045
[14]

Jiaqin Wei. Time-inconsistent optimal control problems with regime-switching. Mathematical Control & Related Fields, 2017, 7 (4) : 585-622. doi: 10.3934/mcrf.2017022
[15]

Joan-Andreu Lázaro-Camí, Juan-Pablo Ortega. The stochastic Hamilton-Jacobi equation. Journal of Geometric Mechanics, 2009, 1 (3) : 295-315. doi: 10.3934/jgm.2009.1.295
[16]

Sebastián Ferrer, Martin Lara. Families of canonical transformations by Hamilton-Jacobi-Poincaré equation. Application to rotational and orbital motion. Journal of Geometric Mechanics, 2010, 2 (3) : 223-241. doi: 10.3934/jgm.2010.2.223
[17]

Manuel de León, Juan Carlos Marrero, David Martín de Diego. Linear almost Poisson structures and Hamilton-Jacobi equation. Applications to nonholonomic mechanics. Journal of Geometric Mechanics, 2010, 2 (2) : 159-198. doi: 10.3934/jgm.2010.2.159
[18]

Mohamed Assellaou, Olivier Bokanowski, Hasnaa Zidani. Error estimates for second order Hamilton-Jacobi-Bellman equations. Approximation of probabilistic reachable sets. Discrete & Continuous Dynamical Systems, 2015, 35 (9) : 3933-3964. doi: 10.3934/dcds.2015.35.3933
[19]

Thierry Horsin, Peter I. Kogut, Olivier Wilk. Optimal $L^2$-control problem in coefficients for a linear elliptic equation. II. Approximation of solutions and optimality conditions. Mathematical Control & Related Fields, 2016, 6 (4) : 595-628. doi: 10.3934/mcrf.2016017
[20]

Thierry Horsin, Peter I. Kogut. Optimal $L^2$-control problem in coefficients for a linear elliptic equation. I. Existence result. Mathematical Control & Related Fields, 2015, 5 (1) : 73-96. doi: 10.3934/mcrf.2015.5.73

2020 Impact Factor: 1.284

Tools

Metrics

  • PDF downloads (104)
  • HTML views (0)
  • Cited by (61)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences