Time-inconsistent optimal control problems and the equilibrium HJB equation

Previous Article
MCRF Home
This Issue
Next Article
Controllability of the cubic Schroedinger equation via a low-dimensional source term

September 2012, 2(3): 271-329. doi: 10.3934/mcrf.2012.2.271

Time-inconsistent optimal control problems and the equilibrium HJB equation

Jiongmin Yong ^1,

1.	Department of Mathematics, University of Central Florida, Orlando, FL 32816

Received April 2012 Revised May 2012 Published August 2012

Abstract
Related Papers

A general time-inconsistent optimal control problem is considered for stochastic differential equations with deterministic coefficients. Under suitable conditions, a Hamilton-Jacobi-Bellman type equation is derived for the equilibrium value function of the problem. Well-posedness such an equation is studied, and time-consistent equilibrium strategies are constructed. As special cases, the linear-quadratic problem and a generalized Merton's portfolio problem are investigated.

Keywords: Time-inconsistent optimal control problem, forward-backward stochastic differential equation., equilibrium value function, equilibrium Hamilton-Jacobi-Bellman equation.

Mathematics Subject Classification: Primary: 93E20, 49L20, 49N10, 49N70; Secondary: 35Q93,91A23, 91A6.

Citation: Jiongmin Yong. Time-inconsistent optimal control problems and the equilibrium HJB equation. Mathematical Control and Related Fields, 2012, 2 (3) : 271-329. doi: 10.3934/mcrf.2012.2.271

References:

[1]	S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation, Rev. Finan. Stud., 23 (2010), 2970-3016. doi: 10.1093/rfs/hhq028.
[2]	T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochasitic control problem,, working paper., ().
[3]	T. Björk, A. Murgoci and X. Y. Zhou, Mean varaiance portfolio optimization with state dependent risk aversion,, Math. Finance, ().
[4]	E. V. Böhm-Bawerk, "The Positive Theory of Capital,'' Books for Libraries Press, Freeport, New York, 1891.
[5]	A. Caplin and J. Leahy, The recursive approach to time inconsistency, J. Econ. Theory, 131 (2006), 134-156. doi: 10.1016/j.jet.2005.05.006.
[6]	I. Ekeland and A. Lazrak, The golden rule when preferences are time inconsistent, Math. Finan. Econ., 4 (2010), 29-55.
[7]	I. Ekeland and T. A. Pirvu, Investment and consumption without commitment, Math. Finan. Econ., 2 (2008), 57-86.
[8]	I. Ekeland, O. Mbodji and T. A. Pirvu, Time-consistent portfolio management, SIAM J. Financial Math., 3 (2012), 1-32.
[9]	W. H. Fleming and H. M. Soner, "Controlled Markov Processes and Viscosity Solutions,'' 2^nd edition, Stochastic Modelling and Applied Probability, 25, Springer, New York, 2006.
[10]	A. Friedman, "Partial Differential Equations of Parabolic Type,'' Prentice Hall, Inc., Englewood Cliffs, NJ, 1964.
[11]	S. M. Goldman, Consistent plans, Review of Economic Studies, 47 (1980), 533-537.
[12]	S. R. Grenadier and N. Wang, Investment under uncertainty and time-inconsistent preferences, J. Finan. Econ., 84 (2007), 2-39.
[13]	P. J.-J. Herings and K. I. M. Rohde, Time-inconsistent preferences in a general equilibrium model, Econ. Theory, 29 (2006), 591-619. doi: 10.1007/s00199-005-0020-3.
[14]	Y. Hu, H. Jin and X. Y. Zhou, Time-inconsistent stochastic linear-quadratic control, arXiv:1111.0818, 2011.
[15]	D. Hume, "A Treatise of Human Nature,'' First edition, 1739; Reprint, Oxford Univ. Press, New York, 1978.
[16]	W. S. Jevons, "Theory of Political Economy,'' McMillan, London, 1871.
[17]	L. Karp and I. H. Lee, Time-consistent policies, J. Econ. Theory, 112 (2003), 353-364. doi: 10.1016/S0022-0531(03)00067-X.
[18]	D. Laibson, Golden eggs and hyperbolic discounting, Quarterly J. Econ., 112 (1997), 443-477. doi: 10.1162/003355397555253.
[19]	J. Ma, P. Protter and J. Yong, Solving forward-backward stochastic differential equations explicitly-a four-step scheme, Probab. Theory Related Fields, 98 (1994), 339-359. doi: 10.1007/BF01192258.
[20]	J. Ma and J. Yong, "Forward-Backward Stochastic Differential Equations and Their Applications,'' Lecture Notes in Math., 1702, Springer-Verlag, Berlin, 1999.
[21]	A. Malthus, An essay on the principle of population, in "The Works of Thomas Robert Malthus," Vols. 2-3 (eds. E. A. Wrigley and D. Souden), W. Pickering, London, 1986.
[22]	A. Marshall, "Principles of Economics,'' 1^st ed., 1890; 8^th ed., Macmillan, London, 1920.
[23]	J. Marin-Solano and J. Navas, Consumption and portfolio rules for time-inconsistent investors, European J. Operational Research, 201 (2010), 860-872. doi: 10.1016/j.ejor.2009.04.005.
[24]	J. Marin-Solano and E. V. Shevkoplyas, Non-constant discounting and differential games with random time horizon, Automatica J. IFAC, 47 (2011), 2626-2638. doi: 10.1016/j.automatica.2011.09.010.
[25]	M. Miller and M. Salmon, Dynamic games and the time inconsistency of optimal policy in open economics, The Economic Journal, 95 (1985), 124-137. doi: 10.2307/2232876.
[26]	I. Palacios-Huerta, Time-inconsistent preferences in Adam Smith and Davis Hume, History of Political Economy, 35 (2003), 241-268. doi: 10.1215/00182702-35-2-241.
[27]	V. Pareto, "Manuel d'Économie Politique,'' Girard and Brieve, Paris, 1909.
[28]	B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing, Review of Economic Studies, 40 (1973), 391-401.
[29]	R. A. Pollak, Consistent planning, Review of Economic Studies, 35 (1968), 185-199. doi: 10.2307/2296547.
[30]	A. Smith, "The Theory of Moral Sentiments,'' First edition, 1759; Reprint, Oxford Univ. Press, 1976.
[31]	R. H. Strotz, Myopia and inconsistency indynamic utility maximization, Review of Econ. Studies, 23 (1955), 165-180. doi: 10.2307/2295722.
[32]	L. Tesfatsion, Time inconsistency of benevolent government economics, J. Public Economics, 31 (1986), 25-52. doi: 10.1016/0047-2727(86)90070-8.
[33]	J. Yong, Backward stochastic Volterra integral equations and some related problems, Stoch. Proc. Appl., 116 (2006), 779-795. doi: 10.1016/j.spa.2006.01.005.
[34]	J. Yong, Well-posedness and regularity of backward stochastic Volterra integral equations, Prob. Theory Rel. Fields, 142 (2008), 21-77. doi: 10.1007/s00440-007-0098-6.
[35]	J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem, Math. Control Related Fields, 1 (2011), 83-118.
[36]	J. Yong, Deterministic time-inconsistent optimal control problems-An essentially cooperative approach, Acta Math. Appl. Sinica Engl. Ser., 28 (2012), 1-30.
[37]	J. Yong and X. Y. Zhou, "Stochastic Controls. Hamiltonian Systems and HJB Equations,'' Applications of Mathematics (New York), 43, Springer-Verlag, New York, 1999.

show all references

References:

[1]	S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation, Rev. Finan. Stud., 23 (2010), 2970-3016. doi: 10.1093/rfs/hhq028.
[2]	T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochasitic control problem,, working paper., ().
[3]	T. Björk, A. Murgoci and X. Y. Zhou, Mean varaiance portfolio optimization with state dependent risk aversion,, Math. Finance, ().
[4]	E. V. Böhm-Bawerk, "The Positive Theory of Capital,'' Books for Libraries Press, Freeport, New York, 1891.
[5]	A. Caplin and J. Leahy, The recursive approach to time inconsistency, J. Econ. Theory, 131 (2006), 134-156. doi: 10.1016/j.jet.2005.05.006.
[6]	I. Ekeland and A. Lazrak, The golden rule when preferences are time inconsistent, Math. Finan. Econ., 4 (2010), 29-55.
[7]	I. Ekeland and T. A. Pirvu, Investment and consumption without commitment, Math. Finan. Econ., 2 (2008), 57-86.
[8]	I. Ekeland, O. Mbodji and T. A. Pirvu, Time-consistent portfolio management, SIAM J. Financial Math., 3 (2012), 1-32.
[9]	W. H. Fleming and H. M. Soner, "Controlled Markov Processes and Viscosity Solutions,'' 2^nd edition, Stochastic Modelling and Applied Probability, 25, Springer, New York, 2006.
[10]	A. Friedman, "Partial Differential Equations of Parabolic Type,'' Prentice Hall, Inc., Englewood Cliffs, NJ, 1964.
[11]	S. M. Goldman, Consistent plans, Review of Economic Studies, 47 (1980), 533-537.
[12]	S. R. Grenadier and N. Wang, Investment under uncertainty and time-inconsistent preferences, J. Finan. Econ., 84 (2007), 2-39.
[13]	P. J.-J. Herings and K. I. M. Rohde, Time-inconsistent preferences in a general equilibrium model, Econ. Theory, 29 (2006), 591-619. doi: 10.1007/s00199-005-0020-3.
[14]	Y. Hu, H. Jin and X. Y. Zhou, Time-inconsistent stochastic linear-quadratic control, arXiv:1111.0818, 2011.
[15]	D. Hume, "A Treatise of Human Nature,'' First edition, 1739; Reprint, Oxford Univ. Press, New York, 1978.
[16]	W. S. Jevons, "Theory of Political Economy,'' McMillan, London, 1871.
[17]	L. Karp and I. H. Lee, Time-consistent policies, J. Econ. Theory, 112 (2003), 353-364. doi: 10.1016/S0022-0531(03)00067-X.
[18]	D. Laibson, Golden eggs and hyperbolic discounting, Quarterly J. Econ., 112 (1997), 443-477. doi: 10.1162/003355397555253.
[19]	J. Ma, P. Protter and J. Yong, Solving forward-backward stochastic differential equations explicitly-a four-step scheme, Probab. Theory Related Fields, 98 (1994), 339-359. doi: 10.1007/BF01192258.
[20]	J. Ma and J. Yong, "Forward-Backward Stochastic Differential Equations and Their Applications,'' Lecture Notes in Math., 1702, Springer-Verlag, Berlin, 1999.
[21]	A. Malthus, An essay on the principle of population, in "The Works of Thomas Robert Malthus," Vols. 2-3 (eds. E. A. Wrigley and D. Souden), W. Pickering, London, 1986.
[22]	A. Marshall, "Principles of Economics,'' 1^st ed., 1890; 8^th ed., Macmillan, London, 1920.
[23]	J. Marin-Solano and J. Navas, Consumption and portfolio rules for time-inconsistent investors, European J. Operational Research, 201 (2010), 860-872. doi: 10.1016/j.ejor.2009.04.005.
[24]	J. Marin-Solano and E. V. Shevkoplyas, Non-constant discounting and differential games with random time horizon, Automatica J. IFAC, 47 (2011), 2626-2638. doi: 10.1016/j.automatica.2011.09.010.
[25]	M. Miller and M. Salmon, Dynamic games and the time inconsistency of optimal policy in open economics, The Economic Journal, 95 (1985), 124-137. doi: 10.2307/2232876.
[26]	I. Palacios-Huerta, Time-inconsistent preferences in Adam Smith and Davis Hume, History of Political Economy, 35 (2003), 241-268. doi: 10.1215/00182702-35-2-241.
[27]	V. Pareto, "Manuel d'Économie Politique,'' Girard and Brieve, Paris, 1909.
[28]	B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing, Review of Economic Studies, 40 (1973), 391-401.
[29]	R. A. Pollak, Consistent planning, Review of Economic Studies, 35 (1968), 185-199. doi: 10.2307/2296547.
[30]	A. Smith, "The Theory of Moral Sentiments,'' First edition, 1759; Reprint, Oxford Univ. Press, 1976.
[31]	R. H. Strotz, Myopia and inconsistency indynamic utility maximization, Review of Econ. Studies, 23 (1955), 165-180. doi: 10.2307/2295722.
[32]	L. Tesfatsion, Time inconsistency of benevolent government economics, J. Public Economics, 31 (1986), 25-52. doi: 10.1016/0047-2727(86)90070-8.
[33]	J. Yong, Backward stochastic Volterra integral equations and some related problems, Stoch. Proc. Appl., 116 (2006), 779-795. doi: 10.1016/j.spa.2006.01.005.
[34]	J. Yong, Well-posedness and regularity of backward stochastic Volterra integral equations, Prob. Theory Rel. Fields, 142 (2008), 21-77. doi: 10.1007/s00440-007-0098-6.
[35]	J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem, Math. Control Related Fields, 1 (2011), 83-118.
[36]	J. Yong, Deterministic time-inconsistent optimal control problems-An essentially cooperative approach, Acta Math. Appl. Sinica Engl. Ser., 28 (2012), 1-30.
[37]	J. Yong and X. Y. Zhou, "Stochastic Controls. Hamiltonian Systems and HJB Equations,'' Applications of Mathematics (New York), 43, Springer-Verlag, New York, 1999.

[1]	Haiyang Wang, Zhen Wu. Time-inconsistent optimal control problem with random coefficients and stochastic equilibrium HJB equation. Mathematical Control and Related Fields, 2015, 5 (3) : 651-678. doi: 10.3934/mcrf.2015.5.651
[2]	Steven Richardson, Song Wang. The viscosity approximation to the Hamilton-Jacobi-Bellman equation in optimal feedback control: Upper bounds for extended domains. Journal of Industrial and Management Optimization, 2010, 6 (1) : 161-175. doi: 10.3934/jimo.2010.6.161
[3]	Zhen-Zhen Tao, Bing Sun. A feedback design for numerical solution to optimal control problems based on Hamilton-Jacobi-Bellman equation. Electronic Research Archive, 2021, 29 (5) : 3429-3447. doi: 10.3934/era.2021046
[4]	Jean-Claude Zambrini. On the geometry of the Hamilton-Jacobi-Bellman equation. Journal of Geometric Mechanics, 2009, 1 (3) : 369-387. doi: 10.3934/jgm.2009.1.369
[5]	Bian-Xia Yang, Shanshan Gu, Guowei Dai. Existence and multiplicity for Hamilton-Jacobi-Bellman equation. Communications on Pure and Applied Analysis, 2021, 20 (11) : 3767-3793. doi: 10.3934/cpaa.2021130
[6]	Ishak Alia. Time-inconsistent stochastic optimal control problems: a backward stochastic partial differential equations approach. Mathematical Control and Related Fields, 2020, 10 (4) : 785-826. doi: 10.3934/mcrf.2020020
[7]	Xuhui Wang, Lei Hu. A new method to solve the Hamilton-Jacobi-Bellman equation for a stochastic portfolio optimization model with boundary memory. Journal of Industrial and Management Optimization, 2021 doi: 10.3934/jimo.2021137
[8]	Daniele Castorina, Annalisa Cesaroni, Luca Rossi. On a parabolic Hamilton-Jacobi-Bellman equation degenerating at the boundary. Communications on Pure and Applied Analysis, 2016, 15 (4) : 1251-1263. doi: 10.3934/cpaa.2016.15.1251
[9]	Ishak Alia. A non-exponential discounting time-inconsistent stochastic optimal control problem for jump-diffusion. Mathematical Control and Related Fields, 2019, 9 (3) : 541-570. doi: 10.3934/mcrf.2019025
[10]	Yufeng Shi, Tianxiao Wang, Jiongmin Yong. Optimal control problems of forward-backward stochastic Volterra integral equations. Mathematical Control and Related Fields, 2015, 5 (3) : 613-649. doi: 10.3934/mcrf.2015.5.613
[11]	Jiongmin Yong. A deterministic linear quadratic time-inconsistent optimal control problem. Mathematical Control and Related Fields, 2011, 1 (1) : 83-118. doi: 10.3934/mcrf.2011.1.83
[12]	Yushi Hamaguchi. Extended backward stochastic Volterra integral equations and their applications to time-Inconsistent stochastic recursive control problems. Mathematical Control and Related Fields, 2021, 11 (2) : 433-478. doi: 10.3934/mcrf.2020043
[13]	Ishak Alia. Open-loop equilibriums for a general class of time-inconsistent stochastic optimal control problems. Mathematical Control and Related Fields, 2021 doi: 10.3934/mcrf.2021053
[14]	Xin Chen, Ana Bela Cruzeiro. Stochastic geodesics and forward-backward stochastic differential equations on Lie groups. Conference Publications, 2013, 2013 (special) : 115-121. doi: 10.3934/proc.2013.2013.115
[15]	Qingmeng Wei, Zhiyong Yu. Time-inconsistent recursive zero-sum stochastic differential games. Mathematical Control and Related Fields, 2018, 8 (3&4) : 1051-1079. doi: 10.3934/mcrf.2018045
[16]	Jiaqin Wei. Time-inconsistent optimal control problems with regime-switching. Mathematical Control and Related Fields, 2017, 7 (4) : 585-622. doi: 10.3934/mcrf.2017022
[17]	Joan-Andreu Lázaro-Camí, Juan-Pablo Ortega. The stochastic Hamilton-Jacobi equation. Journal of Geometric Mechanics, 2009, 1 (3) : 295-315. doi: 10.3934/jgm.2009.1.295
[18]	Sebastián Ferrer, Martin Lara. Families of canonical transformations by Hamilton-Jacobi-Poincaré equation. Application to rotational and orbital motion. Journal of Geometric Mechanics, 2010, 2 (3) : 223-241. doi: 10.3934/jgm.2010.2.223
[19]	Manuel de León, Juan Carlos Marrero, David Martín de Diego. Linear almost Poisson structures and Hamilton-Jacobi equation. Applications to nonholonomic mechanics. Journal of Geometric Mechanics, 2010, 2 (2) : 159-198. doi: 10.3934/jgm.2010.2.159
[20]	Thierry Horsin, Peter I. Kogut, Olivier Wilk. Optimal $L^2$-control problem in coefficients for a linear elliptic equation. II. Approximation of solutions and optimality conditions. Mathematical Control and Related Fields, 2016, 6 (4) : 595-628. doi: 10.3934/mcrf.2016017

2020 Impact Factor: 1.284

Tools

Metrics

Other articles
by authors

on AIMS
Jiongmin Yong
on Google Scholar
Jiongmin Yong

[Back to Top]