Time-inconsistent optimal control problems and the equilibrium HJB equation

Previous Article
Controllability of the cubic Schroedinger equation via a low-dimensional source term
MCRF Home
This Issue
Next Article

September 2012, 2(3): 271-329. doi: 10.3934/mcrf.2012.2.271

Time-inconsistent optimal control problems and the equilibrium HJB equation

1.	Department of Mathematics, University of Central Florida, Orlando, FL 32816

Received April 2012 Revised May 2012 Published August 2012

Abstract
Related Papers

A general time-inconsistent optimal control problem is considered for stochastic differential equations with deterministic coefficients. Under suitable conditions, a Hamilton-Jacobi-Bellman type equation is derived for the equilibrium value function of the problem. Well-posedness such an equation is studied, and time-consistent equilibrium strategies are constructed. As special cases, the linear-quadratic problem and a generalized Merton's portfolio problem are investigated.

Keywords: Time-inconsistent optimal control problem, forward-backward stochastic differential equation., equilibrium value function, equilibrium Hamilton-Jacobi-Bellman equation.

Mathematics Subject Classification: Primary: 93E20, 49L20, 49N10, 49N70; Secondary: 35Q93,91A23, 91A6.

Citation: Jiongmin Yong. Time-inconsistent optimal control problems and the equilibrium HJB equation. Mathematical Control & Related Fields, 2012, 2 (3) : 271-329. doi: 10.3934/mcrf.2012.2.271

References:

[1]	S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation,, Rev. Finan. Stud., 23 (2010), 2970. doi: 10.1093/rfs/hhq028. Google Scholar
[2]	T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochasitic control problem,, working paper., (). Google Scholar
[3]	T. Björk, A. Murgoci and X. Y. Zhou, Mean varaiance portfolio optimization with state dependent risk aversion,, Math. Finance, (). Google Scholar
[4]	E. V. Böhm-Bawerk, "The Positive Theory of Capital,'', Books for Libraries Press, (1891). Google Scholar
[5]	A. Caplin and J. Leahy, The recursive approach to time inconsistency,, J. Econ. Theory, 131 (2006), 134. doi: 10.1016/j.jet.2005.05.006. Google Scholar
[6]	I. Ekeland and A. Lazrak, The golden rule when preferences are time inconsistent,, Math. Finan. Econ., 4 (2010), 29. Google Scholar
[7]	I. Ekeland and T. A. Pirvu, Investment and consumption without commitment,, Math. Finan. Econ., 2 (2008), 57. Google Scholar
[8]	I. Ekeland, O. Mbodji and T. A. Pirvu, Time-consistent portfolio management,, SIAM J. Financial Math., 3 (2012), 1. Google Scholar
[9]	W. H. Fleming and H. M. Soner, "Controlled Markov Processes and Viscosity Solutions,'', 2^nd edition, 25 (2006). Google Scholar
[10]	A. Friedman, "Partial Differential Equations of Parabolic Type,'', Prentice Hall, (1964). Google Scholar
[11]	S. M. Goldman, Consistent plans,, Review of Economic Studies, 47 (1980), 533. Google Scholar
[12]	S. R. Grenadier and N. Wang, Investment under uncertainty and time-inconsistent preferences,, J. Finan. Econ., 84 (2007), 2. Google Scholar
[13]	P. J.-J. Herings and K. I. M. Rohde, Time-inconsistent preferences in a general equilibrium model,, Econ. Theory, 29 (2006), 591. doi: 10.1007/s00199-005-0020-3. Google Scholar
[14]	Y. Hu, H. Jin and X. Y. Zhou, Time-inconsistent stochastic linear-quadratic control,, , (2011). Google Scholar
[15]	D. Hume, "A Treatise of Human Nature,'', First edition, (1739). Google Scholar
[16]	W. S. Jevons, "Theory of Political Economy,'', McMillan, (1871). Google Scholar
[17]	L. Karp and I. H. Lee, Time-consistent policies,, J. Econ. Theory, 112 (2003), 353. doi: 10.1016/S0022-0531(03)00067-X. Google Scholar
[18]	D. Laibson, Golden eggs and hyperbolic discounting,, Quarterly J. Econ., 112 (1997), 443. doi: 10.1162/003355397555253. Google Scholar
[19]	J. Ma, P. Protter and J. Yong, Solving forward-backward stochastic differential equations explicitly-a four-step scheme,, Probab. Theory Related Fields, 98 (1994), 339. doi: 10.1007/BF01192258. Google Scholar
[20]	J. Ma and J. Yong, "Forward-Backward Stochastic Differential Equations and Their Applications,'', Lecture Notes in Math., 1702 (1999). Google Scholar
[21]	A. Malthus, An essay on the principle of population,, in, (1986), 2. Google Scholar
[22]	A. Marshall, "Principles of Economics,'', 1^st ed., (1890). Google Scholar
[23]	J. Marin-Solano and J. Navas, Consumption and portfolio rules for time-inconsistent investors,, European J. Operational Research, 201 (2010), 860. doi: 10.1016/j.ejor.2009.04.005. Google Scholar
[24]	J. Marin-Solano and E. V. Shevkoplyas, Non-constant discounting and differential games with random time horizon,, Automatica J. IFAC, 47 (2011), 2626. doi: 10.1016/j.automatica.2011.09.010. Google Scholar
[25]	M. Miller and M. Salmon, Dynamic games and the time inconsistency of optimal policy in open economics,, The Economic Journal, 95 (1985), 124. doi: 10.2307/2232876. Google Scholar
[26]	I. Palacios-Huerta, Time-inconsistent preferences in Adam Smith and Davis Hume,, History of Political Economy, 35 (2003), 241. doi: 10.1215/00182702-35-2-241. Google Scholar
[27]	V. Pareto, "Manuel d'Économie Politique,'', Girard and Brieve, (1909). Google Scholar
[28]	B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing,, Review of Economic Studies, 40 (1973), 391. Google Scholar
[29]	R. A. Pollak, Consistent planning,, Review of Economic Studies, 35 (1968), 185. doi: 10.2307/2296547. Google Scholar
[30]	A. Smith, "The Theory of Moral Sentiments,'', First edition, (1759). Google Scholar
[31]	R. H. Strotz, Myopia and inconsistency indynamic utility maximization,, Review of Econ. Studies, 23 (1955), 165. doi: 10.2307/2295722. Google Scholar
[32]	L. Tesfatsion, Time inconsistency of benevolent government economics,, J. Public Economics, 31 (1986), 25. doi: 10.1016/0047-2727(86)90070-8. Google Scholar
[33]	J. Yong, Backward stochastic Volterra integral equations and some related problems,, Stoch. Proc. Appl., 116 (2006), 779. doi: 10.1016/j.spa.2006.01.005. Google Scholar
[34]	J. Yong, Well-posedness and regularity of backward stochastic Volterra integral equations,, Prob. Theory Rel. Fields, 142 (2008), 21. doi: 10.1007/s00440-007-0098-6. Google Scholar
[35]	J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem,, Math. Control Related Fields, 1 (2011), 83. Google Scholar
[36]	J. Yong, Deterministic time-inconsistent optimal control problems-An essentially cooperative approach,, Acta Math. Appl. Sinica Engl. Ser., 28 (2012), 1. Google Scholar
[37]	J. Yong and X. Y. Zhou, "Stochastic Controls. Hamiltonian Systems and HJB Equations,'', Applications of Mathematics (New York), 43 (1999). Google Scholar

show all references

References:

[1]	S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation,, Rev. Finan. Stud., 23 (2010), 2970. doi: 10.1093/rfs/hhq028. Google Scholar
[2]	T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochasitic control problem,, working paper., (). Google Scholar
[3]	T. Björk, A. Murgoci and X. Y. Zhou, Mean varaiance portfolio optimization with state dependent risk aversion,, Math. Finance, (). Google Scholar
[4]	E. V. Böhm-Bawerk, "The Positive Theory of Capital,'', Books for Libraries Press, (1891). Google Scholar
[5]	A. Caplin and J. Leahy, The recursive approach to time inconsistency,, J. Econ. Theory, 131 (2006), 134. doi: 10.1016/j.jet.2005.05.006. Google Scholar
[6]	I. Ekeland and A. Lazrak, The golden rule when preferences are time inconsistent,, Math. Finan. Econ., 4 (2010), 29. Google Scholar
[7]	I. Ekeland and T. A. Pirvu, Investment and consumption without commitment,, Math. Finan. Econ., 2 (2008), 57. Google Scholar
[8]	I. Ekeland, O. Mbodji and T. A. Pirvu, Time-consistent portfolio management,, SIAM J. Financial Math., 3 (2012), 1. Google Scholar
[9]	W. H. Fleming and H. M. Soner, "Controlled Markov Processes and Viscosity Solutions,'', 2^nd edition, 25 (2006). Google Scholar
[10]	A. Friedman, "Partial Differential Equations of Parabolic Type,'', Prentice Hall, (1964). Google Scholar
[11]	S. M. Goldman, Consistent plans,, Review of Economic Studies, 47 (1980), 533. Google Scholar
[12]	S. R. Grenadier and N. Wang, Investment under uncertainty and time-inconsistent preferences,, J. Finan. Econ., 84 (2007), 2. Google Scholar
[13]	P. J.-J. Herings and K. I. M. Rohde, Time-inconsistent preferences in a general equilibrium model,, Econ. Theory, 29 (2006), 591. doi: 10.1007/s00199-005-0020-3. Google Scholar
[14]	Y. Hu, H. Jin and X. Y. Zhou, Time-inconsistent stochastic linear-quadratic control,, , (2011). Google Scholar
[15]	D. Hume, "A Treatise of Human Nature,'', First edition, (1739). Google Scholar
[16]	W. S. Jevons, "Theory of Political Economy,'', McMillan, (1871). Google Scholar
[17]	L. Karp and I. H. Lee, Time-consistent policies,, J. Econ. Theory, 112 (2003), 353. doi: 10.1016/S0022-0531(03)00067-X. Google Scholar
[18]	D. Laibson, Golden eggs and hyperbolic discounting,, Quarterly J. Econ., 112 (1997), 443. doi: 10.1162/003355397555253. Google Scholar
[19]	J. Ma, P. Protter and J. Yong, Solving forward-backward stochastic differential equations explicitly-a four-step scheme,, Probab. Theory Related Fields, 98 (1994), 339. doi: 10.1007/BF01192258. Google Scholar
[20]	J. Ma and J. Yong, "Forward-Backward Stochastic Differential Equations and Their Applications,'', Lecture Notes in Math., 1702 (1999). Google Scholar
[21]	A. Malthus, An essay on the principle of population,, in, (1986), 2. Google Scholar
[22]	A. Marshall, "Principles of Economics,'', 1^st ed., (1890). Google Scholar
[23]	J. Marin-Solano and J. Navas, Consumption and portfolio rules for time-inconsistent investors,, European J. Operational Research, 201 (2010), 860. doi: 10.1016/j.ejor.2009.04.005. Google Scholar
[24]	J. Marin-Solano and E. V. Shevkoplyas, Non-constant discounting and differential games with random time horizon,, Automatica J. IFAC, 47 (2011), 2626. doi: 10.1016/j.automatica.2011.09.010. Google Scholar
[25]	M. Miller and M. Salmon, Dynamic games and the time inconsistency of optimal policy in open economics,, The Economic Journal, 95 (1985), 124. doi: 10.2307/2232876. Google Scholar
[26]	I. Palacios-Huerta, Time-inconsistent preferences in Adam Smith and Davis Hume,, History of Political Economy, 35 (2003), 241. doi: 10.1215/00182702-35-2-241. Google Scholar
[27]	V. Pareto, "Manuel d'Économie Politique,'', Girard and Brieve, (1909). Google Scholar
[28]	B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing,, Review of Economic Studies, 40 (1973), 391. Google Scholar
[29]	R. A. Pollak, Consistent planning,, Review of Economic Studies, 35 (1968), 185. doi: 10.2307/2296547. Google Scholar
[30]	A. Smith, "The Theory of Moral Sentiments,'', First edition, (1759). Google Scholar
[31]	R. H. Strotz, Myopia and inconsistency indynamic utility maximization,, Review of Econ. Studies, 23 (1955), 165. doi: 10.2307/2295722. Google Scholar
[32]	L. Tesfatsion, Time inconsistency of benevolent government economics,, J. Public Economics, 31 (1986), 25. doi: 10.1016/0047-2727(86)90070-8. Google Scholar
[33]	J. Yong, Backward stochastic Volterra integral equations and some related problems,, Stoch. Proc. Appl., 116 (2006), 779. doi: 10.1016/j.spa.2006.01.005. Google Scholar
[34]	J. Yong, Well-posedness and regularity of backward stochastic Volterra integral equations,, Prob. Theory Rel. Fields, 142 (2008), 21. doi: 10.1007/s00440-007-0098-6. Google Scholar
[35]	J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem,, Math. Control Related Fields, 1 (2011), 83. Google Scholar
[36]	J. Yong, Deterministic time-inconsistent optimal control problems-An essentially cooperative approach,, Acta Math. Appl. Sinica Engl. Ser., 28 (2012), 1. Google Scholar
[37]	J. Yong and X. Y. Zhou, "Stochastic Controls. Hamiltonian Systems and HJB Equations,'', Applications of Mathematics (New York), 43 (1999). Google Scholar

[1]	Haiyang Wang, Zhen Wu. Time-inconsistent optimal control problem with random coefficients and stochastic equilibrium HJB equation. Mathematical Control & Related Fields, 2015, 5 (3) : 651-678. doi: 10.3934/mcrf.2015.5.651
[2]	Steven Richardson, Song Wang. The viscosity approximation to the Hamilton-Jacobi-Bellman equation in optimal feedback control: Upper bounds for extended domains. Journal of Industrial & Management Optimization, 2010, 6 (1) : 161-175. doi: 10.3934/jimo.2010.6.161
[3]	Jean-Claude Zambrini. On the geometry of the Hamilton-Jacobi-Bellman equation. Journal of Geometric Mechanics, 2009, 1 (3) : 369-387. doi: 10.3934/jgm.2009.1.369
[4]	Daniele Castorina, Annalisa Cesaroni, Luca Rossi. On a parabolic Hamilton-Jacobi-Bellman equation degenerating at the boundary. Communications on Pure & Applied Analysis, 2016, 15 (4) : 1251-1263. doi: 10.3934/cpaa.2016.15.1251
[5]	Yufeng Shi, Tianxiao Wang, Jiongmin Yong. Optimal control problems of forward-backward stochastic Volterra integral equations. Mathematical Control & Related Fields, 2015, 5 (3) : 613-649. doi: 10.3934/mcrf.2015.5.613
[6]	Ishak Alia. A non-exponential discounting time-inconsistent stochastic optimal control problem for jump-diffusion. Mathematical Control & Related Fields, 2019, 9 (3) : 541-570. doi: 10.3934/mcrf.2019025
[7]	Jiongmin Yong. A deterministic linear quadratic time-inconsistent optimal control problem. Mathematical Control & Related Fields, 2011, 1 (1) : 83-118. doi: 10.3934/mcrf.2011.1.83
[8]	Xin Chen, Ana Bela Cruzeiro. Stochastic geodesics and forward-backward stochastic differential equations on Lie groups. Conference Publications, 2013, 2013 (special) : 115-121. doi: 10.3934/proc.2013.2013.115
[9]	Qingmeng Wei, Zhiyong Yu. Time-inconsistent recursive zero-sum stochastic differential games. Mathematical Control & Related Fields, 2018, 8 (3&4) : 1051-1079. doi: 10.3934/mcrf.2018045
[10]	Jiaqin Wei. Time-inconsistent optimal control problems with regime-switching. Mathematical Control & Related Fields, 2017, 7 (4) : 585-622. doi: 10.3934/mcrf.2017022
[11]	Joan-Andreu Lázaro-Camí, Juan-Pablo Ortega. The stochastic Hamilton-Jacobi equation. Journal of Geometric Mechanics, 2009, 1 (3) : 295-315. doi: 10.3934/jgm.2009.1.295
[12]	Mohamed Assellaou, Olivier Bokanowski, Hasnaa Zidani. Error estimates for second order Hamilton-Jacobi-Bellman equations. Approximation of probabilistic reachable sets. Discrete & Continuous Dynamical Systems - A, 2015, 35 (9) : 3933-3964. doi: 10.3934/dcds.2015.35.3933
[13]	Sebastián Ferrer, Martin Lara. Families of canonical transformations by Hamilton-Jacobi-Poincaré equation. Application to rotational and orbital motion. Journal of Geometric Mechanics, 2010, 2 (3) : 223-241. doi: 10.3934/jgm.2010.2.223
[14]	Manuel de León, Juan Carlos Marrero, David Martín de Diego. Linear almost Poisson structures and Hamilton-Jacobi equation. Applications to nonholonomic mechanics. Journal of Geometric Mechanics, 2010, 2 (2) : 159-198. doi: 10.3934/jgm.2010.2.159
[15]	Thierry Horsin, Peter I. Kogut, Olivier Wilk. Optimal $L^2$-control problem in coefficients for a linear elliptic equation. II. Approximation of solutions and optimality conditions. Mathematical Control & Related Fields, 2016, 6 (4) : 595-628. doi: 10.3934/mcrf.2016017
[16]	Thierry Horsin, Peter I. Kogut. Optimal $L^2$-control problem in coefficients for a linear elliptic equation. I. Existence result. Mathematical Control & Related Fields, 2015, 5 (1) : 73-96. doi: 10.3934/mcrf.2015.5.73
[17]	Jie Xiong, Shuaiqi Zhang, Yi Zhuang. A partially observed non-zero sum differential game of forward-backward stochastic differential equations and its application in finance. Mathematical Control & Related Fields, 2019, 9 (2) : 257-276. doi: 10.3934/mcrf.2019013
[18]	Tianxiao Wang. Characterizations of equilibrium controls in time inconsistent mean-field stochastic linear quadratic problems. I. Mathematical Control & Related Fields, 2019, 9 (2) : 385-409. doi: 10.3934/mcrf.2019018
[19]	Nicolas Forcadel, Mamdouh Zaydan. A comparison principle for Hamilton-Jacobi equation with moving in time boundary. Evolution Equations & Control Theory, 2019, 8 (3) : 543-565. doi: 10.3934/eect.2019026
[20]	Nguyen Huy Chieu, Jen-Chih Yao. Subgradients of the optimal value function in a parametric discrete optimal control problem. Journal of Industrial & Management Optimization, 2010, 6 (2) : 401-410. doi: 10.3934/jimo.2010.6.401

2018 Impact Factor: 1.292

American Institute of Mathematical Sciences