Time-inconsistent optimal control problems and the equilibrium HJB equation

Previous Article
Controllability of the cubic Schroedinger equation via a low-dimensional source term
MCRF Home
This Issue
Next Article

September 2012, 2(3): 271-329. doi: 10.3934/mcrf.2012.2.271

Time-inconsistent optimal control problems and the equilibrium HJB equation

1.	Department of Mathematics, University of Central Florida, Orlando, FL 32816

Received April 2012 Revised May 2012 Published August 2012

Abstract
Related Papers

A general time-inconsistent optimal control problem is considered for stochastic differential equations with deterministic coefficients. Under suitable conditions, a Hamilton-Jacobi-Bellman type equation is derived for the equilibrium value function of the problem. Well-posedness such an equation is studied, and time-consistent equilibrium strategies are constructed. As special cases, the linear-quadratic problem and a generalized Merton's portfolio problem are investigated.

Keywords: Time-inconsistent optimal control problem, forward-backward stochastic differential equation., equilibrium value function, equilibrium Hamilton-Jacobi-Bellman equation.

Mathematics Subject Classification: Primary: 93E20, 49L20, 49N10, 49N70; Secondary: 35Q93,91A23, 91A6.

Citation: Jiongmin Yong. Time-inconsistent optimal control problems and the equilibrium HJB equation. Mathematical Control & Related Fields, 2012, 2 (3) : 271-329. doi: 10.3934/mcrf.2012.2.271

References:

[1]	S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation,, Rev. Finan. Stud., 23 (2010), 2970. doi: 10.1093/rfs/hhq028.
[2]	T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochasitic control problem,, working paper., ().
[3]	T. Björk, A. Murgoci and X. Y. Zhou, Mean varaiance portfolio optimization with state dependent risk aversion,, Math. Finance, ().
[4]	E. V. Böhm-Bawerk, "The Positive Theory of Capital,'', Books for Libraries Press, (1891).
[5]	A. Caplin and J. Leahy, The recursive approach to time inconsistency,, J. Econ. Theory, 131 (2006), 134. doi: 10.1016/j.jet.2005.05.006.
[6]	I. Ekeland and A. Lazrak, The golden rule when preferences are time inconsistent,, Math. Finan. Econ., 4 (2010), 29.
[7]	I. Ekeland and T. A. Pirvu, Investment and consumption without commitment,, Math. Finan. Econ., 2 (2008), 57.
[8]	I. Ekeland, O. Mbodji and T. A. Pirvu, Time-consistent portfolio management,, SIAM J. Financial Math., 3 (2012), 1.
[9]	W. H. Fleming and H. M. Soner, "Controlled Markov Processes and Viscosity Solutions,'', 2^nd edition, 25 (2006).
[10]	A. Friedman, "Partial Differential Equations of Parabolic Type,'', Prentice Hall, (1964).
[11]	S. M. Goldman, Consistent plans,, Review of Economic Studies, 47 (1980), 533.
[12]	S. R. Grenadier and N. Wang, Investment under uncertainty and time-inconsistent preferences,, J. Finan. Econ., 84 (2007), 2.
[13]	P. J.-J. Herings and K. I. M. Rohde, Time-inconsistent preferences in a general equilibrium model,, Econ. Theory, 29 (2006), 591. doi: 10.1007/s00199-005-0020-3.
[14]	Y. Hu, H. Jin and X. Y. Zhou, Time-inconsistent stochastic linear-quadratic control,, , (2011).
[15]	D. Hume, "A Treatise of Human Nature,'', First edition, (1739).
[16]	W. S. Jevons, "Theory of Political Economy,'', McMillan, (1871).
[17]	L. Karp and I. H. Lee, Time-consistent policies,, J. Econ. Theory, 112 (2003), 353. doi: 10.1016/S0022-0531(03)00067-X.
[18]	D. Laibson, Golden eggs and hyperbolic discounting,, Quarterly J. Econ., 112 (1997), 443. doi: 10.1162/003355397555253.
[19]	J. Ma, P. Protter and J. Yong, Solving forward-backward stochastic differential equations explicitly-a four-step scheme,, Probab. Theory Related Fields, 98 (1994), 339. doi: 10.1007/BF01192258.
[20]	J. Ma and J. Yong, "Forward-Backward Stochastic Differential Equations and Their Applications,'', Lecture Notes in Math., 1702 (1999).
[21]	A. Malthus, An essay on the principle of population,, in, (1986), 2.
[22]	A. Marshall, "Principles of Economics,'', 1^st ed., (1890).
[23]	J. Marin-Solano and J. Navas, Consumption and portfolio rules for time-inconsistent investors,, European J. Operational Research, 201 (2010), 860. doi: 10.1016/j.ejor.2009.04.005.
[24]	J. Marin-Solano and E. V. Shevkoplyas, Non-constant discounting and differential games with random time horizon,, Automatica J. IFAC, 47 (2011), 2626. doi: 10.1016/j.automatica.2011.09.010.
[25]	M. Miller and M. Salmon, Dynamic games and the time inconsistency of optimal policy in open economics,, The Economic Journal, 95 (1985), 124. doi: 10.2307/2232876.
[26]	I. Palacios-Huerta, Time-inconsistent preferences in Adam Smith and Davis Hume,, History of Political Economy, 35 (2003), 241. doi: 10.1215/00182702-35-2-241.
[27]	V. Pareto, "Manuel d'Économie Politique,'', Girard and Brieve, (1909).
[28]	B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing,, Review of Economic Studies, 40 (1973), 391.
[29]	R. A. Pollak, Consistent planning,, Review of Economic Studies, 35 (1968), 185. doi: 10.2307/2296547.
[30]	A. Smith, "The Theory of Moral Sentiments,'', First edition, (1759).
[31]	R. H. Strotz, Myopia and inconsistency indynamic utility maximization,, Review of Econ. Studies, 23 (1955), 165. doi: 10.2307/2295722.
[32]	L. Tesfatsion, Time inconsistency of benevolent government economics,, J. Public Economics, 31 (1986), 25. doi: 10.1016/0047-2727(86)90070-8.
[33]	J. Yong, Backward stochastic Volterra integral equations and some related problems,, Stoch. Proc. Appl., 116 (2006), 779. doi: 10.1016/j.spa.2006.01.005.
[34]	J. Yong, Well-posedness and regularity of backward stochastic Volterra integral equations,, Prob. Theory Rel. Fields, 142 (2008), 21. doi: 10.1007/s00440-007-0098-6.
[35]	J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem,, Math. Control Related Fields, 1 (2011), 83.
[36]	J. Yong, Deterministic time-inconsistent optimal control problems-An essentially cooperative approach,, Acta Math. Appl. Sinica Engl. Ser., 28 (2012), 1.
[37]	J. Yong and X. Y. Zhou, "Stochastic Controls. Hamiltonian Systems and HJB Equations,'', Applications of Mathematics (New York), 43 (1999).

show all references

References:

[1]	S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation,, Rev. Finan. Stud., 23 (2010), 2970. doi: 10.1093/rfs/hhq028.
[2]	T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochasitic control problem,, working paper., ().
[3]	T. Björk, A. Murgoci and X. Y. Zhou, Mean varaiance portfolio optimization with state dependent risk aversion,, Math. Finance, ().
[4]	E. V. Böhm-Bawerk, "The Positive Theory of Capital,'', Books for Libraries Press, (1891).
[5]	A. Caplin and J. Leahy, The recursive approach to time inconsistency,, J. Econ. Theory, 131 (2006), 134. doi: 10.1016/j.jet.2005.05.006.
[6]	I. Ekeland and A. Lazrak, The golden rule when preferences are time inconsistent,, Math. Finan. Econ., 4 (2010), 29.
[7]	I. Ekeland and T. A. Pirvu, Investment and consumption without commitment,, Math. Finan. Econ., 2 (2008), 57.
[8]	I. Ekeland, O. Mbodji and T. A. Pirvu, Time-consistent portfolio management,, SIAM J. Financial Math., 3 (2012), 1.
[9]	W. H. Fleming and H. M. Soner, "Controlled Markov Processes and Viscosity Solutions,'', 2^nd edition, 25 (2006).
[10]	A. Friedman, "Partial Differential Equations of Parabolic Type,'', Prentice Hall, (1964).
[11]	S. M. Goldman, Consistent plans,, Review of Economic Studies, 47 (1980), 533.
[12]	S. R. Grenadier and N. Wang, Investment under uncertainty and time-inconsistent preferences,, J. Finan. Econ., 84 (2007), 2.
[13]	P. J.-J. Herings and K. I. M. Rohde, Time-inconsistent preferences in a general equilibrium model,, Econ. Theory, 29 (2006), 591. doi: 10.1007/s00199-005-0020-3.
[14]	Y. Hu, H. Jin and X. Y. Zhou, Time-inconsistent stochastic linear-quadratic control,, , (2011).
[15]	D. Hume, "A Treatise of Human Nature,'', First edition, (1739).
[16]	W. S. Jevons, "Theory of Political Economy,'', McMillan, (1871).
[17]	L. Karp and I. H. Lee, Time-consistent policies,, J. Econ. Theory, 112 (2003), 353. doi: 10.1016/S0022-0531(03)00067-X.
[18]	D. Laibson, Golden eggs and hyperbolic discounting,, Quarterly J. Econ., 112 (1997), 443. doi: 10.1162/003355397555253.
[19]	J. Ma, P. Protter and J. Yong, Solving forward-backward stochastic differential equations explicitly-a four-step scheme,, Probab. Theory Related Fields, 98 (1994), 339. doi: 10.1007/BF01192258.
[20]	J. Ma and J. Yong, "Forward-Backward Stochastic Differential Equations and Their Applications,'', Lecture Notes in Math., 1702 (1999).
[21]	A. Malthus, An essay on the principle of population,, in, (1986), 2.
[22]	A. Marshall, "Principles of Economics,'', 1^st ed., (1890).
[23]	J. Marin-Solano and J. Navas, Consumption and portfolio rules for time-inconsistent investors,, European J. Operational Research, 201 (2010), 860. doi: 10.1016/j.ejor.2009.04.005.
[24]	J. Marin-Solano and E. V. Shevkoplyas, Non-constant discounting and differential games with random time horizon,, Automatica J. IFAC, 47 (2011), 2626. doi: 10.1016/j.automatica.2011.09.010.
[25]	M. Miller and M. Salmon, Dynamic games and the time inconsistency of optimal policy in open economics,, The Economic Journal, 95 (1985), 124. doi: 10.2307/2232876.
[26]	I. Palacios-Huerta, Time-inconsistent preferences in Adam Smith and Davis Hume,, History of Political Economy, 35 (2003), 241. doi: 10.1215/00182702-35-2-241.
[27]	V. Pareto, "Manuel d'Économie Politique,'', Girard and Brieve, (1909).
[28]	B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing,, Review of Economic Studies, 40 (1973), 391.
[29]	R. A. Pollak, Consistent planning,, Review of Economic Studies, 35 (1968), 185. doi: 10.2307/2296547.
[30]	A. Smith, "The Theory of Moral Sentiments,'', First edition, (1759).
[31]	R. H. Strotz, Myopia and inconsistency indynamic utility maximization,, Review of Econ. Studies, 23 (1955), 165. doi: 10.2307/2295722.
[32]	L. Tesfatsion, Time inconsistency of benevolent government economics,, J. Public Economics, 31 (1986), 25. doi: 10.1016/0047-2727(86)90070-8.
[33]	J. Yong, Backward stochastic Volterra integral equations and some related problems,, Stoch. Proc. Appl., 116 (2006), 779. doi: 10.1016/j.spa.2006.01.005.
[34]	J. Yong, Well-posedness and regularity of backward stochastic Volterra integral equations,, Prob. Theory Rel. Fields, 142 (2008), 21. doi: 10.1007/s00440-007-0098-6.
[35]	J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem,, Math. Control Related Fields, 1 (2011), 83.
[36]	J. Yong, Deterministic time-inconsistent optimal control problems-An essentially cooperative approach,, Acta Math. Appl. Sinica Engl. Ser., 28 (2012), 1.
[37]	J. Yong and X. Y. Zhou, "Stochastic Controls. Hamiltonian Systems and HJB Equations,'', Applications of Mathematics (New York), 43 (1999).

[1]	Haiyang Wang, Zhen Wu. Time-inconsistent optimal control problem with random coefficients and stochastic equilibrium HJB equation. Mathematical Control & Related Fields, 2015, 5 (3) : 651-678. doi: 10.3934/mcrf.2015.5.651
[2]	Steven Richardson, Song Wang. The viscosity approximation to the Hamilton-Jacobi-Bellman equation in optimal feedback control: Upper bounds for extended domains. Journal of Industrial & Management Optimization, 2010, 6 (1) : 161-175. doi: 10.3934/jimo.2010.6.161
[3]	Jean-Claude Zambrini. On the geometry of the Hamilton-Jacobi-Bellman equation. Journal of Geometric Mechanics, 2009, 1 (3) : 369-387. doi: 10.3934/jgm.2009.1.369
[4]	Daniele Castorina, Annalisa Cesaroni, Luca Rossi. On a parabolic Hamilton-Jacobi-Bellman equation degenerating at the boundary. Communications on Pure & Applied Analysis, 2016, 15 (4) : 1251-1263. doi: 10.3934/cpaa.2016.15.1251
[5]	Yufeng Shi, Tianxiao Wang, Jiongmin Yong. Optimal control problems of forward-backward stochastic Volterra integral equations. Mathematical Control & Related Fields, 2015, 5 (3) : 613-649. doi: 10.3934/mcrf.2015.5.613
[6]	Ishak Alia. A non-exponential discounting time-inconsistent stochastic optimal control problem for jump-diffusion. Mathematical Control & Related Fields, 2019, 9 (3) : 541-570. doi: 10.3934/mcrf.2019025
[7]	Jiongmin Yong. A deterministic linear quadratic time-inconsistent optimal control problem. Mathematical Control & Related Fields, 2011, 1 (1) : 83-118. doi: 10.3934/mcrf.2011.1.83
[8]	Xin Chen, Ana Bela Cruzeiro. Stochastic geodesics and forward-backward stochastic differential equations on Lie groups. Conference Publications, 2013, 2013 (special) : 115-121. doi: 10.3934/proc.2013.2013.115
[9]	Qingmeng Wei, Zhiyong Yu. Time-inconsistent recursive zero-sum stochastic differential games. Mathematical Control & Related Fields, 2018, 8 (3&4) : 1051-1079. doi: 10.3934/mcrf.2018045
[10]	Jiaqin Wei. Time-inconsistent optimal control problems with regime-switching. Mathematical Control & Related Fields, 2017, 7 (4) : 585-622. doi: 10.3934/mcrf.2017022
[11]	Joan-Andreu Lázaro-Camí, Juan-Pablo Ortega. The stochastic Hamilton-Jacobi equation. Journal of Geometric Mechanics, 2009, 1 (3) : 295-315. doi: 10.3934/jgm.2009.1.295
[12]	Mohamed Assellaou, Olivier Bokanowski, Hasnaa Zidani. Error estimates for second order Hamilton-Jacobi-Bellman equations. Approximation of probabilistic reachable sets. Discrete & Continuous Dynamical Systems - A, 2015, 35 (9) : 3933-3964. doi: 10.3934/dcds.2015.35.3933
[13]	Sebastián Ferrer, Martin Lara. Families of canonical transformations by Hamilton-Jacobi-Poincaré equation. Application to rotational and orbital motion. Journal of Geometric Mechanics, 2010, 2 (3) : 223-241. doi: 10.3934/jgm.2010.2.223
[14]	Manuel de León, Juan Carlos Marrero, David Martín de Diego. Linear almost Poisson structures and Hamilton-Jacobi equation. Applications to nonholonomic mechanics. Journal of Geometric Mechanics, 2010, 2 (2) : 159-198. doi: 10.3934/jgm.2010.2.159
[15]	Thierry Horsin, Peter I. Kogut, Olivier Wilk. Optimal $L^2$-control problem in coefficients for a linear elliptic equation. II. Approximation of solutions and optimality conditions. Mathematical Control & Related Fields, 2016, 6 (4) : 595-628. doi: 10.3934/mcrf.2016017
[16]	Thierry Horsin, Peter I. Kogut. Optimal $L^2$-control problem in coefficients for a linear elliptic equation. I. Existence result. Mathematical Control & Related Fields, 2015, 5 (1) : 73-96. doi: 10.3934/mcrf.2015.5.73
[17]	Jie Xiong, Shuaiqi Zhang, Yi Zhuang. A partially observed non-zero sum differential game of forward-backward stochastic differential equations and its application in finance. Mathematical Control & Related Fields, 2019, 9 (2) : 257-276. doi: 10.3934/mcrf.2019013
[18]	Tianxiao Wang. Characterizations of equilibrium controls in time inconsistent mean-field stochastic linear quadratic problems. I. Mathematical Control & Related Fields, 2019, 9 (2) : 385-409. doi: 10.3934/mcrf.2019018
[19]	Nicolas Forcadel, Mamdouh Zaydan. A comparison principle for Hamilton-Jacobi equation with moving in time boundary. Evolution Equations & Control Theory, 2019, 8 (3) : 543-565. doi: 10.3934/eect.2019026
[20]	Nguyen Huy Chieu, Jen-Chih Yao. Subgradients of the optimal value function in a parametric discrete optimal control problem. Journal of Industrial & Management Optimization, 2010, 6 (2) : 401-410. doi: 10.3934/jimo.2010.6.401

2018 Impact Factor: 1.292

American Institute of Mathematical Sciences