# American Institute of Mathematical Sciences

2012, 2(3): 271-329. doi: 10.3934/mcrf.2012.2.271

## Time-inconsistent optimal control problems and the equilibrium HJB equation

 1 Department of Mathematics, University of Central Florida, Orlando, FL 32816

Received  April 2012 Revised  May 2012 Published  August 2012

A general time-inconsistent optimal control problem is considered for stochastic differential equations with deterministic coefficients. Under suitable conditions, a Hamilton-Jacobi-Bellman type equation is derived for the equilibrium value function of the problem. Well-posedness such an equation is studied, and time-consistent equilibrium strategies are constructed. As special cases, the linear-quadratic problem and a generalized Merton's portfolio problem are investigated.
Citation: Jiongmin Yong. Time-inconsistent optimal control problems and the equilibrium HJB equation. Mathematical Control & Related Fields, 2012, 2 (3) : 271-329. doi: 10.3934/mcrf.2012.2.271
##### References:
 [1] S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation,, Rev. Finan. Stud., 23 (2010), 2970. doi: 10.1093/rfs/hhq028. [2] T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochasitic control problem,, working paper., (). [3] T. Björk, A. Murgoci and X. Y. Zhou, Mean varaiance portfolio optimization with state dependent risk aversion,, Math. Finance, (). [4] E. V. Böhm-Bawerk, "The Positive Theory of Capital,'', Books for Libraries Press, (1891). [5] A. Caplin and J. Leahy, The recursive approach to time inconsistency,, J. Econ. Theory, 131 (2006), 134. doi: 10.1016/j.jet.2005.05.006. [6] I. Ekeland and A. Lazrak, The golden rule when preferences are time inconsistent,, Math. Finan. Econ., 4 (2010), 29. [7] I. Ekeland and T. A. Pirvu, Investment and consumption without commitment,, Math. Finan. Econ., 2 (2008), 57. [8] I. Ekeland, O. Mbodji and T. A. Pirvu, Time-consistent portfolio management,, SIAM J. Financial Math., 3 (2012), 1. [9] W. H. Fleming and H. M. Soner, "Controlled Markov Processes and Viscosity Solutions,'', 2nd edition, 25 (2006). [10] A. Friedman, "Partial Differential Equations of Parabolic Type,'', Prentice Hall, (1964). [11] S. M. Goldman, Consistent plans,, Review of Economic Studies, 47 (1980), 533. [12] S. R. Grenadier and N. Wang, Investment under uncertainty and time-inconsistent preferences,, J. Finan. Econ., 84 (2007), 2. [13] P. J.-J. Herings and K. I. M. Rohde, Time-inconsistent preferences in a general equilibrium model,, Econ. Theory, 29 (2006), 591. doi: 10.1007/s00199-005-0020-3. [14] Y. Hu, H. Jin and X. Y. Zhou, Time-inconsistent stochastic linear-quadratic control,, , (2011). [15] D. Hume, "A Treatise of Human Nature,'', First edition, (1739). [16] W. S. Jevons, "Theory of Political Economy,'', McMillan, (1871). [17] L. Karp and I. H. Lee, Time-consistent policies,, J. Econ. Theory, 112 (2003), 353. doi: 10.1016/S0022-0531(03)00067-X. [18] D. Laibson, Golden eggs and hyperbolic discounting,, Quarterly J. Econ., 112 (1997), 443. doi: 10.1162/003355397555253. [19] J. Ma, P. Protter and J. Yong, Solving forward-backward stochastic differential equations explicitly-a four-step scheme,, Probab. Theory Related Fields, 98 (1994), 339. doi: 10.1007/BF01192258. [20] J. Ma and J. Yong, "Forward-Backward Stochastic Differential Equations and Their Applications,'', Lecture Notes in Math., 1702 (1999). [21] A. Malthus, An essay on the principle of population,, in, (1986), 2. [22] A. Marshall, "Principles of Economics,'', 1st ed., (1890). [23] J. Marin-Solano and J. Navas, Consumption and portfolio rules for time-inconsistent investors,, European J. Operational Research, 201 (2010), 860. doi: 10.1016/j.ejor.2009.04.005. [24] J. Marin-Solano and E. V. Shevkoplyas, Non-constant discounting and differential games with random time horizon,, Automatica J. IFAC, 47 (2011), 2626. doi: 10.1016/j.automatica.2011.09.010. [25] M. Miller and M. Salmon, Dynamic games and the time inconsistency of optimal policy in open economics,, The Economic Journal, 95 (1985), 124. doi: 10.2307/2232876. [26] I. Palacios-Huerta, Time-inconsistent preferences in Adam Smith and Davis Hume,, History of Political Economy, 35 (2003), 241. doi: 10.1215/00182702-35-2-241. [27] V. Pareto, "Manuel d'Économie Politique,'', Girard and Brieve, (1909). [28] B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing,, Review of Economic Studies, 40 (1973), 391. [29] R. A. Pollak, Consistent planning,, Review of Economic Studies, 35 (1968), 185. doi: 10.2307/2296547. [30] A. Smith, "The Theory of Moral Sentiments,'', First edition, (1759). [31] R. H. Strotz, Myopia and inconsistency indynamic utility maximization,, Review of Econ. Studies, 23 (1955), 165. doi: 10.2307/2295722. [32] L. Tesfatsion, Time inconsistency of benevolent government economics,, J. Public Economics, 31 (1986), 25. doi: 10.1016/0047-2727(86)90070-8. [33] J. Yong, Backward stochastic Volterra integral equations and some related problems,, Stoch. Proc. Appl., 116 (2006), 779. doi: 10.1016/j.spa.2006.01.005. [34] J. Yong, Well-posedness and regularity of backward stochastic Volterra integral equations,, Prob. Theory Rel. Fields, 142 (2008), 21. doi: 10.1007/s00440-007-0098-6. [35] J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem,, Math. Control Related Fields, 1 (2011), 83. [36] J. Yong, Deterministic time-inconsistent optimal control problems-An essentially cooperative approach,, Acta Math. Appl. Sinica Engl. Ser., 28 (2012), 1. [37] J. Yong and X. Y. Zhou, "Stochastic Controls. Hamiltonian Systems and HJB Equations,'', Applications of Mathematics (New York), 43 (1999).

show all references

##### References:
 [1] S. Basak and G. Chabakauri, Dynamic mean-variance asset allocation,, Rev. Finan. Stud., 23 (2010), 2970. doi: 10.1093/rfs/hhq028. [2] T. Björk and A. Murgoci, A general theory of Markovian time inconsistent stochasitic control problem,, working paper., (). [3] T. Björk, A. Murgoci and X. Y. Zhou, Mean varaiance portfolio optimization with state dependent risk aversion,, Math. Finance, (). [4] E. V. Böhm-Bawerk, "The Positive Theory of Capital,'', Books for Libraries Press, (1891). [5] A. Caplin and J. Leahy, The recursive approach to time inconsistency,, J. Econ. Theory, 131 (2006), 134. doi: 10.1016/j.jet.2005.05.006. [6] I. Ekeland and A. Lazrak, The golden rule when preferences are time inconsistent,, Math. Finan. Econ., 4 (2010), 29. [7] I. Ekeland and T. A. Pirvu, Investment and consumption without commitment,, Math. Finan. Econ., 2 (2008), 57. [8] I. Ekeland, O. Mbodji and T. A. Pirvu, Time-consistent portfolio management,, SIAM J. Financial Math., 3 (2012), 1. [9] W. H. Fleming and H. M. Soner, "Controlled Markov Processes and Viscosity Solutions,'', 2nd edition, 25 (2006). [10] A. Friedman, "Partial Differential Equations of Parabolic Type,'', Prentice Hall, (1964). [11] S. M. Goldman, Consistent plans,, Review of Economic Studies, 47 (1980), 533. [12] S. R. Grenadier and N. Wang, Investment under uncertainty and time-inconsistent preferences,, J. Finan. Econ., 84 (2007), 2. [13] P. J.-J. Herings and K. I. M. Rohde, Time-inconsistent preferences in a general equilibrium model,, Econ. Theory, 29 (2006), 591. doi: 10.1007/s00199-005-0020-3. [14] Y. Hu, H. Jin and X. Y. Zhou, Time-inconsistent stochastic linear-quadratic control,, , (2011). [15] D. Hume, "A Treatise of Human Nature,'', First edition, (1739). [16] W. S. Jevons, "Theory of Political Economy,'', McMillan, (1871). [17] L. Karp and I. H. Lee, Time-consistent policies,, J. Econ. Theory, 112 (2003), 353. doi: 10.1016/S0022-0531(03)00067-X. [18] D. Laibson, Golden eggs and hyperbolic discounting,, Quarterly J. Econ., 112 (1997), 443. doi: 10.1162/003355397555253. [19] J. Ma, P. Protter and J. Yong, Solving forward-backward stochastic differential equations explicitly-a four-step scheme,, Probab. Theory Related Fields, 98 (1994), 339. doi: 10.1007/BF01192258. [20] J. Ma and J. Yong, "Forward-Backward Stochastic Differential Equations and Their Applications,'', Lecture Notes in Math., 1702 (1999). [21] A. Malthus, An essay on the principle of population,, in, (1986), 2. [22] A. Marshall, "Principles of Economics,'', 1st ed., (1890). [23] J. Marin-Solano and J. Navas, Consumption and portfolio rules for time-inconsistent investors,, European J. Operational Research, 201 (2010), 860. doi: 10.1016/j.ejor.2009.04.005. [24] J. Marin-Solano and E. V. Shevkoplyas, Non-constant discounting and differential games with random time horizon,, Automatica J. IFAC, 47 (2011), 2626. doi: 10.1016/j.automatica.2011.09.010. [25] M. Miller and M. Salmon, Dynamic games and the time inconsistency of optimal policy in open economics,, The Economic Journal, 95 (1985), 124. doi: 10.2307/2232876. [26] I. Palacios-Huerta, Time-inconsistent preferences in Adam Smith and Davis Hume,, History of Political Economy, 35 (2003), 241. doi: 10.1215/00182702-35-2-241. [27] V. Pareto, "Manuel d'Économie Politique,'', Girard and Brieve, (1909). [28] B. Peleg and M. E. Yaari, On the existence of a consistent course of action when tastes are changing,, Review of Economic Studies, 40 (1973), 391. [29] R. A. Pollak, Consistent planning,, Review of Economic Studies, 35 (1968), 185. doi: 10.2307/2296547. [30] A. Smith, "The Theory of Moral Sentiments,'', First edition, (1759). [31] R. H. Strotz, Myopia and inconsistency indynamic utility maximization,, Review of Econ. Studies, 23 (1955), 165. doi: 10.2307/2295722. [32] L. Tesfatsion, Time inconsistency of benevolent government economics,, J. Public Economics, 31 (1986), 25. doi: 10.1016/0047-2727(86)90070-8. [33] J. Yong, Backward stochastic Volterra integral equations and some related problems,, Stoch. Proc. Appl., 116 (2006), 779. doi: 10.1016/j.spa.2006.01.005. [34] J. Yong, Well-posedness and regularity of backward stochastic Volterra integral equations,, Prob. Theory Rel. Fields, 142 (2008), 21. doi: 10.1007/s00440-007-0098-6. [35] J. Yong, A deterministic linear quadratic time-inconsistent optimal control problem,, Math. Control Related Fields, 1 (2011), 83. [36] J. Yong, Deterministic time-inconsistent optimal control problems-An essentially cooperative approach,, Acta Math. Appl. Sinica Engl. Ser., 28 (2012), 1. [37] J. Yong and X. Y. Zhou, "Stochastic Controls. Hamiltonian Systems and HJB Equations,'', Applications of Mathematics (New York), 43 (1999).
 [1] Haiyang Wang, Zhen Wu. Time-inconsistent optimal control problem with random coefficients and stochastic equilibrium HJB equation. Mathematical Control & Related Fields, 2015, 5 (3) : 651-678. doi: 10.3934/mcrf.2015.5.651 [2] Steven Richardson, Song Wang. The viscosity approximation to the Hamilton-Jacobi-Bellman equation in optimal feedback control: Upper bounds for extended domains. Journal of Industrial & Management Optimization, 2010, 6 (1) : 161-175. doi: 10.3934/jimo.2010.6.161 [3] Jean-Claude Zambrini. On the geometry of the Hamilton-Jacobi-Bellman equation. Journal of Geometric Mechanics, 2009, 1 (3) : 369-387. doi: 10.3934/jgm.2009.1.369 [4] Daniele Castorina, Annalisa Cesaroni, Luca Rossi. On a parabolic Hamilton-Jacobi-Bellman equation degenerating at the boundary. Communications on Pure & Applied Analysis, 2016, 15 (4) : 1251-1263. doi: 10.3934/cpaa.2016.15.1251 [5] Yufeng Shi, Tianxiao Wang, Jiongmin Yong. Optimal control problems of forward-backward stochastic Volterra integral equations. Mathematical Control & Related Fields, 2015, 5 (3) : 613-649. doi: 10.3934/mcrf.2015.5.613 [6] Jiongmin Yong. A deterministic linear quadratic time-inconsistent optimal control problem. Mathematical Control & Related Fields, 2011, 1 (1) : 83-118. doi: 10.3934/mcrf.2011.1.83 [7] Xin Chen, Ana Bela Cruzeiro. Stochastic geodesics and forward-backward stochastic differential equations on Lie groups. Conference Publications, 2013, 2013 (special) : 115-121. doi: 10.3934/proc.2013.2013.115 [8] Jiaqin Wei. Time-inconsistent optimal control problems with regime-switching. Mathematical Control & Related Fields, 2017, 7 (4) : 585-622. doi: 10.3934/mcrf.2017022 [9] Joan-Andreu Lázaro-Camí, Juan-Pablo Ortega. The stochastic Hamilton-Jacobi equation. Journal of Geometric Mechanics, 2009, 1 (3) : 295-315. doi: 10.3934/jgm.2009.1.295 [10] Sebastián Ferrer, Martin Lara. Families of canonical transformations by Hamilton-Jacobi-Poincaré equation. Application to rotational and orbital motion. Journal of Geometric Mechanics, 2010, 2 (3) : 223-241. doi: 10.3934/jgm.2010.2.223 [11] Manuel de León, Juan Carlos Marrero, David Martín de Diego. Linear almost Poisson structures and Hamilton-Jacobi equation. Applications to nonholonomic mechanics. Journal of Geometric Mechanics, 2010, 2 (2) : 159-198. doi: 10.3934/jgm.2010.2.159 [12] Mohamed Assellaou, Olivier Bokanowski, Hasnaa Zidani. Error estimates for second order Hamilton-Jacobi-Bellman equations. Approximation of probabilistic reachable sets. Discrete & Continuous Dynamical Systems - A, 2015, 35 (9) : 3933-3964. doi: 10.3934/dcds.2015.35.3933 [13] Thierry Horsin, Peter I. Kogut. Optimal $L^2$-control problem in coefficients for a linear elliptic equation. I. Existence result. Mathematical Control & Related Fields, 2015, 5 (1) : 73-96. doi: 10.3934/mcrf.2015.5.73 [14] Thierry Horsin, Peter I. Kogut, Olivier Wilk. Optimal $L^2$-control problem in coefficients for a linear elliptic equation. II. Approximation of solutions and optimality conditions. Mathematical Control & Related Fields, 2016, 6 (4) : 595-628. doi: 10.3934/mcrf.2016017 [15] Dariusz Borkowski. Forward and backward filtering based on backward stochastic differential equations. Inverse Problems & Imaging, 2016, 10 (2) : 305-325. doi: 10.3934/ipi.2016002 [16] Nguyen Huy Chieu, Jen-Chih Yao. Subgradients of the optimal value function in a parametric discrete optimal control problem. Journal of Industrial & Management Optimization, 2010, 6 (2) : 401-410. doi: 10.3934/jimo.2010.6.401 [17] Jiongmin Yong. Forward-backward evolution equations and applications. Mathematical Control & Related Fields, 2016, 6 (4) : 653-704. doi: 10.3934/mcrf.2016019 [18] Juan Li, Wenqiang Li. Controlled reflected mean-field backward stochastic differential equations coupled with value function and related PDEs. Mathematical Control & Related Fields, 2015, 5 (3) : 501-516. doi: 10.3934/mcrf.2015.5.501 [19] Qian Zhao, Rongming Wang, Jiaqin Wei. Time-inconsistent consumption-investment problem for a member in a defined contribution pension plan. Journal of Industrial & Management Optimization, 2016, 12 (4) : 1557-1585. doi: 10.3934/jimo.2016.12.1557 [20] Nalini Anantharaman, Renato Iturriaga, Pablo Padilla, Héctor Sánchez-Morgado. Physical solutions of the Hamilton-Jacobi equation. Discrete & Continuous Dynamical Systems - B, 2005, 5 (3) : 513-528. doi: 10.3934/dcdsb.2005.5.513

2017 Impact Factor: 0.631