January  2014, 1(1): 57-78. doi: 10.3934/jdg.2014.1.57

On the Euler equation approach to discrete--time nonstationary optimal control problems

1. 

Departamento de Matemáticas, Instituto Tecnológico Autónomo de México (ITAM), Río Hondo 1, México D.F. 01000, Mexico

2. 

Mathematics Department, CINVESTAV-IPN, A. Postal 14-740, México D.F. 07000, Mexico

Received  April 2012 Revised  March 2013 Published  June 2013

We are concerned with deterministic and stochastic nonstationary discrete--time optimal control problems in infinite horizon. We show, using Gâteaux differentials, that the so--called Euler equation and a transversality condition are necessary conditions for optimality. In particular, the transversality condition is obtained in a more general form and under milder hypotheses than in previous works. Sufficient conditions are also provided. We also find closed--form solutions to several (discounted) stationary and nonstationary control problems.
Citation: David González-Sánchez, Onésimo Hernández-Lerma. On the Euler equation approach to discrete--time nonstationary optimal control problems. Journal of Dynamics & Games, 2014, 1 (1) : 57-78. doi: 10.3934/jdg.2014.1.57
References:
[1]

D. Acemoglu, "Introduction to Modern Economic Growth," Princeton University Press, Princeton, 2009. Google Scholar

[2]

J. Adda and R. Cooper, "Dynamic Economics. Quantitative Methods and Applications," MIT Press, Cambridge, MA, 2003. Google Scholar

[3]

V. I. Arkin and I. V. Evstigneev, "Stochastic Models of Control and Economic Dynamics," Academic Press, Orlando, FL, 1987. Google Scholar

[4]

Y. Bar-Ness, The discrete Euler equation on the normed linear space $l_n^1$, Int. J. Control, 21 (1975), 625-640. doi: 10.1080/00207177508922017.  Google Scholar

[5]

W. A. Brock and L. Mirman, Optimal economic growth and uncertainty: The discounted case, J. Econ. Theory, 4 (1972), 479-513. doi: 10.1016/0022-0531(72)90135-4.  Google Scholar

[6]

J. A. Cadzow, Discrete calculus of variations, Int. J. Control, 11 (1970), 393-407. doi: 10.1080/00207177008905922.  Google Scholar

[7]

G. C. Chow, "Dynamic Economics: Optimization by the Lagrange Method," Oxford University Press, New York, 1997. Google Scholar

[8]

I. Ekeland and J. A. Scheinkman, Transversality conditions for some infinite horizon discrete time optimization problems, Math. Oper. Res., 11 (1986), 216-229. doi: 10.1287/moor.11.2.216.  Google Scholar

[9]

S. Elaydi, "An Introduction to Difference Equations," Third edition, Undergraduate Texts in Mathematics, Springer, New York, 2005.  Google Scholar

[10]

J. Engwerda, "LQ Dynamic Optimization and Differential Games," John Wiley & Sons, Chichester, 2005. Google Scholar

[11]

S. Flåm and A. Fougères, Infinite horizon programs; Convergence of approximate solutions, Ann. Oper. Res., 29 (1991), 333-350. doi: 10.1007/BF02283604.  Google Scholar

[12]

W. H. Fleming and R. W. Rishel, "Deterministic and Stochastic Optimal Control," Applications of Mathematics, No. 1, Springer-Verlag, Berlin-New York, 1975.  Google Scholar

[13]

X. Guo, A. Hernández-del-Valle and O. Hernández-Lerma, Nonstationary discrete-time deterministic and stochastic control systems: Bounded and unbounded cases, Systems Control Lett., 60 (2011), 503-509. doi: 10.1016/j.sysconle.2011.04.006.  Google Scholar

[14]

O. Hernández-Lerma and J. B. Lasserre, "Discrete-Time Markov Control Processes: Basic Optimality Criteria," Applications of Mathematics (New York), 30, Springer-Verlag, New York, 1996.  Google Scholar

[15]

T. Kamihigashi, A simple proof of the necessity of the transversality condition, Econ. Theory, 20 (2002), 427-433. doi: 10.1007/s001990100198.  Google Scholar

[16]

T. Kamihigashi, Transversality conditions and dynamic economic behaviour, in "The New Palgrave Dictionary of Economics" (eds. S. N. Durlauf and L. E. Blume), Second edition, Palgrave Macmillan, Hampshire, (2008), 384-387. doi: 10.1057/9780230226203.1737.  Google Scholar

[17]

W. G. Kelley and A. C. Peterson, "Difference Equations. An Introduction with Applications," Academic Press, Inc., Boston, MA, 1991.  Google Scholar

[18]

C. Le Van and R.-A. Dana, "Dynamic Programming in Economics," Dynamic Modeling and Econometrics in Economics and Finance, 5, Kluwer Academic Publishers, Dordrecht, 2003.  Google Scholar

[19]

D. Levhari and L. D. Mirman, The great fish war: An example using dynamic Cournot-Nash solution, Bell J. Econom., 11 (1980), 322-334. doi: 10.2307/3003416.  Google Scholar

[20]

L. Ljungqvist and T. J. Sargent, "Recursive Macroeconomic Theory," Second edition, MIT Press, Cambridge, MA, 2004. Google Scholar

[21]

D. G. Luenberger, "Optimization by Vector Space Methods," John Wiley & Sons, Inc., New York-London-Sydney, 1969.  Google Scholar

[22]

K. Okuguchi, A dynamic Cournot-Nash equilibrium in fishery: The effects of entry, Riv. Mat. Sci. Econom. Social., 4 (1981), 59-64. doi: 10.1007/BF02123580.  Google Scholar

[23]

W. Rudin, "Principles of Mathematical Analysis," Third edition, McGraw-Hill Book Co., New York-Auckland-Düsseldorf, 1976.  Google Scholar

[24]

I. Schochetman and R. L. Smith, Finite dimensional approximation in infinite-dimensional mathematical programming, Math. Programming, 54 (1992), 307-333. doi: 10.1007/BF01586057.  Google Scholar

[25]

N. L. Stokey, R. E. Lucas and E. C. Prescott, Jr., "Recursive Methods in Economic Dynamics," With the collaboration of Edward C. Prescott, Harvard University Press, Cambridge, MA, 1989.  Google Scholar

[26]

K. Sydsæter, P. J. Hammond, A. Seierstad and A. Strøm, "Further Mathematics for Economic Analysis," Second edition, Prentice-Hall, New York, 2008. Google Scholar

show all references

References:
[1]

D. Acemoglu, "Introduction to Modern Economic Growth," Princeton University Press, Princeton, 2009. Google Scholar

[2]

J. Adda and R. Cooper, "Dynamic Economics. Quantitative Methods and Applications," MIT Press, Cambridge, MA, 2003. Google Scholar

[3]

V. I. Arkin and I. V. Evstigneev, "Stochastic Models of Control and Economic Dynamics," Academic Press, Orlando, FL, 1987. Google Scholar

[4]

Y. Bar-Ness, The discrete Euler equation on the normed linear space $l_n^1$, Int. J. Control, 21 (1975), 625-640. doi: 10.1080/00207177508922017.  Google Scholar

[5]

W. A. Brock and L. Mirman, Optimal economic growth and uncertainty: The discounted case, J. Econ. Theory, 4 (1972), 479-513. doi: 10.1016/0022-0531(72)90135-4.  Google Scholar

[6]

J. A. Cadzow, Discrete calculus of variations, Int. J. Control, 11 (1970), 393-407. doi: 10.1080/00207177008905922.  Google Scholar

[7]

G. C. Chow, "Dynamic Economics: Optimization by the Lagrange Method," Oxford University Press, New York, 1997. Google Scholar

[8]

I. Ekeland and J. A. Scheinkman, Transversality conditions for some infinite horizon discrete time optimization problems, Math. Oper. Res., 11 (1986), 216-229. doi: 10.1287/moor.11.2.216.  Google Scholar

[9]

S. Elaydi, "An Introduction to Difference Equations," Third edition, Undergraduate Texts in Mathematics, Springer, New York, 2005.  Google Scholar

[10]

J. Engwerda, "LQ Dynamic Optimization and Differential Games," John Wiley & Sons, Chichester, 2005. Google Scholar

[11]

S. Flåm and A. Fougères, Infinite horizon programs; Convergence of approximate solutions, Ann. Oper. Res., 29 (1991), 333-350. doi: 10.1007/BF02283604.  Google Scholar

[12]

W. H. Fleming and R. W. Rishel, "Deterministic and Stochastic Optimal Control," Applications of Mathematics, No. 1, Springer-Verlag, Berlin-New York, 1975.  Google Scholar

[13]

X. Guo, A. Hernández-del-Valle and O. Hernández-Lerma, Nonstationary discrete-time deterministic and stochastic control systems: Bounded and unbounded cases, Systems Control Lett., 60 (2011), 503-509. doi: 10.1016/j.sysconle.2011.04.006.  Google Scholar

[14]

O. Hernández-Lerma and J. B. Lasserre, "Discrete-Time Markov Control Processes: Basic Optimality Criteria," Applications of Mathematics (New York), 30, Springer-Verlag, New York, 1996.  Google Scholar

[15]

T. Kamihigashi, A simple proof of the necessity of the transversality condition, Econ. Theory, 20 (2002), 427-433. doi: 10.1007/s001990100198.  Google Scholar

[16]

T. Kamihigashi, Transversality conditions and dynamic economic behaviour, in "The New Palgrave Dictionary of Economics" (eds. S. N. Durlauf and L. E. Blume), Second edition, Palgrave Macmillan, Hampshire, (2008), 384-387. doi: 10.1057/9780230226203.1737.  Google Scholar

[17]

W. G. Kelley and A. C. Peterson, "Difference Equations. An Introduction with Applications," Academic Press, Inc., Boston, MA, 1991.  Google Scholar

[18]

C. Le Van and R.-A. Dana, "Dynamic Programming in Economics," Dynamic Modeling and Econometrics in Economics and Finance, 5, Kluwer Academic Publishers, Dordrecht, 2003.  Google Scholar

[19]

D. Levhari and L. D. Mirman, The great fish war: An example using dynamic Cournot-Nash solution, Bell J. Econom., 11 (1980), 322-334. doi: 10.2307/3003416.  Google Scholar

[20]

L. Ljungqvist and T. J. Sargent, "Recursive Macroeconomic Theory," Second edition, MIT Press, Cambridge, MA, 2004. Google Scholar

[21]

D. G. Luenberger, "Optimization by Vector Space Methods," John Wiley & Sons, Inc., New York-London-Sydney, 1969.  Google Scholar

[22]

K. Okuguchi, A dynamic Cournot-Nash equilibrium in fishery: The effects of entry, Riv. Mat. Sci. Econom. Social., 4 (1981), 59-64. doi: 10.1007/BF02123580.  Google Scholar

[23]

W. Rudin, "Principles of Mathematical Analysis," Third edition, McGraw-Hill Book Co., New York-Auckland-Düsseldorf, 1976.  Google Scholar

[24]

I. Schochetman and R. L. Smith, Finite dimensional approximation in infinite-dimensional mathematical programming, Math. Programming, 54 (1992), 307-333. doi: 10.1007/BF01586057.  Google Scholar

[25]

N. L. Stokey, R. E. Lucas and E. C. Prescott, Jr., "Recursive Methods in Economic Dynamics," With the collaboration of Edward C. Prescott, Harvard University Press, Cambridge, MA, 1989.  Google Scholar

[26]

K. Sydsæter, P. J. Hammond, A. Seierstad and A. Strøm, "Further Mathematics for Economic Analysis," Second edition, Prentice-Hall, New York, 2008. Google Scholar

[1]

Le Viet Cuong, Thai Son Doan. Assignability of dichotomy spectra for discrete time-varying linear control systems. Discrete & Continuous Dynamical Systems - B, 2020, 25 (9) : 3597-3607. doi: 10.3934/dcdsb.2020074

[2]

Senda Ounaies, Jean-Marc Bonnisseau, Souhail Chebbi, Halil Mete Soner. Merton problem in an infinite horizon and a discrete time with frictions. Journal of Industrial & Management Optimization, 2016, 12 (4) : 1323-1331. doi: 10.3934/jimo.2016.12.1323

[3]

Hamid Maarouf. Local Kalman rank condition for linear time varying systems. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021029

[4]

Vladimir Gaitsgory, Alex Parkinson, Ilya Shvartsman. Linear programming based optimality conditions and approximate solution of a deterministic infinite horizon discounted optimal control problem in discrete time. Discrete & Continuous Dynamical Systems - B, 2019, 24 (4) : 1743-1767. doi: 10.3934/dcdsb.2018235

[5]

Vladimir Gaitsgory, Alex Parkinson, Ilya Shvartsman. Linear programming formulations of deterministic infinite horizon optimal control problems in discrete time. Discrete & Continuous Dynamical Systems - B, 2017, 22 (10) : 3821-3838. doi: 10.3934/dcdsb.2017192

[6]

Jingang Zhao, Chi Zhang. Finite-horizon optimal control of discrete-time linear systems with completely unknown dynamics using Q-learning. Journal of Industrial & Management Optimization, 2021, 17 (3) : 1471-1483. doi: 10.3934/jimo.2020030

[7]

Yi Wang, Dun Zhou. Transversality for time-periodic competitive-cooperative tridiagonal systems. Discrete & Continuous Dynamical Systems - B, 2015, 20 (6) : 1821-1830. doi: 10.3934/dcdsb.2015.20.1821

[8]

Xiaoqiang Dai, Chao Yang, Shaobin Huang, Tao Yu, Yuanran Zhu. Finite time blow-up for a wave equation with dynamic boundary condition at critical and high energy levels in control systems. Electronic Research Archive, 2020, 28 (1) : 91-102. doi: 10.3934/era.2020006

[9]

Elimhan N. Mahmudov. Second order discrete time-varying and time-invariant linear continuous systems and Kalman type conditions. Numerical Algebra, Control & Optimization, 2021  doi: 10.3934/naco.2021010

[10]

Fabio Bagagiolo. An infinite horizon optimal control problem for some switching systems. Discrete & Continuous Dynamical Systems - B, 2001, 1 (4) : 443-462. doi: 10.3934/dcdsb.2001.1.443

[11]

Hongbiao Fan, Jun-E Feng, Min Meng. Piecewise observers of rectangular discrete fuzzy descriptor systems with multiple time-varying delays. Journal of Industrial & Management Optimization, 2016, 12 (4) : 1535-1556. doi: 10.3934/jimo.2016.12.1535

[12]

Haijun Sun, Xinquan Zhang. Guaranteed cost control of discrete-time switched saturated systems. Discrete & Continuous Dynamical Systems - B, 2021, 26 (8) : 4515-4522. doi: 10.3934/dcdsb.2020300

[13]

Tao Pang, Azmat Hussain. An infinite time horizon portfolio optimization model with delays. Mathematical Control & Related Fields, 2016, 6 (4) : 629-651. doi: 10.3934/mcrf.2016018

[14]

K. Aruna Sakthi, A. Vinodkumar. Stabilization on input time-varying delay for linear switched systems with truncated predictor control. Numerical Algebra, Control & Optimization, 2020, 10 (2) : 237-247. doi: 10.3934/naco.2019050

[15]

Chuandong Li, Fali Ma, Tingwen Huang. 2-D analysis based iterative learning control for linear discrete-time systems with time delay. Journal of Industrial & Management Optimization, 2011, 7 (1) : 175-181. doi: 10.3934/jimo.2011.7.175

[16]

Ran Dong, Xuerong Mao. Asymptotic stabilization of continuous-time periodic stochastic systems by feedback control based on periodic discrete-time observations. Mathematical Control & Related Fields, 2020, 10 (4) : 715-734. doi: 10.3934/mcrf.2020017

[17]

Valery Y. Glizer, Oleg Kelis. Asymptotic properties of an infinite horizon partial cheap control problem for linear systems with known disturbances. Numerical Algebra, Control & Optimization, 2018, 8 (2) : 211-235. doi: 10.3934/naco.2018013

[18]

Tingwen Huang, Guanrong Chen, Juergen Kurths. Synchronization of chaotic systems with time-varying coupling delays. Discrete & Continuous Dynamical Systems - B, 2011, 16 (4) : 1071-1082. doi: 10.3934/dcdsb.2011.16.1071

[19]

Karl P. Hadeler. Quiescent phases and stability in discrete time dynamical systems. Discrete & Continuous Dynamical Systems - B, 2015, 20 (1) : 129-152. doi: 10.3934/dcdsb.2015.20.129

[20]

Piotr Oprocha. Chain recurrence in multidimensional time discrete dynamical systems. Discrete & Continuous Dynamical Systems, 2008, 20 (4) : 1039-1056. doi: 10.3934/dcds.2008.20.1039

 Impact Factor: 

Metrics

  • PDF downloads (169)
  • HTML views (0)
  • Cited by (4)

[Back to Top]