Strict dissipativity for discrete time discounted optimal control problems

Grüne Lars; Müller Matthias A.; Kellett Christopher M.; Weller Steven R.

doi:10.3934/mcrf.2020046

Article Contents

2021, Volume 11, Issue 4: 771-796. Doi: 10.3934/mcrf.2020046 open access

This issue Previous Article First order necessary conditions of optimality for the two dimensional tidal dynamics system Next Article A stackelberg game of backward stochastic differential equations with partial information

Strict dissipativity for discrete time discounted optimal control problems

1.
Mathematisches Institut, Universität Bayreuth, 95440 Bayreuth, Germany
2.
Institute of Automatic Control, Leibniz University Hannover, 30167 Hannover, Germany
3.
Research School of Electrical, Energy and Materials Engineering, Australian National University, Canberra, ACT 2600, Australia
4.
School of Electrical Engineering and Computing, University of Newcastle, Callaghan, NSW 2308, Australia

^* Corresponding author: Lars Grüne
^* Corresponding author: Lars Grüne

Received: September 2019

Revised: May 2020

Early access: November 2020

Published: December 2021

The research was supported by the Australian Research Council under grants DP160102138 and DP180103026 and by the Deutsche Forschungsgemeinschaft under grant Gr1569/13-2.

Abstract / Introduction Full Text(HTML) Figure(1) Related Papers Cited by

Abstract

The paradigm of discounting future costs is a common feature of economic applications of optimal control. In this paper, we provide several results for such discounted optimal control aimed at replicating the now well-known results in the standard, undiscounted, setting whereby (strict) dissipativity, turnpike properties, and near-optimality of closed-loop systems using model predictive control are essentially equivalent. To that end, we introduce a notion of discounted strict dissipativity and show that this implies various properties including the existence of available storage functions, required supply functions, and robustness of optimal equilibria. Additionally, for discount factors sufficiently close to one we demonstrate that strict dissipativity implies discounted strict dissipativity and that optimally controlled systems, derived from a discounted cost function, yield practically asymptotically stable equilibria. Several examples are provided throughout.

Keywords:

Mathematics Subject Classification: Primary: 49J21, 93C55; Secondary: 93D20.

Citation:

FullText(HTML)

Figure 1. Illustration of the steady-states (blue solid line), level sets of $ \ell $ (black ellipses), and the additional constraint $ g_{ad} $ (red dashed) of the example in Section 8.4. The optimal steady-state $ (x^e,u^e)=(0,0) $ for the undiscounted case is marked with a circle

Download: Full-size image PowerPoint slide

Related Papers

Cited by

References

[1]	D. Acemoglu, Introduction to Modern Economic Growth, Princeton University Press, 2009.
[2]	D. Angeli, R. Amrit and J. B. Rawlings, On average performance and stability of economic model predictive control, IEEE Trans. Automat. Control, 57 (2012), 1615-1626. doi: 10.1109/TAC.2011.2179349.
[3]	M. Bardi and I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations, Birkhäuser Boston, Inc., Boston, MA, 1997. doi: 10.1007/978-0-8176-4755-1.
[4]	S. Becker, L. Grüne and W. Semmler, Comparing accuracy of second order approximation and dynamic programming, Comput. Econ., 30 (2007), 65-91. doi: 10.1007/s10614-007-9087-1.
[5]	D. P. Bertsekas, Nonlinear Programming, 2nd edition, Athena Scientific, Belmont, Massachusetts, 1995.
[6]	J. Blot and N. Hayek, Infinite-Horizon Optimal Control in the Discrete-Time Framework, SpringerBriefs in Optimization, Springer, New York, 2014. doi: 10.1007/978-1-4614-9038-8.
[7]	W. A. Brock and L. Mirman, Optimal economic growth and uncertainty: The discounted case, J. Econom. Theory, 4 (1972), 479-513. doi: 10.1016/0022-0531(72)90135-4.
[8]	B. Brogliato, R. Lozano, B. Maschke and O. Egeland, Dissipative Systems Analysis and Control, Second edition, Springer-Verlag, London, Ltd., 2007. doi: 10.1007/978-1-84628-517-2.
[9]	C. I. Byrnes and W. Lin, Losslessness, feedback equivalence, and the global stabilization of discrete-time nonlinear systems, IEEE Trans. Automat. Control, 39 (1994), 83-98. doi: 10.1109/9.273341.
[10]	D. Cass, Optimum growth in an aggregative model of capital accumulation, Rev. Econ. Stud., 32 (1965), 233-240. doi: 10.2307/2295827.
[11]	T. Damm, L. Grüne, M. Stieler and K. Worthmann, An exponential turnpike theorem for dissipative discrete time optimal control problems, SIAM J. Control Optim., 52 (2014), 1935-1957. doi: 10.1137/120888934.
[12]	M. Diehl, R. Amrit and J. B. Rawlings, A Lyapunov function for economic optimizing model predictive control, IEEE Trans. Automat. Control, 56 (2011), 703-707. doi: 10.1109/TAC.2010.2101291.
[13]	T. Faulwasser, C. M. Kellett and S. R. Weller, MPC-DICE: An open-source Matlab implementation of receding horizon solutions to DICE, in Proc. 1st IFAC Workshop on Integrated Assessment Modelling for Environmental Systems, Brescia, Italy, 2018,126–131.
[14]	T. Faulwasser, M. Korda, C. N. Jones and D. Bonvin, On turnpike and dissipativity properties of continuous-time optimal control problems, Automatica J. IFAC, 81 (2017), 297-304. doi: 10.1016/j.automatica.2017.03.012.
[15]	V. Gaitsgory, L. Grüne, M. Höger, C. M. Kellett and S. R. Weller, Stabilization of strictly dissipative discrete time systems with discounted optimal control, Automatica J. IFAC, 93 (2018), 311-320. doi: 10.1016/j.automatica.2018.03.076.
[16]	V. Gaitsgory, L. Grüne and N. Thatcher, Stabilization with discounted optimal control, Systems Control Lett., 82 (2015), 91-98. doi: 10.1016/j.sysconle.2015.05.010.
[17]	L. Grüne, Economic receding horizon control without terminal constraints, Automatica J. IFAC, 49 (2013), 725-734. doi: 10.1016/j.automatica.2012.12.003.
[18]	L. Grüne, C. M. Kellett and S. R. Weller, On a discounted notion of strict dissipativity, in Proc. 10th IFAC Symposium on Nonlinear Control Systems, 2016,247–252.
[19]	L. Grüne, C. M. Kellett and S. R. Weller, On the relation between turnpike properties for finite and infinite horizon optimal control problems, J. Optim. Theory Appl., 173 (2017), 727-745. doi: 10.1007/s10957-017-1103-6.
[20]	L. Grüne and M. A. Müller, On the relation between strict dissipativity and turnpike properties, Systems Control Lett., 90 (2016), 45-53. doi: 10.1016/j.sysconle.2016.01.003.
[21]	L. Grüne and A. Panin, On non-averaged performance of economic MPC with terminal conditions, in Proc. 54th IEEE Conf. Decis. Control, Osaka, Japan, 2015, 4332–4337.
[22]	L. Grüne and J. Pannek, Nonlinear Model Predictive Control. Theory and Algorithms, 2nd edition, Springer, Cham, 2017. doi: 10.1007/978-3-319-46024-6.
[23]	L. Grüne and M. Stieler, Asymptotic stability and transient optimality of economic MPC without terminal conditions, J. Proc. Control, 24 (2014), 1187-1196.
[24]	L. Grüne, W. Semmler and M. Stieler, Using nonlinear model predictive control for dynamic decision problems in economics, J. Econom. Dynam. Control, 60 (2015), 112-133. doi: 10.1016/j.jedc.2015.08.010.
[25]	Interagency Working Group on Social Cost of Carbon, United States Government, Technical Support Document: Technical Update of the Social Cost of Carbon for Regulatory Impact Analysis - Under Executive Order 12866, Technical report, 2013.
[26]	C. M. Kellett, S. R. Weller, T. Faulwasser, L. Grüne and W. Semmler, Feedback, dynamics, and optimal control in climate economics, Annu. Rev. Control, 47 (2019), 7-20. doi: 10.1016/j.arcontrol.2019.04.003.
[27]	T. C. Koopmans, On the concept of optimal economic growth, in The Economic Approach to Development Planning, Rand McNally, Chicago, IL, 1965,225–287.
[28]	R. Lopezlena and J. M. A. Scherpen, Energy functions for dissipativity-based balancing of discrete-time nonlinear systems, Math. Control Signals Systems, 18 (2006), 345-368. doi: 10.1007/s00498-006-0007-z.
[29]	P. Moylan, Dissipative Systems and Stability [Online], 2014.
[30]	M. A. Müller, D. Angeli and F. Allgöwer, On necessity and robustness of dissipativity in economic model predictive control, IEEE Trans. Automat. Control, 60 (2015), 1671-1676. doi: 10.1109/TAC.2014.2361193.
[31]	M. A. Müller and L. Grüne, On the relation between dissipativity and discounted dissipativity, in Proc. 56th IEEE Conf. Decis. Control, Melbourne, Australia, 2017, 5570–5575.
[32]	W. D. Nordhaus, An optimal transition path for controlling greenhouse gases, Science, 258 (1992), 1315-1319. doi: 10.1126/science.258.5086.1315.
[33]	W. D. Nordhaus, Revisiting the social cost of carbon, Proc. Natl. Acad. Scie. USA (PNAS), 114 (2017), 1518-1523. doi: 10.1073/pnas.1609244114.
[34]	R. Postoyan, L. Buşoniu, D. Nešić and J. Daafouz, Stability of infinite-horizon optimal control with discounted cost, in Proc. 53rd IEEE Conf. Decis. Control doi: 10.1109/CDC.2014.7039995.
[35]	R. Postoyan, L. Busoniu, D. Nešić and J. Daafouz, Stability analysis of discrete-time infinite-horizon optimal control with discounted cost, IEEE Trans. Automat. Control, 62 (2017), 2736-2749. doi: 10.1109/TAC.2016.2616644.
[36]	F. P. Ramsey, A Mathematical Theory of Saving, Econ. J., 38 (1928), 543-559. doi: 10.2307/2224098.
[37]	M. S. Santos and J. Vigo-Aguiar, Analysis of a numerical dynamic programming algorithm applied to economic models, Econometrica, 66 (1998), 409-426. doi: 10.2307/2998564.
[38]	A. Seierstad and K. Sydsæter, Optimal Control Theory with Economic Applications, North-Holland, Amsterdam, 1987.
[39]	N. Stern, The Economics of Climate Change: The Stern Review, Cambridge University Press, 2007. doi: 10.1017/CBO9780511817434.
[40]	E. Trélat and C. Zhang, Integral and measure-turnpike properties for infinite-dimensional optimal control systems, Math. Control Signals Systems, 30 (2018), Art. 3, 34 pp. doi: 10.1007/s00498-018-0209-1.
[41]	A. van der Schaft, $L_2$-Gain and Passivity Techniques in Nonlinear Control, Lecture Notes in Control and Information Sciences, vol. 218, Springer-Verlag London, Ltd., London, 1996. doi: 10.1007/3-540-76074-1.
[42]	S. R. Weller, S. Hafeez and C. M. Kellett, A receding horizon control approach to estimating the social cost of carbon in the presence of emissions and temperature uncertainty, in Proc. 54th IEEE Conf. Decis. Control, Osaka, Japan, 2015, 5384–5390. doi: 10.1109/CDC.2015.7403062.
[43]	J. C. Willems, Dissipative dynamical systems. I. General theory, Arch. Rational Mech. Anal., 45 (1972), 321-351. doi: 10.1007/BF00276493.