# American Institute of Mathematical Sciences

June  2012, 2(2): 195-215. doi: 10.3934/mcrf.2012.2.195

## A unified theory of maximum principle for continuous and discrete time optimal control problems

 1 Department of Mathematics, Zhejiang University, Zhejiang, Hangzhou, 310027, China, China 2 Department of Mathematics, Guizhou University, Guiyang, Guizhou, 550025, China

Received  February 2011 Revised  February 2012 Published  May 2012

Traditionally, the time domains that are widely used in mathematical descriptions are limited to real numbers for the case of continuous-time optimal control problems or to integers for the case of discrete-time optimal control problems. In this paper, based on a family of "needle variations", we derive maximum principle for optimal control problem on time scales. The results not only unify the theory of continuous and discrete optimal control problems but also conclude problems involving time domains in partly continuous and partly discrete ingredients. A simple optimal control problem on time scales is discussed in detail. Meanwhile, the results also unify the theory of some hybrid systems, for example, impulsive systems.
Citation: Zaidong Zhan, Shuping Chen, Wei Wei. A unified theory of maximum principle for continuous and discrete time optimal control problems. Mathematical Control & Related Fields, 2012, 2 (2) : 195-215. doi: 10.3934/mcrf.2012.2.195
 [1] R. Agarwal, V. O. Espinar, K. Perera and D. R. Vivero, Basic properties of Sobolev's space on time scales, Adv. Differential Equations, 2006, Art. ID 38121, 1-14.  Google Scholar [2] F. M. Atici, D. C. Biles and A. Lebedinsky, An application of time scales to economics, Math. Comput. Modelling., 43 (2006), 718-726. doi: 10.1016/j.mcm.2005.08.014.  Google Scholar [3] F. M. Atici and F. Uysal, A production-inventory model of HMMS on time scales, Appl. Math. Lett., 21 (2008), 236-243. doi: 10.1016/j.aml.2007.03.013.  Google Scholar [4] M. Bohner, Calculus of variations on time scales, J. Dynam. Systems Appl., 13 (2004), 339-349.  Google Scholar [5] M. Bohner and G. S. Gudeinov, Double integral calculus of variations of time scales, Comput. Math. Appl., 54 (2007), 45-57. doi: 10.1016/j.camwa.2006.10.032.  Google Scholar [6] M. Bohner and A. Peterson, "Dynamic Equations on Time Scales. An Introduction with Applications," Birkhäuser Boston, Inc., Boston, MA, 2001.  Google Scholar [7] A. G. Butkovskiĭ, The necessary and sufficient optimality conditions for of sampled-data control systems, (Russian), Avtomat. i Telemeh., 24 (1963), 1056-1064.  Google Scholar [8] A. Cabada and D. R. Vivero, Criterions for absolute continuity on time scales, J. Difference Equ. Appl., 11 (2005), 1013-1028. doi: 10.1080/10236190500272830.  Google Scholar [9] A. Cabada and D. R. Vivero, Expression of the Lebesgue $\Delta$-integral on time scales as a usual Lebesgue integral: Application to the calculus of $\Delta$-antiderivatives, Math. Comput. Modelling, 43 (2006), 194-207. doi: 10.1016/j.mcm.2005.09.028.  Google Scholar [10] I. Ekeland, Nonconvex minimization problems, Bull. Amer. Math. Soc. (N.S.), 1 (1979), 443-474.  Google Scholar [11] Rui A. C. Ferreira and D. F. M. Torres, Higher-order calculus of variations on time scales, in "Mathematical Control Theory and Finance," Springer, Berlin, (2008), 149-159.  Google Scholar [12] Rui. A. C. Ferreira and D. F. M. Torres, Isoperimetric problems of the calculus of variations on time scales, in "Nonlinear Analysis and Optimization II. Optimization," Contemp. Math., 514, Amer. Math. Soc., Providence, RI, (2010), 123-131.  Google Scholar [13] J. G. P. Gamarra and R. V. Solé, Complex discrete dynamics from simple continuous population models, Bull. Math. Biol., 64 (2002), 611-620. doi: 10.1006/bulm.2002.0286.  Google Scholar [14] H. Geering, Continuous-time optimal control theory for cost functionals including discrete state penalty terms, IEEE Trans. Automat. Control., AC-21 (1976), 866-869. doi: 10.1109/TAC.1976.1101377.  Google Scholar [15] H. Halkin, A maximum principle of the Pontryagin type for systems described by nonlinear difference equations, SIAM J. Control., 4 (1966), 90-111.  Google Scholar [16] S. Hilger, Analysis on measure chains-a unified approach to continues and discrete calculus, Results Math., 18 (1990), 18-56.  Google Scholar [17] R. Hilscher and V. Zeidan, Calculus of variations on time scales: Weak local piecewise $C^$$1_{rd} solutions with variable endpoints, J. Math. Anal. Appl., 289 (2004), 143-166. doi: 10.1016/j.jmaa.2003.09.031. Google Scholar [18] J. M. Holtzman, On the maximum principle for nonlinear discrete-time systems, IEEE Trans. Automat. Control, AC-11 (1966), 273-274. doi: 10.1109/TAC.1966.1098311. Google Scholar [19] J. M. Holtzman, Convexity and the maximum principle for discrete systems, IEEE Trans. Automat. Control, AC-11 (1966), 30-35. doi: 10.1109/TAC.1966.1098311. Google Scholar [20] F. Horn and R. Jackson, Correspondence discrete maximum principle, Ind. Eng. Chem. Fundamen., 4 (1965), 110-112. Google Scholar [21] R. M. May, Simple mathematical models with very complicated dynamics, Nature., 261 (1976), 459-467. doi: 10.1038/261459a0. Google Scholar [22] J. A. Oriega and R. J. Leake, Discrete maximum principle with state constrained control, SIAM J. Control. Optim., 15 (1977), 984-990. doi: 10.1137/0315063. Google Scholar [23] J. B. Pearson, Jr. and R. Sridhar, A discrete optimal control problem, IEEE Trans. Automat. Control, AC-11 (1966), 171-174. doi: 10.1109/TAC.1966.1098287. Google Scholar [24] A. I. Propoĭ, The maximum principle for discrete systems, Autom. Remote Control, 26 (1965), 1167-1177. Google Scholar [25] J. B. Rosen, Optimal control and convex programming, in "1967 Nonlinear Programming"(NATO Summer School, Menton, 1964) (ed. J. Abadie), North-Holland, Amsterdam, 287-302. Google Scholar [26] L. I. Rozonoèr, L.S. Pontryagin maximum principle of in the theory of optimum systems, Part III, Autom. Remote Control, 20 (1959), 1517-1532. Google Scholar [27] C. C. Tisdell and A. Zaidi, Basic qualitative and quantitative results for solutions to nonlinear, dynamic equations on time scales with an application to economic modelling, Nonlinear Anal., 68 (2008), 3504-3524. doi: 10.1016/j.na.2007.03.043. Google Scholar [28] J. M. Yong and H. W. Lou, "A Concise Course to Theory of Optimal Control," (in Chinese), Higher Education Press, Beijing, China, 2006. Google Scholar [29] Z. Zhan and W. Wei, Necessary conditions for optimal control problems on time scales, Abstr. Appl. Anal., 2009, Art. ID 974394, 14 pp. Google Scholar [30] Z. Zhan and W. Wei, On existence of optimal control governed by a class of the first-order linear dynamic systems on time scales, Appl. Math. Comput., 215 (2009), 2070-2081. doi: 10.1016/j.amc.2009.08.009. Google Scholar [31] Z. Zhan, W. Wei and Y. F. Li, Optimal control problem with terminal state constraint on time scales,, Pacific J. Optim., (). Google Scholar [32] Z. Zhan, W. Wei and H. Xu, Hamilton-Jacobi-Bellman equations on time scales, Math. Comput. Modelling, 49 (2009), 2019-2028. doi: 10.1016/j.mcm.2008.12.008. Google Scholar show all references ##### References:  [1] R. Agarwal, V. O. Espinar, K. Perera and D. R. Vivero, Basic properties of Sobolev's space on time scales, Adv. Differential Equations, 2006, Art. ID 38121, 1-14. Google Scholar [2] F. M. Atici, D. C. Biles and A. Lebedinsky, An application of time scales to economics, Math. Comput. Modelling., 43 (2006), 718-726. doi: 10.1016/j.mcm.2005.08.014. Google Scholar [3] F. M. Atici and F. Uysal, A production-inventory model of HMMS on time scales, Appl. Math. Lett., 21 (2008), 236-243. doi: 10.1016/j.aml.2007.03.013. Google Scholar [4] M. Bohner, Calculus of variations on time scales, J. Dynam. Systems Appl., 13 (2004), 339-349. Google Scholar [5] M. Bohner and G. S. Gudeinov, Double integral calculus of variations of time scales, Comput. Math. Appl., 54 (2007), 45-57. doi: 10.1016/j.camwa.2006.10.032. Google Scholar [6] M. Bohner and A. Peterson, "Dynamic Equations on Time Scales. An Introduction with Applications," Birkhäuser Boston, Inc., Boston, MA, 2001. Google Scholar [7] A. G. Butkovskiĭ, The necessary and sufficient optimality conditions for of sampled-data control systems, (Russian), Avtomat. i Telemeh., 24 (1963), 1056-1064. Google Scholar [8] A. Cabada and D. R. Vivero, Criterions for absolute continuity on time scales, J. Difference Equ. Appl., 11 (2005), 1013-1028. doi: 10.1080/10236190500272830. Google Scholar [9] A. Cabada and D. R. Vivero, Expression of the Lebesgue \Delta-integral on time scales as a usual Lebesgue integral: Application to the calculus of \Delta-antiderivatives, Math. Comput. Modelling, 43 (2006), 194-207. doi: 10.1016/j.mcm.2005.09.028. Google Scholar [10] I. Ekeland, Nonconvex minimization problems, Bull. Amer. Math. Soc. (N.S.), 1 (1979), 443-474. Google Scholar [11] Rui A. C. Ferreira and D. F. M. Torres, Higher-order calculus of variations on time scales, in "Mathematical Control Theory and Finance," Springer, Berlin, (2008), 149-159. Google Scholar [12] Rui. A. C. Ferreira and D. F. M. Torres, Isoperimetric problems of the calculus of variations on time scales, in "Nonlinear Analysis and Optimization II. Optimization," Contemp. Math., 514, Amer. Math. Soc., Providence, RI, (2010), 123-131. Google Scholar [13] J. G. P. Gamarra and R. V. Solé, Complex discrete dynamics from simple continuous population models, Bull. Math. Biol., 64 (2002), 611-620. doi: 10.1006/bulm.2002.0286. Google Scholar [14] H. Geering, Continuous-time optimal control theory for cost functionals including discrete state penalty terms, IEEE Trans. Automat. Control., AC-21 (1976), 866-869. doi: 10.1109/TAC.1976.1101377. Google Scholar [15] H. Halkin, A maximum principle of the Pontryagin type for systems described by nonlinear difference equations, SIAM J. Control., 4 (1966), 90-111. Google Scholar [16] S. Hilger, Analysis on measure chains-a unified approach to continues and discrete calculus, Results Math., 18 (1990), 18-56. Google Scholar [17] R. Hilscher and V. Zeidan, Calculus of variations on time scales: Weak local piecewise C^$$1_{rd}$ solutions with variable endpoints, J. Math. Anal. Appl., 289 (2004), 143-166. doi: 10.1016/j.jmaa.2003.09.031.  Google Scholar [18] J. M. Holtzman, On the maximum principle for nonlinear discrete-time systems, IEEE Trans. Automat. Control, AC-11 (1966), 273-274. doi: 10.1109/TAC.1966.1098311.  Google Scholar [19] J. M. Holtzman, Convexity and the maximum principle for discrete systems, IEEE Trans. Automat. Control, AC-11 (1966), 30-35. doi: 10.1109/TAC.1966.1098311.  Google Scholar [20] F. Horn and R. Jackson, Correspondence discrete maximum principle, Ind. Eng. Chem. Fundamen., 4 (1965), 110-112. Google Scholar [21] R. M. May, Simple mathematical models with very complicated dynamics, Nature., 261 (1976), 459-467. doi: 10.1038/261459a0.  Google Scholar [22] J. A. Oriega and R. J. Leake, Discrete maximum principle with state constrained control, SIAM J. Control. Optim., 15 (1977), 984-990. doi: 10.1137/0315063.  Google Scholar [23] J. B. Pearson, Jr. and R. Sridhar, A discrete optimal control problem, IEEE Trans. Automat. Control, AC-11 (1966), 171-174. doi: 10.1109/TAC.1966.1098287.  Google Scholar [24] A. I. Propoĭ, The maximum principle for discrete systems, Autom. Remote Control, 26 (1965), 1167-1177.  Google Scholar [25] J. B. Rosen, Optimal control and convex programming, in "1967 Nonlinear Programming"(NATO Summer School, Menton, 1964) (ed. J. Abadie), North-Holland, Amsterdam, 287-302.  Google Scholar [26] L. I. Rozonoèr, L.S. Pontryagin maximum principle of in the theory of optimum systems, Part III, Autom. Remote Control, 20 (1959), 1517-1532.  Google Scholar [27] C. C. Tisdell and A. Zaidi, Basic qualitative and quantitative results for solutions to nonlinear, dynamic equations on time scales with an application to economic modelling, Nonlinear Anal., 68 (2008), 3504-3524. doi: 10.1016/j.na.2007.03.043.  Google Scholar [28] J. M. Yong and H. W. Lou, "A Concise Course to Theory of Optimal Control," (in Chinese), Higher Education Press, Beijing, China, 2006. Google Scholar [29] Z. Zhan and W. Wei, Necessary conditions for optimal control problems on time scales, Abstr. Appl. Anal., 2009, Art. ID 974394, 14 pp.  Google Scholar [30] Z. Zhan and W. Wei, On existence of optimal control governed by a class of the first-order linear dynamic systems on time scales, Appl. Math. Comput., 215 (2009), 2070-2081. doi: 10.1016/j.amc.2009.08.009.  Google Scholar [31] Z. Zhan, W. Wei and Y. F. Li, Optimal control problem with terminal state constraint on time scales,, Pacific J. Optim., ().   Google Scholar [32] Z. Zhan, W. Wei and H. Xu, Hamilton-Jacobi-Bellman equations on time scales, Math. Comput. Modelling, 49 (2009), 2019-2028. doi: 10.1016/j.mcm.2008.12.008.  Google Scholar
 [1] Hans Josef Pesch. Carathéodory's royal road of the calculus of variations: Missed exits to the maximum principle of optimal control theory. Numerical Algebra, Control & Optimization, 2013, 3 (1) : 161-173. doi: 10.3934/naco.2013.3.161 [2] Zhen Wu, Feng Zhang. Maximum principle for discrete-time stochastic optimal control problem and stochastic game. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021031 [3] Md. Haider Ali Biswas, Maria do Rosário de Pinho. A nonsmooth maximum principle for optimal control problems with state and mixed constraints - convex case. Conference Publications, 2011, 2011 (Special) : 174-183. doi: 10.3934/proc.2011.2011.174 [4] Stefan Doboszczak, Manil T. Mohan, Sivaguru S. Sritharan. Pontryagin maximum principle for the optimal control of linearized compressible navier-stokes equations with state constraints. Evolution Equations & Control Theory, 2020  doi: 10.3934/eect.2020110 [5] H. O. Fattorini. The maximum principle for linear infinite dimensional control systems with state constraints. Discrete & Continuous Dynamical Systems, 1995, 1 (1) : 77-101. doi: 10.3934/dcds.1995.1.77 [6] Shaolin Ji, Xiaole Xue. A stochastic maximum principle for linear quadratic problem with nonconvex control domain. Mathematical Control & Related Fields, 2019, 9 (3) : 495-507. doi: 10.3934/mcrf.2019022 [7] Gastão S. F. Frederico, Delfim F. M. Torres. Noether's symmetry Theorem for variational and optimal control problems with time delay. Numerical Algebra, Control & Optimization, 2012, 2 (3) : 619-630. doi: 10.3934/naco.2012.2.619 [8] Huaiqiang Yu, Bin Liu. Pontryagin's principle for local solutions of optimal control governed by the 2D Navier-Stokes equations with mixed control-state constraints. Mathematical Control & Related Fields, 2012, 2 (1) : 61-80. doi: 10.3934/mcrf.2012.2.61 [9] Torsten Lindström. Discrete models and Fisher's maximum principle in ecology. Conference Publications, 2003, 2003 (Special) : 571-579. doi: 10.3934/proc.2003.2003.571 [10] Shi'an Wang, N. U. Ahmed. Optimal control and stabilization of building maintenance units based on minimum principle. Journal of Industrial & Management Optimization, 2021, 17 (4) : 1713-1727. doi: 10.3934/jimo.2020041 [11] Hancheng Guo, Jie Xiong. A second-order stochastic maximum principle for generalized mean-field singular control problem. Mathematical Control & Related Fields, 2018, 8 (2) : 451-473. doi: 10.3934/mcrf.2018018 [12] Guy Barles, Ariela Briani, Emmanuel Trélat. Value function for regional control problems via dynamic programming and Pontryagin maximum principle. Mathematical Control & Related Fields, 2018, 8 (3&4) : 509-533. doi: 10.3934/mcrf.2018021 [13] Yunkyong Hyon, Do Young Kwak, Chun Liu. Energetic variational approach in complex fluids: Maximum dissipation principle. Discrete & Continuous Dynamical Systems, 2010, 26 (4) : 1291-1304. doi: 10.3934/dcds.2010.26.1291 [14] Loïc Bourdin, Emmanuel Trélat. Optimal sampled-data control, and generalizations on time scales. Mathematical Control & Related Fields, 2016, 6 (1) : 53-94. doi: 10.3934/mcrf.2016.6.53 [15] H. O. Fattorini. The maximum principle in infinite dimension. Discrete & Continuous Dynamical Systems, 2000, 6 (3) : 557-574. doi: 10.3934/dcds.2000.6.557 [16] Yan Wang, Yanxiang Zhao, Lei Wang, Aimin Song, Yanping Ma. Stochastic maximum principle for partial information optimal investment and dividend problem of an insurer. Journal of Industrial & Management Optimization, 2018, 14 (2) : 653-671. doi: 10.3934/jimo.2017067 [17] Shanjian Tang. A second-order maximum principle for singular optimal stochastic controls. Discrete & Continuous Dynamical Systems - B, 2010, 14 (4) : 1581-1599. doi: 10.3934/dcdsb.2010.14.1581 [18] Adel Chala, Dahbia Hafayed. On stochastic maximum principle for risk-sensitive of fully coupled forward-backward stochastic control of mean-field type with application. Evolution Equations & Control Theory, 2020, 9 (3) : 817-843. doi: 10.3934/eect.2020035 [19] C.Z. Wu, K. L. Teo. Global impulsive optimal control computation. Journal of Industrial & Management Optimization, 2006, 2 (4) : 435-450. doi: 10.3934/jimo.2006.2.435 [20] Dingjun Yao, Kun Fan. Optimal risk control and dividend strategies in the presence of two reinsurers: Variance premium principle. Journal of Industrial & Management Optimization, 2018, 14 (3) : 1055-1083. doi: 10.3934/jimo.2017090

2020 Impact Factor: 1.284