2012, 2(2): 195-215. doi: 10.3934/mcrf.2012.2.195

A unified theory of maximum principle for continuous and discrete time optimal control problems

1. 

Department of Mathematics, Zhejiang University, Zhejiang, Hangzhou, 310027, China, China

2. 

Department of Mathematics, Guizhou University, Guiyang, Guizhou, 550025, China

Received  February 2011 Revised  February 2012 Published  May 2012

Traditionally, the time domains that are widely used in mathematical descriptions are limited to real numbers for the case of continuous-time optimal control problems or to integers for the case of discrete-time optimal control problems. In this paper, based on a family of "needle variations", we derive maximum principle for optimal control problem on time scales. The results not only unify the theory of continuous and discrete optimal control problems but also conclude problems involving time domains in partly continuous and partly discrete ingredients. A simple optimal control problem on time scales is discussed in detail. Meanwhile, the results also unify the theory of some hybrid systems, for example, impulsive systems.
Citation: Zaidong Zhan, Shuping Chen, Wei Wei. A unified theory of maximum principle for continuous and discrete time optimal control problems. Mathematical Control & Related Fields, 2012, 2 (2) : 195-215. doi: 10.3934/mcrf.2012.2.195
References:
[1]

R. Agarwal, V. O. Espinar, K. Perera and D. R. Vivero, Basic properties of Sobolev's space on time scales,, Adv. Differential Equations, 2006 (3812), 1.

[2]

F. M. Atici, D. C. Biles and A. Lebedinsky, An application of time scales to economics,, Math. Comput. Modelling., 43 (2006), 718. doi: 10.1016/j.mcm.2005.08.014.

[3]

F. M. Atici and F. Uysal, A production-inventory model of HMMS on time scales,, Appl. Math. Lett., 21 (2008), 236. doi: 10.1016/j.aml.2007.03.013.

[4]

M. Bohner, Calculus of variations on time scales,, J. Dynam. Systems Appl., 13 (2004), 339.

[5]

M. Bohner and G. S. Gudeinov, Double integral calculus of variations of time scales,, Comput. Math. Appl., 54 (2007), 45. doi: 10.1016/j.camwa.2006.10.032.

[6]

M. Bohner and A. Peterson, "Dynamic Equations on Time Scales. An Introduction with Applications,", Birkhäuser Boston, (2001).

[7]

A. G. Butkovskiĭ, The necessary and sufficient optimality conditions for of sampled-data control systems,, (Russian), 24 (1963), 1056.

[8]

A. Cabada and D. R. Vivero, Criterions for absolute continuity on time scales,, J. Difference Equ. Appl., 11 (2005), 1013. doi: 10.1080/10236190500272830.

[9]

A. Cabada and D. R. Vivero, Expression of the Lebesgue $\Delta$-integral on time scales as a usual Lebesgue integral: Application to the calculus of $\Delta$-antiderivatives,, Math. Comput. Modelling, 43 (2006), 194. doi: 10.1016/j.mcm.2005.09.028.

[10]

I. Ekeland, Nonconvex minimization problems,, Bull. Amer. Math. Soc. (N.S.), 1 (1979), 443.

[11]

Rui A. C. Ferreira and D. F. M. Torres, Higher-order calculus of variations on time scales, in "Mathematical Control Theory and Finance,", Springer, (2008), 149.

[12]

Rui. A. C. Ferreira and D. F. M. Torres, Isoperimetric problems of the calculus of variations on time scales,, in, 514 (2010), 123.

[13]

J. G. P. Gamarra and R. V. Solé, Complex discrete dynamics from simple continuous population models,, Bull. Math. Biol., 64 (2002), 611. doi: 10.1006/bulm.2002.0286.

[14]

H. Geering, Continuous-time optimal control theory for cost functionals including discrete state penalty terms,, IEEE Trans. Automat. Control., AC-21 (1976), 866. doi: 10.1109/TAC.1976.1101377.

[15]

H. Halkin, A maximum principle of the Pontryagin type for systems described by nonlinear difference equations,, SIAM J. Control., 4 (1966), 90.

[16]

S. Hilger, Analysis on measure chains-a unified approach to continues and discrete calculus,, Results Math., 18 (1990), 18.

[17]

R. Hilscher and V. Zeidan, Calculus of variations on time scales: Weak local piecewise $C^$$1_{rd}$ solutions with variable endpoints,, J. Math. Anal. Appl., 289 (2004), 143. doi: 10.1016/j.jmaa.2003.09.031.

[18]

J. M. Holtzman, On the maximum principle for nonlinear discrete-time systems,, IEEE Trans. Automat. Control, AC-11 (1966), 273. doi: 10.1109/TAC.1966.1098311.

[19]

J. M. Holtzman, Convexity and the maximum principle for discrete systems,, IEEE Trans. Automat. Control, AC-11 (1966), 30. doi: 10.1109/TAC.1966.1098311.

[20]

F. Horn and R. Jackson, Correspondence discrete maximum principle,, Ind. Eng. Chem. Fundamen., 4 (1965), 110.

[21]

R. M. May, Simple mathematical models with very complicated dynamics,, Nature., 261 (1976), 459. doi: 10.1038/261459a0.

[22]

J. A. Oriega and R. J. Leake, Discrete maximum principle with state constrained control,, SIAM J. Control. Optim., 15 (1977), 984. doi: 10.1137/0315063.

[23]

J. B. Pearson, Jr. and R. Sridhar, A discrete optimal control problem,, IEEE Trans. Automat. Control, AC-11 (1966), 171. doi: 10.1109/TAC.1966.1098287.

[24]

A. I. Propoĭ, The maximum principle for discrete systems,, Autom. Remote Control, 26 (1965), 1167.

[25]

J. B. Rosen, Optimal control and convex programming,, in, (1964), 287.

[26]

L. I. Rozonoèr, L.S. Pontryagin maximum principle of in the theory of optimum systems, Part III,, Autom. Remote Control, 20 (1959), 1517.

[27]

C. C. Tisdell and A. Zaidi, Basic qualitative and quantitative results for solutions to nonlinear, dynamic equations on time scales with an application to economic modelling,, Nonlinear Anal., 68 (2008), 3504. doi: 10.1016/j.na.2007.03.043.

[28]

J. M. Yong and H. W. Lou, "A Concise Course to Theory of Optimal Control,", (in Chinese), (2006).

[29]

Z. Zhan and W. Wei, Necessary conditions for optimal control problems on time scales,, Abstr. Appl. Anal., 2009 (9743).

[30]

Z. Zhan and W. Wei, On existence of optimal control governed by a class of the first-order linear dynamic systems on time scales,, Appl. Math. Comput., 215 (2009), 2070. doi: 10.1016/j.amc.2009.08.009.

[31]

Z. Zhan, W. Wei and Y. F. Li, Optimal control problem with terminal state constraint on time scales,, Pacific J. Optim., ().

[32]

Z. Zhan, W. Wei and H. Xu, Hamilton-Jacobi-Bellman equations on time scales,, Math. Comput. Modelling, 49 (2009), 2019. doi: 10.1016/j.mcm.2008.12.008.

show all references

References:
[1]

R. Agarwal, V. O. Espinar, K. Perera and D. R. Vivero, Basic properties of Sobolev's space on time scales,, Adv. Differential Equations, 2006 (3812), 1.

[2]

F. M. Atici, D. C. Biles and A. Lebedinsky, An application of time scales to economics,, Math. Comput. Modelling., 43 (2006), 718. doi: 10.1016/j.mcm.2005.08.014.

[3]

F. M. Atici and F. Uysal, A production-inventory model of HMMS on time scales,, Appl. Math. Lett., 21 (2008), 236. doi: 10.1016/j.aml.2007.03.013.

[4]

M. Bohner, Calculus of variations on time scales,, J. Dynam. Systems Appl., 13 (2004), 339.

[5]

M. Bohner and G. S. Gudeinov, Double integral calculus of variations of time scales,, Comput. Math. Appl., 54 (2007), 45. doi: 10.1016/j.camwa.2006.10.032.

[6]

M. Bohner and A. Peterson, "Dynamic Equations on Time Scales. An Introduction with Applications,", Birkhäuser Boston, (2001).

[7]

A. G. Butkovskiĭ, The necessary and sufficient optimality conditions for of sampled-data control systems,, (Russian), 24 (1963), 1056.

[8]

A. Cabada and D. R. Vivero, Criterions for absolute continuity on time scales,, J. Difference Equ. Appl., 11 (2005), 1013. doi: 10.1080/10236190500272830.

[9]

A. Cabada and D. R. Vivero, Expression of the Lebesgue $\Delta$-integral on time scales as a usual Lebesgue integral: Application to the calculus of $\Delta$-antiderivatives,, Math. Comput. Modelling, 43 (2006), 194. doi: 10.1016/j.mcm.2005.09.028.

[10]

I. Ekeland, Nonconvex minimization problems,, Bull. Amer. Math. Soc. (N.S.), 1 (1979), 443.

[11]

Rui A. C. Ferreira and D. F. M. Torres, Higher-order calculus of variations on time scales, in "Mathematical Control Theory and Finance,", Springer, (2008), 149.

[12]

Rui. A. C. Ferreira and D. F. M. Torres, Isoperimetric problems of the calculus of variations on time scales,, in, 514 (2010), 123.

[13]

J. G. P. Gamarra and R. V. Solé, Complex discrete dynamics from simple continuous population models,, Bull. Math. Biol., 64 (2002), 611. doi: 10.1006/bulm.2002.0286.

[14]

H. Geering, Continuous-time optimal control theory for cost functionals including discrete state penalty terms,, IEEE Trans. Automat. Control., AC-21 (1976), 866. doi: 10.1109/TAC.1976.1101377.

[15]

H. Halkin, A maximum principle of the Pontryagin type for systems described by nonlinear difference equations,, SIAM J. Control., 4 (1966), 90.

[16]

S. Hilger, Analysis on measure chains-a unified approach to continues and discrete calculus,, Results Math., 18 (1990), 18.

[17]

R. Hilscher and V. Zeidan, Calculus of variations on time scales: Weak local piecewise $C^$$1_{rd}$ solutions with variable endpoints,, J. Math. Anal. Appl., 289 (2004), 143. doi: 10.1016/j.jmaa.2003.09.031.

[18]

J. M. Holtzman, On the maximum principle for nonlinear discrete-time systems,, IEEE Trans. Automat. Control, AC-11 (1966), 273. doi: 10.1109/TAC.1966.1098311.

[19]

J. M. Holtzman, Convexity and the maximum principle for discrete systems,, IEEE Trans. Automat. Control, AC-11 (1966), 30. doi: 10.1109/TAC.1966.1098311.

[20]

F. Horn and R. Jackson, Correspondence discrete maximum principle,, Ind. Eng. Chem. Fundamen., 4 (1965), 110.

[21]

R. M. May, Simple mathematical models with very complicated dynamics,, Nature., 261 (1976), 459. doi: 10.1038/261459a0.

[22]

J. A. Oriega and R. J. Leake, Discrete maximum principle with state constrained control,, SIAM J. Control. Optim., 15 (1977), 984. doi: 10.1137/0315063.

[23]

J. B. Pearson, Jr. and R. Sridhar, A discrete optimal control problem,, IEEE Trans. Automat. Control, AC-11 (1966), 171. doi: 10.1109/TAC.1966.1098287.

[24]

A. I. Propoĭ, The maximum principle for discrete systems,, Autom. Remote Control, 26 (1965), 1167.

[25]

J. B. Rosen, Optimal control and convex programming,, in, (1964), 287.

[26]

L. I. Rozonoèr, L.S. Pontryagin maximum principle of in the theory of optimum systems, Part III,, Autom. Remote Control, 20 (1959), 1517.

[27]

C. C. Tisdell and A. Zaidi, Basic qualitative and quantitative results for solutions to nonlinear, dynamic equations on time scales with an application to economic modelling,, Nonlinear Anal., 68 (2008), 3504. doi: 10.1016/j.na.2007.03.043.

[28]

J. M. Yong and H. W. Lou, "A Concise Course to Theory of Optimal Control,", (in Chinese), (2006).

[29]

Z. Zhan and W. Wei, Necessary conditions for optimal control problems on time scales,, Abstr. Appl. Anal., 2009 (9743).

[30]

Z. Zhan and W. Wei, On existence of optimal control governed by a class of the first-order linear dynamic systems on time scales,, Appl. Math. Comput., 215 (2009), 2070. doi: 10.1016/j.amc.2009.08.009.

[31]

Z. Zhan, W. Wei and Y. F. Li, Optimal control problem with terminal state constraint on time scales,, Pacific J. Optim., ().

[32]

Z. Zhan, W. Wei and H. Xu, Hamilton-Jacobi-Bellman equations on time scales,, Math. Comput. Modelling, 49 (2009), 2019. doi: 10.1016/j.mcm.2008.12.008.

[1]

Hans Josef Pesch. Carathéodory's royal road of the calculus of variations: Missed exits to the maximum principle of optimal control theory. Numerical Algebra, Control & Optimization, 2013, 3 (1) : 161-173. doi: 10.3934/naco.2013.3.161

[2]

Md. Haider Ali Biswas, Maria do Rosário de Pinho. A nonsmooth maximum principle for optimal control problems with state and mixed constraints - convex case. Conference Publications, 2011, 2011 (Special) : 174-183. doi: 10.3934/proc.2011.2011.174

[3]

H. O. Fattorini. The maximum principle for linear infinite dimensional control systems with state constraints. Discrete & Continuous Dynamical Systems - A, 1995, 1 (1) : 77-101. doi: 10.3934/dcds.1995.1.77

[4]

Huaiqiang Yu, Bin Liu. Pontryagin's principle for local solutions of optimal control governed by the 2D Navier-Stokes equations with mixed control-state constraints. Mathematical Control & Related Fields, 2012, 2 (1) : 61-80. doi: 10.3934/mcrf.2012.2.61

[5]

Gastão S. F. Frederico, Delfim F. M. Torres. Noether's symmetry Theorem for variational and optimal control problems with time delay. Numerical Algebra, Control & Optimization, 2012, 2 (3) : 619-630. doi: 10.3934/naco.2012.2.619

[6]

Torsten Lindström. Discrete models and Fisher's maximum principle in ecology. Conference Publications, 2003, 2003 (Special) : 571-579. doi: 10.3934/proc.2003.2003.571

[7]

Hancheng Guo, Jie Xiong. A second-order stochastic maximum principle for generalized mean-field singular control problem. Mathematical Control & Related Fields, 2018, 8 (2) : 451-473. doi: 10.3934/mcrf.2018018

[8]

Guy Barles, Ariela Briani, Emmanuel Trélat. Value function for regional control problems via dynamic programming and Pontryagin maximum principle. Mathematical Control & Related Fields, 2018, 8 (3&4) : 509-533. doi: 10.3934/mcrf.2018021

[9]

Yunkyong Hyon, Do Young Kwak, Chun Liu. Energetic variational approach in complex fluids: Maximum dissipation principle. Discrete & Continuous Dynamical Systems - A, 2010, 26 (4) : 1291-1304. doi: 10.3934/dcds.2010.26.1291

[10]

Loïc Bourdin, Emmanuel Trélat. Optimal sampled-data control, and generalizations on time scales. Mathematical Control & Related Fields, 2016, 6 (1) : 53-94. doi: 10.3934/mcrf.2016.6.53

[11]

H. O. Fattorini. The maximum principle in infinite dimension. Discrete & Continuous Dynamical Systems - A, 2000, 6 (3) : 557-574. doi: 10.3934/dcds.2000.6.557

[12]

Yan Wang, Yanxiang Zhao, Lei Wang, Aimin Song, Yanping Ma. Stochastic maximum principle for partial information optimal investment and dividend problem of an insurer. Journal of Industrial & Management Optimization, 2018, 14 (2) : 653-671. doi: 10.3934/jimo.2017067

[13]

Shanjian Tang. A second-order maximum principle for singular optimal stochastic controls. Discrete & Continuous Dynamical Systems - B, 2010, 14 (4) : 1581-1599. doi: 10.3934/dcdsb.2010.14.1581

[14]

C.Z. Wu, K. L. Teo. Global impulsive optimal control computation. Journal of Industrial & Management Optimization, 2006, 2 (4) : 435-450. doi: 10.3934/jimo.2006.2.435

[15]

Dingjun Yao, Kun Fan. Optimal risk control and dividend strategies in the presence of two reinsurers: Variance premium principle. Journal of Industrial & Management Optimization, 2018, 14 (3) : 1055-1083. doi: 10.3934/jimo.2017090

[16]

Carlo Orrieri. A stochastic maximum principle with dissipativity conditions. Discrete & Continuous Dynamical Systems - A, 2015, 35 (11) : 5499-5519. doi: 10.3934/dcds.2015.35.5499

[17]

Ricardo Almeida, Agnieszka B. Malinowska. Fractional variational principle of Herglotz. Discrete & Continuous Dynamical Systems - B, 2014, 19 (8) : 2367-2381. doi: 10.3934/dcdsb.2014.19.2367

[18]

Hang-Chin Lai, Jin-Chirng Lee, Shuh-Jye Chern. A variational problem and optimal control. Journal of Industrial & Management Optimization, 2011, 7 (4) : 967-975. doi: 10.3934/jimo.2011.7.967

[19]

Yunfei Peng, X. Xiang. A class of nonlinear impulsive differential equation and optimal controls on time scales. Discrete & Continuous Dynamical Systems - B, 2011, 16 (4) : 1137-1155. doi: 10.3934/dcdsb.2011.16.1137

[20]

Y. Gong, X. Xiang. A class of optimal control problems of systems governed by the first order linear dynamic equations on time scales. Journal of Industrial & Management Optimization, 2009, 5 (1) : 1-10. doi: 10.3934/jimo.2009.5.1

2017 Impact Factor: 0.631

Metrics

  • PDF downloads (3)
  • HTML views (0)
  • Cited by (3)

Other articles
by authors

[Back to Top]