Control parameterization approach to time-delay optimal control problems: A survey

Di Wu; Yin Chen; Changjun Yu; Yanqin Bai; Kok Lay Teo

doi:10.3934/jimo.2022108

Article Contents

2023, Volume 19, Issue 5: 3750-3783. Doi: 10.3934/jimo.2022108

This issue Previous Article A unified and tight linear convergence analysis of the relaxed proximal point algorithm Next Article Traffic flow modeling and optimization control in the approach airspace

Control parameterization approach to time-delay optimal control problems: A survey

1.
Department of Mathematics, Shanghai University, Shanghai 200444, China
2.
School of Mathematical Sciences, Sunway University, Malaysia

^*Corresponding author: Changjun Yu (yuchangjun@126.com)
^*Corresponding author: Changjun Yu (yuchangjun@126.com)

Received: March 2022

Revised: April 2022

Early access: June 2022

Published: May 2023

This work is supported by National Natural Science Foundation of China(NSFC), Grant No.11871039, No. 12171307 and Science and Technology Commission of Shanghai Municipality(STCSM), Grant No. 20JC1413900

Abstract Full Text(HTML) Figure(3) Related Papers Cited by

Abstract

Control parameterization technique is an effective method to solve optimal control problems. It works by approximating the control function by piece-wise constant (or linear) function. In this way, the optimal control problems are approximated by optimal parameter selection problems, which can be regarded as finite-dimensional optimization problems, where the control heights and switching times of the piece-wise constant function are taken as the decision variables. They can be solved by using gradient-based optimization methods. For this, it requires the gradient formulas of the objective and constraint functions with respect to the decision variables. There are two methods for deriving the required gradient formulas. The aim of this paper is to survey various numerical methods developed based on control parameterization for solving optimal control problems governed by time-delay systems. Particular focus is on those numerical methods for optimizing switching time points. This is because the process of optimizing the switching time points is much more complicated for optimal control problems with time delays. In addition, we shall also review the computational methods developed based on control parameterization technique for solving two optimal control problems involving more complicated time delays.

Keywords:

Mathematics Subject Classification: Primary: 49M37; Secondary: 65K10, 65P10, 90C30, 93C15.

Citation:

Full Text(HTML)

Figure 1. Piece-wise constant function approximation with equidistant switching times over the planning horizon $ (p=5) $

Download: Full-size image PowerPoint slide

Figure 2. The illustration of the time-scaling function $ (p=5) $

Download: Full-size image PowerPoint slide

Figure 3. The relationship between $ t $, $ t' $, $ s $ and $ s' $

Download: Full-size image PowerPoint slide

Related Papers

Cited by

References

[1]	N. U. Ahmed, Dynamic Systems and Control with Applications, World Scientific, Singapore, 2006. doi: 10.1142/6262.
[2]	C. Abdallah and J. Chiasson, Stability of communication networks in the presence of delays, IFAC Proceedings Volumes, 34 (2001), 171-174. doi: 10.1016/S1474-6670(17)32885-9.
[3]	M. Alipour, M. A. Vali and A. H. Borzabadi, A hybrid parameterization approach for a class of nonlinear optimal control problems, Numerical Algebra, 9 (2019), 493-506. doi: 10.3934/naco.2019037.
[4]	H. Banks, Necessary conditions for control problems with variable time lags, SIAM Journal on Control, 6 (1968), 9-47. doi: 10.1137/0306002.
[5]	J. C. Butcher, Numerical Methods for Ordinary Differential Equations, 2$^nd$ edition, John Wiley & Sons, New York, 2008. doi: 10.1002/9780470753767.
[6]	L. Bushnell, Networks and control [guest editorial], IEEE Control System Magazine, 21 (2001), 22-99. doi: 10.1109/MCS.2001.898789.
[7]	J. A. E. Bryson and Y. C. Ho, Applied optimal control: Optimization, estimation and control, Taylor & Francis, 1975.
[8]	E. Blanchard, R. Loxton and V. Rehbock, A computational algorithm for a class of non-smooth optimal control problems arising in aquaculture operations, Applied Mathematics and Computation, 219 (2013) 8738–8746. doi: 10.1016/j. amc. 2013.02.070.
[9]	C. Bukens and H. Maurer, Nonlinear programming methods for real-time control of an industrial robot, Journal of Optimization Theory and Applications, 107 (2000), 505-527. doi: 10.1023/A:1026491014283.
[10]	P. T. Boggs and J. W. Tolle, Sequential quadratic programming, Acta Numerica, 4 (1995), 1-51. doi: 10.1017/S0962492900002518.
[11]	J. Banas and A. Vacroux, Optimal piecewise constant control of continuous time systems with time-varying delay, Automatica, 6 (1970), 809-811. doi: 10.1016/0005-1098(70)90029-4.
[12]	E. Biberovic, A. Iftar and H. Ozbay, A solution to the robust flow control problem for networks with multiple bottlenecks, in 40th IEEE CDC'01 (Conference on Decision and Control), Orlando, FL, December, 2001, 2303–2308. doi: 10.1109/CDC. 2001.980603.
[13]	F. Ceragioli, Discontinuous ordinary differential equations and stabilization, Universita di Firenze, 1999.
[14]	Q. Q. Chai, R. Loxton, K. L. Teo and C. H. Yang, A class of optimal state-delay control problems, Nonlinear Anal. Real World Appl., 14 (2013) 1536–1550. doi: 10.1016/j. nonrwa. 2012.10.017.
[15]	Q. Q. Chai, R. Loxton, K. L. Teo and C. H. Yang, Time-delay estimation for nonlinear systems with piecewise-constant input, Applied Mathematics and Computation, 219 (2013) 9543–9560. doi: 10.1016/j. amc. 2013.03.015.
[16]	Q. Q. Chai, C. H. Yang, K. L. Teo and W. H. Gui, Optimal control of an industrial-scale evaporation process: Sodium aluminate solution, Control Engineering Practice, 20 (2012), 618-628. doi: 10.1016/j.conengprac.2012.03.001.
[17]	P. J. Davis, Iterpolation and approximation, Dover Publications, 1975.
[18]	A. L. Dontchev and W. W. Hager, The Euler approximation in state constrained optimal control, Mathematics of Computation, 70 (2001), 173-203. doi: 10.1090/S0025-5718-00-01184-4.
[19]	S. Dadebo and R. Luus, Optimal control of time-delay systems by dynamic programming, Optimal Control Applications & Methods, 13 (1992), 29-41. doi: 10.1002/oca.4660130103.
[20]	D. Giang, Y. Lenbury and Y. Seidman, Delay effect in models of population growth, Journal of Mathematical Analysis and Applications, 305 (2005), 631-643. doi: 10.1016/j.jmaa.2004.12.018.
[21]	Z. H. Gong, C. Y. Liu, K. L. Teo and X. P. Yi, Optimal control of nonlinear fractional systems with multiple pantograph-delays, Applied Mathematics and Computation, 425 (2022), 127094. doi: 10.1016/j.amc.2022.127094.
[22]	D. Garg, M. Patterson, W. W. Hager, A. V. Rao, D. A. Benson and G. T. Huntington, A unified framework for the numerical solution of optimal control problems using pseudospectral methods, Automatica, 46 (2010), 1843-1851. doi: 10.1016/j.automatica.2010.06.048.
[23]	C. J. Goh and K. L. Teo, Control parameterization: A unified approach to optimal control problems with general constraints, Automatica, 24 (1988), 3-18. doi: 10.1016/0005-1098(88)90003-9.
[24]	L. Hasdorff, Gradient optimization and nonlinear control, John Wiley, New York, 1976.
[25]	W. W. Hager, Runge-Kutta methods in optimal control and the transformed adjoint system, Numerische Mathematik, 87 (2000), 247-282. doi: 10.1007/s002110000178.
[26]	I. Hussein and A. Bloch, Optimal control of underactuated nonholonomic mechanical systems, IEEE Transactions on Automatic Control, 53 (2008), 668-682. doi: 10.1109/TAC.2008.919853.
[27]	R. F. Hartl, S. P. Serhi and R. G. Vickson, A survey of the maximum principles for optimal control problems with state constraints, SIAM Review, 37 (1995), 181-218. doi: 10.1137/1037043.
[28]	C. H. Jiang, K. Xie, C. J. Yu, M. Yu, H. Wang and Y. G. He, A sequential computational approach to optimal control problems for differential-algebraic systems based on efficient implicit Runge-Kutta integration, Applied Mathematical Modeling, 58 (2018), 313-330. doi: 10.1016/j.apm.2017.05.015.
[29]	C. Y. Kaya and J. L. Noakes, Computational method for time-optimal switching control, Journal of Optimization Theory and Applications, 117 (2003) 69–92. doi: 10.1023/A: 1023600422807.
[30]	C. Y. Liu, Z. H. Gong, C. J. Yu, S. Wang and K. L. Teo, Optimal control computation for nonlinear fractional time-delay systems with inequality constraints, Journal of Optimization Theory and Applications, 191 (2021), 83-117. doi: 10.1007/s10957-021-01926-8.
[31]	C. Y. Liu, R. Loxton, Q. Lin and K. L. Teo, Dynamic optimization for switched time-delay systems with state-dependent switching conditions, SIAM Journal on Control and Optimization, 56 (2018), 3499-3523. doi: 10.1137/16M1070530.
[32]	C. Y. Liu, R. Loxton and K. L. Teo, Optimal parameter selection for nonlinear multistage systems with time-delays, Comput. Optim. Appl., 59 (2014) 285–306. doi: 10.1007/s10589-013-9632-x.
[33]	C. Liu, R. Loxton and K. L. Teo, Switching time and parameter optimization in nonlinear switched systems with multiple time delays, Journal of Optimization Theory and Applications, 163 (2014), 957-988. doi: 10.1007/s10957-014-0533-7.
[34]	C. Y. Liu, R. Loxton and K. L. Teo, A computational method for solving time-delay optimal control problems with free terminal time, Systems & Control Letters, 72 (2014), 53-60. doi: 10.1016/j.sysconle.2014.07.001.
[35]	Q. Lin, R. Loxton and K. L. Teo, Optimal control of nonlinear switched systems: Computational methods and applications, Journal of the Operations Research Society of China, 1 (2013), 275-311. doi: 10.1007/s40305-013-0021-z.
[36]	Q. Lin, R. C. Loxton and K. L. Teo, The control parameterization method for nonlinear optimal control: A survey, Journal of Industrial and Management Optimization, 10 (2014), 275-309. doi: 10.3934/jimo.2014.10.275.
[37]	R. Loxton, Q. Lin and K. L. Teo, Minimizing control variation in nonlinear optimal control, Automatica, 49 (2013), 2652-2664. doi: 10.1016/j.automatica.2013.05.027.
[38]	C. Y. Liu, R. Loxton, K. L. Teo and S. Wang, Optimal state-delay control in nonlinear dynamic systems, Automatica, 135 (2022), 109981. doi: 10.1016/j.automatica.2021.109981.
[39]	Q. Lin, R. Loxton, K. L. Teo and Y. H. Wu, A new computational method for optimizing nonlinear impulsive systems, Dynamics of Continuous, Discrete and Impulsive Systems Series B: Applications and Algorithms, 18 (2011), 59-76.
[40]	Q. Lin, R. Loxton, K. L. Teo and Y. H. Wu, Optimal control computation for nonlinear systems with state-dependent stopping criteria, Automatica, 48 (2012), 2116-2129. doi: 10.1016/j.automatica.2012.06.055.
[41]	H. W. J. Lee, K. L. Teo and L. S. Jennings, On optimal control of multi-link vertical planar robot arms systems moving under the effect of gravity, Journal of the Australian Mathematical Society Series B, 39 (1997), 195-213. doi: 10.1017/S0334270000008808.
[42]	R. C. Loxton, K. L. Teo and V. Rehbock, Optimal control problems with multiple characteristic time points in the objecticve and constraints, Automatica, 44 (2008), 2923-2929. doi: 10.1016/j.automatica.2008.04.011.
[43]	H. W. J. Lee, K. L. Teo, V. Rehbock and L. S. Jennings, Control parametrization enhancing technique for time optimal control problems, Dynamic Systems and Applications, 6 (1997), 243-261. doi: 10.1007/978-1-4757-3333-4_9.
[44]	H. W. J. Lee, K. L. Teo, V. Rehbock and L. S. Jennings, Control parameterization enhancing technique for optimal discrete-valued control problems, Automatica, 35 (1999), 1401-1407. doi: 10.1016/S0005-1098(99)00050-3.
[45]	R. Loxton, K. L. Teo, V. Rehbock and W. K. Ling, Optimal switching instants for a switched-capacitor DC/DC power converter, Automatica, 45 (2009), 973-980. doi: 10.1016/j.automatica.2008.10.031.
[46]	L. N. Li, C. J. Yu, N. Zhang, Y. Q. Bai and Z. Y. Gao, A time-scaling technique for time-delay switched systems, Discrete and Continuous Dynamical Systems Series S, 13 (2020), 1825-1843. doi: 10.3934/dcdss.2020108.
[47]	B. Li, G. Zhu, Y. F. Sun, G. Aw and K. L. Teo, Multi-period portfolio selection problem under uncertain environment with bankruptcy constraint, Applied Mathematical Modelling, 39 (2018), 539-550. doi: 10.1016/j.apm.2017.12.016.
[48]	B. Li, G. Zhu, Y. F. Sun, G. Aw and K. L. Teo, Deterministic conversion of uncertain manpower planning optimization problem, IEEE Transactions on Fuzzy Systems, 26 (2018), 2748-2757. doi: 10.1109/TFUZZ.2018.2803736.
[49]	R. B. Martin, Optimal control drug scheduling of cancer chemotherapy, Automatica, 28 (1992), 1113-1123. doi: 10.1016/0005-1098(92)90054-J.
[50]	A. M$\acute{e}$rigot and H. Mounier, Quality of service and MPEG4 video transmission, in MTNS'00 (14th Symposium on Mathematical Theory of Networks and Systems), Perpignan, France, 2000.
[51]	R. Martin and K. L. Teo, Optimal control of drug administration in cancer chemotherapy, World Scientific, Singapore, 1994. doi: 10.1142/2048.
[52]	J. Nilsson, B. Bernhardsson and B. Wittenmark, Stochastic analysis and control of real-time systems with random delays, Automatica, 34 (1998), 57-64. doi: 10.1016/S0005-1098(97)00170-2.
[53]	I. M. Ross and M. Karpenko, A review of pseudospectral optimal control: From theory to flight, Annual Reviews in Control, 36 (2012), 182-197. doi: 10.1016/j.arcontrol.2012.09.002.
[54]	B. Ni, D. Xiao and S. L. Shah, Time delay estimation for MIMO dynamical systems – with time-frequency domain analysis, Journal of Process Control, 20 (2010), 83-94. doi: 10.1016/j.jprocont.2009.10.002.
[55]	F. A. Potra and S. J. Wright, Interior-point methods, Journal of Computational and Applied Mathematics, 124 (2000), 281-302. doi: 10.1016/S0377-0427(00)00433-7.
[56]	J. P. Richard, Time-delay systems: An overview of some recent advances and open problems, Automatica, 39 (2003), 1667-1694. doi: 10.1016/S0005-1098(03)00167-5.
[57]	Y. Sun, E. Aw, K. L. Teo and X. Wang, Chance constrained optimization for pension fund portfolios in the presence of default risk, European Journal of Operational Research, 256 (2017), 205-214. doi: 10.1016/j.ejor.2016.06.019.
[58]	A. Siburian and V. Rehbock, Numerical procedure for solving a class of singular optimal control problems, Optimization Methods and Software, 19 (2004), 413-426. doi: 10.1080/10556780310001656637.
[59]	G. Teschl, Ordinary Differential Equations and Dynamical Systems, American Mathematical Society, 2011. doi: 10.1090/gsm/140.
[60]	Y. Tang and X. Guan, Parameter estimation for time-delay chaotic systems by particle swarm optimization, Chaos, Solitons and Fractals, 40 (2009), 1391-1398. doi: 10.1016/j.chaos.2007.09.055.
[61]	K. L. Teo, C. J. Goh and K. H. Wong, A unified computational approach to optimal control problems, Longman Scientific and Technical, Essex, 1991.
[62]	K. L. Teo and L. S. Jennings, Nonlinear optimal control problems with continuous state inequality constraints, Journal of Optimization Theory and Application, 63 (1989), 1-22. doi: 10.1007/BF00940727.
[63]	K. L. Teo, K. K. Leong and C. J. Goh, Nonlinearly constrained optimal control problems involving piecewise smooth controls, Journal of Australian Mathematical Society, Series B, 32 (1990), 151-179. doi: 10.1017/S0334270000008407.
[64]	X. J. Tang and H. Y. Xu, Multiple-interval pseudospectral approximation for nonlinear optimal control problems with time-varying delays, Applied Mathematical Modelling, 68 (2019), 137-151. doi: 10.1016/j.apm.2018.09.039.
[65]	K. L. Teo, B. Li, C. J. Yu and V. Rehbock, Applied and computational optimal control: A control parameterization approach, Springer, 2021. doi: 10.1007/978-3-030-69913-0.
[66]	K. L. Teo, K. H. Wong and D. J. Clements, Optimal control computation for linear time-lag systems with linear terminal constraints, Journal of Optimization Theory and Applications, 44 (1984), 509-526. doi: 10.1007/BF00935465.
[67]	X. Tang and H. Xu, Multiple-interval pseudospectral approximation for nonlinear optimal control problems with time-varying delays, Applied Mathematical Modeling, 68 (2019), 137-151. doi: 10.1016/j.apm.2018.09.039.
[68]	A. Tyatyushkin and T. Zarodnyuk, Numerical method for solving optimal control problems with phase constaints, Numerical Algebra, 7 (2017), 481-492. doi: 10.3934/naco.2017030.
[69]	T. L. Vincent and W. J. Grantham, Optimality in Parametric Systems, John Wiley, New York, 1981. doi: 10.1115/1.3167074.
[70]	K. H. Wong, Convergence analysis of a computational method for time-lag optimal control problems, International Journal of Systems Science, 19 (1988), 1437-1450. doi: 10.1080/00207728808964048.
[71]	D. Wu, Y. Q. Bai and F. S. Xie, Time-scaling transformation for optimal control problem with time-varying delay, Discrete and Continuous Dynamical Systems Series S, 13 (2020), 1683-1695. doi: 10.3934/dcdss.2020098.
[72]	D. Wu, Y. Q. Bai and C. J. Yu, A new computational approach for optimal control problems with multiple time-delay, Automatica, 101 (2019), 388-395. doi: 10.1016/j.automatica.2018.12.036.
[73]	K. H. Wong, D. J. Clements and K. L. Teo, Optimal control computation for nonlinear time-lag systems, Journal of Optimization Theory and Applications, 47 (1985), 91-107. doi: 10.1007/BF00941318.
[74]	C. Z. Wu and K. L. Teo, Optimal impulsive control computation, Journal of Industrial and Management Optimization, 2 (2006), 435-450. doi: 10.3934/jimo.2006.2.435.
[75]	C. Z. Wu, K. L. Teo, R. Li and Y. Zhao, Optimal control of switched systems with time delay, Applied Mathematics Letters, 19 (2006), 1062-1067. doi: 10.1016/j.aml.2005.11.018.
[76]	C. J. Yu, Q. Lin, R. Loxton, K. L.Teo and G. Q. Wang, A hybrid time-scaling transformation for time-delay optimal control problems, Journal of Optimization Theory and Applications, 169 (2016), 876-901. doi: 10.1007/s10957-015-0783-z.
[77]	C. J. Yu, L. Yuan and S. Su, A new gradient computational formula for optimal control problems with time-delay, Journal of Industrial and Management Optimization, 18 (2022), 2469-2482. doi: 10.3934/jimo.2021076.
[78]	N. Zhang, C. J. Yu and F. S. Xie, The time-scaling transformation technique for optimal control problems with time-varying time-delay switched systems, Journal of the Operations Research Society of China, 8 (2020), 581-600. doi: 10.1007/s40305-020-00299-5.