July  2022, 18(4): 2469-2482. doi: 10.3934/jimo.2021076

A new gradient computational formula for optimal control problems with time-delay

Shanghai University, Shanghai, 200444, China

* Corresponding author: Lei Yuan

Received  December 2020 Revised  February 2021 Published  July 2022 Early access  April 2021

Fund Project: This work is supported by National Natural Science Foundation of China(NSFC), Grant No.11871039 and Science and Technology Commission of Shanghai Municipality(STCSM), Grant No. 20JC1413900.

In this paper, we consider a class of time-delay optimal control problem (TDOCP) with canonical equality and inequality constraints. By applying control parameterization method together with time-scaling transformation, a TDOCP can be readily solved by gradient-based optimization methods. The partial derivative of the cost as well as the constraint functions with respect to the decision variables are obtained by variational approach, which is inefficient when the discretization for the control function is relatively dense. For general optimal control problem without time-delay, co-state approach is an effective way to compute the gradients, however, when time-delay is involved in the dynamic system, the co-state system is not known. In this paper, we derive the co-state system for TDOCP to compute the gradients of the cost and constraints. Numerical results show that the computational efficiency is much higher when compared with the traditional variational approach.

Citation: Changjun Yu, Lei Yuan, Shuxuan Su. A new gradient computational formula for optimal control problems with time-delay. Journal of Industrial and Management Optimization, 2022, 18 (4) : 2469-2482. doi: 10.3934/jimo.2021076
References:
[1]

J. T. Betts, S. L. Campbell and K. C. Thompson, Optimal control software for constrained nonlinear systems with delays, in IEEE International Symposium on Computer-Aided Control System Design (CACSD), IEEE, 2011,444–449.

[2]

Q. ChaiR. LoxtonK. L. Teo and C. Yang, A class of optimal state-delay control problems, Nonlinear Analysis: Real World Applications, 14 (2013), 1536-1550.  doi: 10.1016/j.nonrwa.2012.10.017.

[3]

Q. ChaiR. LoxtonK. L. Teo and C. Yang, A unified parameter identification method for nonlinear time-delay systems, Journal of Industrial and Management Optimization, 9 (2013), 471-486.  doi: 10.3934/jimo.2013.9.471.

[4]

Q. Q. ChaiC. H. YangK. L. Teo and W. H. Gui, Optimal control of an industrial-scale evaporation process: Sodium aluminate solution, Control Engineering Practice, 20 (2012), 618-628.  doi: 10.1016/j.conengprac.2012.03.001.

[5]

L. Denis-VidalC. Jauberthie and G. Joly-Blanchard, Identifiability of a nonlinear delayed-differential aerospace model, IEEE Transactions on Automatic Control, 51 (2006), 154-158.  doi: 10.1109/TAC.2005.861700.

[6]

V. DeshmukhH. Ma and E. A. Butcher, Optimal control of parametrically excited linear delay differential systems via chebyshev polynomials, Optimal Control Applications and Methods, 27 (2006), 123-136.  doi: 10.1002/oca.769.

[7]

C. J. Goh and K. L. Teo, Control parametrization: A unified approach to optimal control problems with general constraints, Automatica, 24 (1988), 3-18.  doi: 10.1016/0005-1098(88)90003-9.

[8]

L. GöllmannD. Kern and H. Maurer, Optimal control problems with delays in state and control variables subject to mixed control–state constraints, Optimal Control Applications and Methods, 30 (2009), 341-365.  doi: 10.1002/oca.843.

[9]

L. Göllmann and H. Maurer, Theory and applications of optimal control problems with multiple time-delays, Journal of Industrial and Management Optimization, 10 (2014), 413-441.  doi: 10.3934/jimo.2014.10.413.

[10]

K. Kaji and K. H. Wong, Nonlinearly constrained time-delayed optimal control problems, Journal of Optimization Theory and Applications, 82 (1994), 295-313.  doi: 10.1007/BF02191855.

[11]

C. Y. Kaya and J. L. Noakes, Computational method for time-optimal switching control, Journal of Optimization Theory and Applications, 117 (2003), 69-92.  doi: 10.1023/A:1023600422807.

[12]

H. W. J. LeeK. L. TeoV. Rehbock and L. S. Jennings, Control parametrization enhancing technique for time optimal control problems, Dynamic Systems and Applications, 6 (1997), 243-261. 

[13]

H. W. J. LeeK. L. TeoV. Rehbock and L. S. Jennings, Control parametrization enhancing technique for optimal discrete-valued control problems, Automatica, 35 (1999), 1401-1407.  doi: 10.1016/S0005-1098(99)00050-3.

[14]

B. LiC. J. YuK. L. Teo and G. R. Duan, An exact penalty function method for continuous inequality constrained optimal control problem, Journal of Optimization Theory and Applications, 151 (2011), 260-291.  doi: 10.1007/s10957-011-9904-5.

[15]

L. LiC. YuN. ZhangY. Bai and Z. Gao, A time-scaling technique for time-delay switched systems, Discrete and Continuous Dynamical Systems-S, 13 (2020), 1825-1843.  doi: 10.3934/dcdss.2020108.

[16]

Q. LinR. Loxton and K. L. Teo, The control parameterization method for nonlinear optimal control: A survey, Journal of Industrial and Management Optimization, 10 (2014), 275-309.  doi: 10.3934/jimo.2014.10.275.

[17]

Q. LinR. LoxtonK. L. Teo and Y. H. Wu, A new computational method for optimizing nonlinear impulsive systems, Dynamics of Continuous, Discrete and Impulsive Systems B, 18 (2011), 59-76. 

[18]

R. C. LoxtonK. L. Teo and V. Rehbock, Optimal control problems with multiple characteristic time points in the objective and constraints, Automatica, 44 (2008), 2923-2929.  doi: 10.1016/j.automatica.2008.04.011.

[19]

M. Malek-Zavarei and M. Jamshidi, Time-Delay Systems: Analysis, Optimization and Applications, North-Holland Systems and Control Series, 9. North-Holland Publishing Co., Amsterdam, 1987.

[20]

H. R. Marzban and S. M. Hoseini, An efficient discretization scheme for solving nonlinear optimal control problems with multiple time delays, Optimal Control Applications and Methods, 37 (2016), 682-707.  doi: 10.1002/oca.2187.

[21]

P. MuL. Wang and C. Liu, A control parameterization method to solve the fractional-order optimal control problem, Journal of Optimization Theory and Applications, 187 (2020), 234-247.  doi: 10.1007/s10957-017-1163-7.

[22]

A. NasirE. M. Atkins and I. Kolmanovsky, Robust science-optimal spacecraft control for circular orbit missions, IEEE Transactions on Systems Man and Cybernetics Systems, 50 (2020), 923-934.  doi: 10.1109/TSMC.2017.2767077.

[23]

J. Nocedal and S. J. Wright, Numerical Optimization, 2$^nd$ edition, Springer Series in Operations Research and Financial Engineering, Springer, New York, 2006.

[24]

D. Stefanatos, Optimal shortcuts to adiabaticity for a quantum piston, Automatica, 49 (2013), 3079-3083.  doi: 10.1016/j.automatica.2013.07.020.

[25]

R. F. StengelR. GhigliazzaN. Kulkarni and O. Laplace, Optimal control of innate immune response, Optimal Control Applications and Methods, 23 (2002), 91-104.  doi: 10.1002/oca.704.

[26]

K. L. Teo, C. J. Goh and K. H. Wong, A Unified Computational Approach to Optimal Control Problems, Longman Scientific and Technical, 1991.

[27]

T. L. Vincent, W. J. Grantham and W. Stadler, Optimality in Parametric Systems, American Society of Mechanical Engineers Digital Collection, 1983.

[28]

L. WangJ. YuanC. Wu and X. Wang, Practical algorithm for stochastic optimal control problem about microbial fermentation in batch culture, Optimization Letters, 13 (2019), 527-541.  doi: 10.1007/s11590-017-1220-z.

[29]

K. H. WongL. S. Jennings and F. Benyah, The control parametrization enhancing transform for constrained time–delayed optimal control problems, ANZIAM Journal, 43 (2001), 154-185. 

[30]

D. WuY. Bai and F. Xie, Time-scaling transformation for optimal control problem with time-varying delay, Discrete and Continuous Dynamical Systems-S, 13 (2020), 1683-1695.  doi: 10.3934/dcdss.2020098.

[31]

D. WuY. Bai and C. Yu, A new computational approach for optimal control problems with multiple time-delay, Automatica, 101 (2019), 388-395.  doi: 10.1016/j.automatica.2018.12.036.

[32]

X. Xu and P. J. Antsaklis, Optimal control of switched systems based on parameterization of the switching instants, IEEE Transactions on Automatic Control, 49 (2004), 2-16.  doi: 10.1109/TAC.2003.821417.

[33]

C. YuB. LiR. Loxton and K. L. Teo, Optimal discrete-valued control computation, Journal of Global Optimization, 56 (2013), 503-518.  doi: 10.1007/s10898-012-9858-7.

[34]

C. YuQ. LinR. LoxtonK. L. Teo and G. Wang, A hybrid time-scaling transformation for time-delay optimal control problems, Journal of Optimization Theory and Applications, 169 (2016), 876-901.  doi: 10.1007/s10957-015-0783-z.

[35]

N. ZhangC. -J. Yu and F. -S. Xie, The time-scaling transformation technique for optimal control problems with time-varying time-delay switched systems, Journal of the Operations Research Society of China, 8 (2020), 581-600.  doi: 10.1007/s40305-020-00299-5.

show all references

References:
[1]

J. T. Betts, S. L. Campbell and K. C. Thompson, Optimal control software for constrained nonlinear systems with delays, in IEEE International Symposium on Computer-Aided Control System Design (CACSD), IEEE, 2011,444–449.

[2]

Q. ChaiR. LoxtonK. L. Teo and C. Yang, A class of optimal state-delay control problems, Nonlinear Analysis: Real World Applications, 14 (2013), 1536-1550.  doi: 10.1016/j.nonrwa.2012.10.017.

[3]

Q. ChaiR. LoxtonK. L. Teo and C. Yang, A unified parameter identification method for nonlinear time-delay systems, Journal of Industrial and Management Optimization, 9 (2013), 471-486.  doi: 10.3934/jimo.2013.9.471.

[4]

Q. Q. ChaiC. H. YangK. L. Teo and W. H. Gui, Optimal control of an industrial-scale evaporation process: Sodium aluminate solution, Control Engineering Practice, 20 (2012), 618-628.  doi: 10.1016/j.conengprac.2012.03.001.

[5]

L. Denis-VidalC. Jauberthie and G. Joly-Blanchard, Identifiability of a nonlinear delayed-differential aerospace model, IEEE Transactions on Automatic Control, 51 (2006), 154-158.  doi: 10.1109/TAC.2005.861700.

[6]

V. DeshmukhH. Ma and E. A. Butcher, Optimal control of parametrically excited linear delay differential systems via chebyshev polynomials, Optimal Control Applications and Methods, 27 (2006), 123-136.  doi: 10.1002/oca.769.

[7]

C. J. Goh and K. L. Teo, Control parametrization: A unified approach to optimal control problems with general constraints, Automatica, 24 (1988), 3-18.  doi: 10.1016/0005-1098(88)90003-9.

[8]

L. GöllmannD. Kern and H. Maurer, Optimal control problems with delays in state and control variables subject to mixed control–state constraints, Optimal Control Applications and Methods, 30 (2009), 341-365.  doi: 10.1002/oca.843.

[9]

L. Göllmann and H. Maurer, Theory and applications of optimal control problems with multiple time-delays, Journal of Industrial and Management Optimization, 10 (2014), 413-441.  doi: 10.3934/jimo.2014.10.413.

[10]

K. Kaji and K. H. Wong, Nonlinearly constrained time-delayed optimal control problems, Journal of Optimization Theory and Applications, 82 (1994), 295-313.  doi: 10.1007/BF02191855.

[11]

C. Y. Kaya and J. L. Noakes, Computational method for time-optimal switching control, Journal of Optimization Theory and Applications, 117 (2003), 69-92.  doi: 10.1023/A:1023600422807.

[12]

H. W. J. LeeK. L. TeoV. Rehbock and L. S. Jennings, Control parametrization enhancing technique for time optimal control problems, Dynamic Systems and Applications, 6 (1997), 243-261. 

[13]

H. W. J. LeeK. L. TeoV. Rehbock and L. S. Jennings, Control parametrization enhancing technique for optimal discrete-valued control problems, Automatica, 35 (1999), 1401-1407.  doi: 10.1016/S0005-1098(99)00050-3.

[14]

B. LiC. J. YuK. L. Teo and G. R. Duan, An exact penalty function method for continuous inequality constrained optimal control problem, Journal of Optimization Theory and Applications, 151 (2011), 260-291.  doi: 10.1007/s10957-011-9904-5.

[15]

L. LiC. YuN. ZhangY. Bai and Z. Gao, A time-scaling technique for time-delay switched systems, Discrete and Continuous Dynamical Systems-S, 13 (2020), 1825-1843.  doi: 10.3934/dcdss.2020108.

[16]

Q. LinR. Loxton and K. L. Teo, The control parameterization method for nonlinear optimal control: A survey, Journal of Industrial and Management Optimization, 10 (2014), 275-309.  doi: 10.3934/jimo.2014.10.275.

[17]

Q. LinR. LoxtonK. L. Teo and Y. H. Wu, A new computational method for optimizing nonlinear impulsive systems, Dynamics of Continuous, Discrete and Impulsive Systems B, 18 (2011), 59-76. 

[18]

R. C. LoxtonK. L. Teo and V. Rehbock, Optimal control problems with multiple characteristic time points in the objective and constraints, Automatica, 44 (2008), 2923-2929.  doi: 10.1016/j.automatica.2008.04.011.

[19]

M. Malek-Zavarei and M. Jamshidi, Time-Delay Systems: Analysis, Optimization and Applications, North-Holland Systems and Control Series, 9. North-Holland Publishing Co., Amsterdam, 1987.

[20]

H. R. Marzban and S. M. Hoseini, An efficient discretization scheme for solving nonlinear optimal control problems with multiple time delays, Optimal Control Applications and Methods, 37 (2016), 682-707.  doi: 10.1002/oca.2187.

[21]

P. MuL. Wang and C. Liu, A control parameterization method to solve the fractional-order optimal control problem, Journal of Optimization Theory and Applications, 187 (2020), 234-247.  doi: 10.1007/s10957-017-1163-7.

[22]

A. NasirE. M. Atkins and I. Kolmanovsky, Robust science-optimal spacecraft control for circular orbit missions, IEEE Transactions on Systems Man and Cybernetics Systems, 50 (2020), 923-934.  doi: 10.1109/TSMC.2017.2767077.

[23]

J. Nocedal and S. J. Wright, Numerical Optimization, 2$^nd$ edition, Springer Series in Operations Research and Financial Engineering, Springer, New York, 2006.

[24]

D. Stefanatos, Optimal shortcuts to adiabaticity for a quantum piston, Automatica, 49 (2013), 3079-3083.  doi: 10.1016/j.automatica.2013.07.020.

[25]

R. F. StengelR. GhigliazzaN. Kulkarni and O. Laplace, Optimal control of innate immune response, Optimal Control Applications and Methods, 23 (2002), 91-104.  doi: 10.1002/oca.704.

[26]

K. L. Teo, C. J. Goh and K. H. Wong, A Unified Computational Approach to Optimal Control Problems, Longman Scientific and Technical, 1991.

[27]

T. L. Vincent, W. J. Grantham and W. Stadler, Optimality in Parametric Systems, American Society of Mechanical Engineers Digital Collection, 1983.

[28]

L. WangJ. YuanC. Wu and X. Wang, Practical algorithm for stochastic optimal control problem about microbial fermentation in batch culture, Optimization Letters, 13 (2019), 527-541.  doi: 10.1007/s11590-017-1220-z.

[29]

K. H. WongL. S. Jennings and F. Benyah, The control parametrization enhancing transform for constrained time–delayed optimal control problems, ANZIAM Journal, 43 (2001), 154-185. 

[30]

D. WuY. Bai and F. Xie, Time-scaling transformation for optimal control problem with time-varying delay, Discrete and Continuous Dynamical Systems-S, 13 (2020), 1683-1695.  doi: 10.3934/dcdss.2020098.

[31]

D. WuY. Bai and C. Yu, A new computational approach for optimal control problems with multiple time-delay, Automatica, 101 (2019), 388-395.  doi: 10.1016/j.automatica.2018.12.036.

[32]

X. Xu and P. J. Antsaklis, Optimal control of switched systems based on parameterization of the switching instants, IEEE Transactions on Automatic Control, 49 (2004), 2-16.  doi: 10.1109/TAC.2003.821417.

[33]

C. YuB. LiR. Loxton and K. L. Teo, Optimal discrete-valued control computation, Journal of Global Optimization, 56 (2013), 503-518.  doi: 10.1007/s10898-012-9858-7.

[34]

C. YuQ. LinR. LoxtonK. L. Teo and G. Wang, A hybrid time-scaling transformation for time-delay optimal control problems, Journal of Optimization Theory and Applications, 169 (2016), 876-901.  doi: 10.1007/s10957-015-0783-z.

[35]

N. ZhangC. -J. Yu and F. -S. Xie, The time-scaling transformation technique for optimal control problems with time-varying time-delay switched systems, Journal of the Operations Research Society of China, 8 (2020), 581-600.  doi: 10.1007/s40305-020-00299-5.

Table 3.  Experimental results of co-state method and variational method
Problem numbers of partitions co-state method CPU time(s) variational method CPU time(s) optimal cost
Prob 1 p=4 9.62 11.98 1.7500
p=6 18.04 29.49 1.7407
p=8 26.14 31.73 1.7405
p=12 61.41 117.46 1.7403
Prob 2 p=5 10.2 147.23 2.4046
Prob 3 p=10 183 2702 2.0356
Prob 4 p=1 1.180 1.729 0.0218
p=3 4.485 22.39 0.0176
p=6 12.92 329.01 0.0142
Prob 5 p=5 8.72 51.56 2.1502
Problem numbers of partitions co-state method CPU time(s) variational method CPU time(s) optimal cost
Prob 1 p=4 9.62 11.98 1.7500
p=6 18.04 29.49 1.7407
p=8 26.14 31.73 1.7405
p=12 61.41 117.46 1.7403
Prob 2 p=5 10.2 147.23 2.4046
Prob 3 p=10 183 2702 2.0356
Prob 4 p=1 1.180 1.729 0.0218
p=3 4.485 22.39 0.0176
p=6 12.92 329.01 0.0142
Prob 5 p=5 8.72 51.56 2.1502
Table 1.  Parameters in Problem 2
a b c $ t_f $ Q R S
0.2 0.5 0.2 1.5 $ I_{2\times2} $ $ I_{2\times2} $ $ 10^4I_{2\times2} $
a b c $ t_f $ Q R S
0.2 0.5 0.2 1.5 $ I_{2\times2} $ $ I_{2\times2} $ $ 10^4I_{2\times2} $
Table 2.  Parameters in Problemk 5
a b c h $ t_f $ Q R S
0.2 0.5 0.2 1 2 $ I_{2\times2} $ $ I_{2\times2} $ $ 10^4I_{2\times2} $
a b c h $ t_f $ Q R S
0.2 0.5 0.2 1 2 $ I_{2\times2} $ $ I_{2\times2} $ $ 10^4I_{2\times2} $
[1]

Di Wu, Yanqin Bai, Fusheng Xie. Time-scaling transformation for optimal control problem with time-varying delay. Discrete and Continuous Dynamical Systems - S, 2020, 13 (6) : 1683-1695. doi: 10.3934/dcdss.2020098

[2]

Di Wu, Yin Chen, Changjun Yu, Yanqin Bai, Kok Lay Teo. Control parameterization approach to time-delay optimal control problems: A survey. Journal of Industrial and Management Optimization, 2022  doi: 10.3934/jimo.2022108

[3]

Linna Li, Changjun Yu, Ning Zhang, Yanqin Bai, Zhiyuan Gao. A time-scaling technique for time-delay switched systems. Discrete and Continuous Dynamical Systems - S, 2020, 13 (6) : 1825-1843. doi: 10.3934/dcdss.2020108

[4]

Jérome Lohéac, Jean-François Scheid. Time optimal control for a nonholonomic system with state constraint. Mathematical Control and Related Fields, 2013, 3 (2) : 185-208. doi: 10.3934/mcrf.2013.3.185

[5]

Chongyang Liu, Meijia Han. Time-delay optimal control of a fed-batch production involving multiple feeds. Discrete and Continuous Dynamical Systems - S, 2020, 13 (6) : 1697-1709. doi: 10.3934/dcdss.2020099

[6]

Canghua Jiang, Cheng Jin, Ming Yu, Zongqi Xu. Direct optimal control for time-delay systems via a lifted multiple shooting algorithm. Journal of Industrial and Management Optimization, 2021  doi: 10.3934/jimo.2021135

[7]

Maoli Chen, Xiao Wang, Yicheng Liu. Collision-free flocking for a time-delay system. Discrete and Continuous Dynamical Systems - B, 2021, 26 (2) : 1223-1241. doi: 10.3934/dcdsb.2020251

[8]

Juanjuan Huang, Yan Zhou, Xuerong Shi, Zuolei Wang. A single finite-time synchronization scheme of time-delay chaotic system with external periodic disturbance. Mathematical Foundations of Computing, 2019, 2 (4) : 333-346. doi: 10.3934/mfc.2019021

[9]

Richard H. Rand, Asok K. Sen. A numerical investigation of the dynamics of a system of two time-delay coupled relaxation oscillators. Communications on Pure and Applied Analysis, 2003, 2 (4) : 567-577. doi: 10.3934/cpaa.2003.2.567

[10]

Zhong-Jie Han, Gen-Qi Xu. Dynamical behavior of networks of non-uniform Timoshenko beams system with boundary time-delay inputs. Networks and Heterogeneous Media, 2011, 6 (2) : 297-327. doi: 10.3934/nhm.2011.6.297

[11]

Qun Lin, Ryan Loxton, Kok Lay Teo. The control parameterization method for nonlinear optimal control: A survey. Journal of Industrial and Management Optimization, 2014, 10 (1) : 275-309. doi: 10.3934/jimo.2014.10.275

[12]

Luís Tiago Paiva, Fernando A. C. C. Fontes. Adaptive time--mesh refinement in optimal control problems with state constraints. Discrete and Continuous Dynamical Systems, 2015, 35 (9) : 4553-4572. doi: 10.3934/dcds.2015.35.4553

[13]

Gastão S. F. Frederico, Delfim F. M. Torres. Noether's symmetry Theorem for variational and optimal control problems with time delay. Numerical Algebra, Control and Optimization, 2012, 2 (3) : 619-630. doi: 10.3934/naco.2012.2.619

[14]

Jingtao Shi, Juanjuan Xu, Huanshui Zhang. Stochastic recursive optimal control problem with time delay and applications. Mathematical Control and Related Fields, 2015, 5 (4) : 859-888. doi: 10.3934/mcrf.2015.5.859

[15]

Akram Kheirabadi, Asadollah Mahmoudzadeh Vaziri, Sohrab Effati. Linear optimal control of time delay systems via Hermite wavelet. Numerical Algebra, Control and Optimization, 2020, 10 (2) : 143-156. doi: 10.3934/naco.2019044

[16]

Hai Huang, Xianlong Fu. Optimal control problems for a neutral integro-differential system with infinite delay. Evolution Equations and Control Theory, 2022, 11 (1) : 177-197. doi: 10.3934/eect.2020107

[17]

Alex Bombrun, Jean-Baptiste Pomet. Asymptotic behavior of time optimal orbital transfer for low thrust 2-body control system. Conference Publications, 2007, 2007 (Special) : 122-129. doi: 10.3934/proc.2007.2007.122

[18]

Ting Kang, Qimin Zhang, Haiyan Wang. Optimal control of an avian influenza model with multiple time delays in state and control variables. Discrete and Continuous Dynamical Systems - B, 2021, 26 (8) : 4147-4171. doi: 10.3934/dcdsb.2020278

[19]

Changjun Yu, Shuxuan Su, Yanqin Bai. On the optimal control problems with characteristic time control constraints. Journal of Industrial and Management Optimization, 2022, 18 (2) : 1305-1320. doi: 10.3934/jimo.2021021

[20]

Stefano Galatolo, Mark Holland, Tomas Persson, Yiwei Zhang. Anomalous time-scaling of extreme events in infinite systems and Birkhoff sums of infinite observables. Discrete and Continuous Dynamical Systems, 2021, 41 (4) : 1799-1841. doi: 10.3934/dcds.2020341

2021 Impact Factor: 1.411

Metrics

  • PDF downloads (505)
  • HTML views (544)
  • Cited by (0)

Other articles
by authors

[Back to Top]