American Institute of Mathematical Sciences

Advanced
May  2022, 18(3): 1583-1602. doi: 10.3934/jimo.2021034

Linear-quadratic optimal control for discrete-time stochastic descriptor systems

Yadong Shu 1,2,, and  Bo Li 1,2,

1. 

School of Mathematics and Statistics, Nanjing University of Information Science and Technology, Nanjing 210044, China
2. 

School of Applied Mathematics, Nanjing University of Finance and Economics, Nanjing 210023, China

* Corresponding author

Received  October 2020 Revised  December 2020 Published  May 2022 Early access  February 2021

In this paper, an optimal control model ruled by a class of linear discrete-time stochastic descriptor systems is considered under quadratic index performance. Employing dynamic programming method, a recurrence equation to simplify the optimal control problem is presented provided that the descriptor systems are both regular and impulse-free. When the objective function is quadratic, according to the recurrence equation, a discrete-time linear-quadratic optimal control problem is completely settled, that is, optimal controls and optimal values of the problem are both obtained through analytical expressions. At last, a numerical example about linear-quadratic optimal control for a discrete-time stochastic descriptor system is provided to illustrate the validness of the results derived.

Keywords: Linear-quadratic optimal control, stochastic descriptor systems, dynamic programming method, recurrence equation, sub-state feedback control.
Mathematics Subject Classification: 49N10, 93E20.
Citation: Yadong Shu, Bo Li. Linear-quadratic optimal control for discrete-time stochastic descriptor systems. Journal of Industrial and Management Optimization, 2022, 18 (3) : 1583-1602. doi: 10.3934/jimo.2021034
References:
[1]

K. Avrachenkov, O. Habachi, A. Piunovskiy and Y. Zhang, Infinite horizon optimal impulsive control with applications to Internet congestion control, International Journal of Control, 88 (2015), 703-716.  doi: 10.1080/00207179.2014.971436.

[2]

A. V. Balakrishnan, On stochastic bang-bang control, Applied Mathematics and Optimization, 6 (1980), 91-96.  doi: 10.1007/BF01442885.

[3]

M. Basin and and J. Rodriguez-Gonzalez, Optimal control for linear systems with time delay in control input, IEEE Transactions on Automatic Control, 51 (2006), 91-97.  doi: 10.1109/TAC.2005.861718.

[4]

A. Cairns, Some notes on the dynamics and optimal control of stochastic pension fund models in continuous time, Astin Bulletin, 30 (2000), 19-55.  doi: 10.2143/AST.30.1.504625.

[5]

S. L. Campbell, Singular Systems of Differential Equations II, Pitman, Boston, Mass.-London, 1982.

[6]

M. Chadli, H. R. Karimi and P. Shi, On stability and stabilization of singular uncertain Takagi-Sugeno fuzzy systems, Journal of the Franklin Institute, 351 (2014), 1453-1463.  doi: 10.1016/j.jfranklin.2013.11.008.

[7]

L. Chen and Z. Wu, Maximum principle for the stochastic optimal control problem with delay and application, Automatica J. IFAC, 46 (2010), 1074-1080.  doi: 10.1016/j.automatica.2010.03.005.

[8]

D. Cobb, Controllability, observability, and duality in singular systems, IEEE Transactions on Automatic Control, 29 (1984), 1076-1082.  doi: 10.1109/TAC.1984.1103451.

[9]

L. Dai, Singular Control Systems, Springer-Verlag, Berlin, Germany, 1989. doi: 10.1007/BFb0002475.

[10]

J.-e Feng, P. Cui and Z. Hou, Singular linear quadratic optimal control for singular stochastic discrete-time systems, Optimal Control Applications and Methods, 34 (2013), 505-516.  doi: 10.1002/oca.2033.

[11]

J. Y. Ishihara and M. H. Terra, On the Lyapunov theorem for singular systems, IEEE Transactions on Automatic Control, 47 (2002), 1926-1930.  doi: 10.1109/TAC.2002.804463.

[12]

R. E. Kalman, Contribution to the theory of optimal control, Boletin Sociedad Matem$\acute{a}$tica Mexicana, 5 (1960), 102–119.

[13]

Y. Kang and Y. Zhu, Bang-bang optimal control for multi-stage uncertain systems, International Journal on Information, 15 (2012), 3229-3237. 

[14]

D. E. Kirk, Optimal Control Theory: An Introduction, Courier Corporation, 2012.

[15]

N. Kumaresan and P. Balasubramaniam, Optimal control for stochastic nonlinear singular system using neural networks, Computers and Mathematics with Applications, 56 (2008), 2145-2154.  doi: 10.1016/j.camwa.2008.03.041.

[16]

P. Kunkel and V. Mehrmann, The linear quadratic optimal control problem for linear descriptor systems with variable coefficients, Mathematics of Control, Signals, and Systems, 10 (1997), 247-264.  doi: 10.1007/BF01211506.

[17]

B. Li and and Y. Zhu, The piecewise parametric optimal control of uncertain linear quadratic models, International Journal of Systems Science, 50 (2019), 961-969.  doi: 10.1080/00207721.2019.1586003.

[18]

F. Liao, Z. Ren, M. Tomizuka and J. Wu, Preview control for impulse-free continuous-time descriptor systems, International Journal of Control, 88 (2015), 1142-1149.  doi: 10.1080/00207179.2014.996769.

[19]

J. Lian, C. Li and D. Liu, Input-to-state stability for discrete-time nonlinear switched singular systems, IET Control Theory Appl., 11 (2017), 2893–2899. doi: 10.1049/iet-cta.2017.0028.

[20]

W. Liu, X. Liang, Y. Ma and W. Liu, Aircraft trajectory optimization for collision avoidance using stochastic optimal control, Asian Journal of Control, 21 (2019), 2308-2320.  doi: 10.1002/asjc.1855.

[21]

D. G. Luenberger, Dynamic equations in descriptor form, IEEE Transactions on Automatic Control, 22 (1977), 312-321.  doi: 10.1109/tac.1977.1101502.

[22]

D. G. Luenberger and A. Arbel, Singular dynamic Leontief systems, Econometrica, 45 (1977), 991-995.  doi: 10.2307/1912686.

[23]

G. P. Mantas and N. J. Krikelis, LQ optimal control for discrete descriptor systems with a complete set of boundary information, International Journal of Control, 47 (1988), 1467-1477.  doi: 10.1080/00207178808906108.

[24]

D. S. Naidu, Singular perturbations and time scales in control theory and applications: An overview., Dynamics of Continuous Discrete & Impulsive Systems Ser. B Appl. Algorithms, 9 (2002), 233-278. 

[25]

H. Peng, Fei. Li, J. Liu and Z. Ju, A symplectic instantaneous optimal control for robot trajectory tracking with differential-algebraic equation models, IEEE Transactions on Industrial Electronics, 67 (2020), 3819-3829.  doi: 10.1109/TIE.2019.2916390.

[26]

Y. Shu and Y. Zhu, Optimistic value based optimal control for uncertain linear singular systems and application to a dynamic input-output model, ISA Transactions, 71 (2017), 235-251.  doi: 10.1016/j.isatra.2017.08.007.

[27]

Y. Shu and Y. Zhu, Stability analysis of uncertain singular systems, Soft Computing, 22 (2018), 5671-5681.  doi: 10.1007/s00500-017-2599-2.

[28]

K. Wang, L. Xie and Y. Liu, Characteristics of self-excited oscillation in a curved diffuser., Journal of Propulsion Technology, 39 (2018), 1955-1964. 

[29]

W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM Journal on Control, 6 (1968), 681-697.  doi: 10.1137/0306044.

[30]

H. Yan, Y. Sun and Y. Zhu, A linear-quadratic control problem of uncertain discrete-time switched systems, Journal of Industrial and Management Optimization, 13 (2017), 267-282.  doi: 10.3934/jimo.2016016.

[31]

W. Zhang, J. Hu and A. Abate, On the value function of the discrete-time switched LQR problem, IEEE Transactions on Automatic Control, 54 (2009), 2669-2674.  doi: 10.1109/TAC.2009.2031574.

[32]

W. Zhang, Y. Lin and L. Xue, Linear quadratic Pareto optimal control problem of stochastic singular systems, Journal of the Franklin Institute, 354 (2017), 1220-1238.  doi: 10.1016/j.jfranklin.2016.11.021.

show all references

References:
[1]

K. Avrachenkov, O. Habachi, A. Piunovskiy and Y. Zhang, Infinite horizon optimal impulsive control with applications to Internet congestion control, International Journal of Control, 88 (2015), 703-716.  doi: 10.1080/00207179.2014.971436.

[2]

A. V. Balakrishnan, On stochastic bang-bang control, Applied Mathematics and Optimization, 6 (1980), 91-96.  doi: 10.1007/BF01442885.

[3]

M. Basin and and J. Rodriguez-Gonzalez, Optimal control for linear systems with time delay in control input, IEEE Transactions on Automatic Control, 51 (2006), 91-97.  doi: 10.1109/TAC.2005.861718.

[4]

A. Cairns, Some notes on the dynamics and optimal control of stochastic pension fund models in continuous time, Astin Bulletin, 30 (2000), 19-55.  doi: 10.2143/AST.30.1.504625.

[5]

S. L. Campbell, Singular Systems of Differential Equations II, Pitman, Boston, Mass.-London, 1982.

[6]

M. Chadli, H. R. Karimi and P. Shi, On stability and stabilization of singular uncertain Takagi-Sugeno fuzzy systems, Journal of the Franklin Institute, 351 (2014), 1453-1463.  doi: 10.1016/j.jfranklin.2013.11.008.

[7]

L. Chen and Z. Wu, Maximum principle for the stochastic optimal control problem with delay and application, Automatica J. IFAC, 46 (2010), 1074-1080.  doi: 10.1016/j.automatica.2010.03.005.

[8]

D. Cobb, Controllability, observability, and duality in singular systems, IEEE Transactions on Automatic Control, 29 (1984), 1076-1082.  doi: 10.1109/TAC.1984.1103451.

[9]

L. Dai, Singular Control Systems, Springer-Verlag, Berlin, Germany, 1989. doi: 10.1007/BFb0002475.

[10]

J.-e Feng, P. Cui and Z. Hou, Singular linear quadratic optimal control for singular stochastic discrete-time systems, Optimal Control Applications and Methods, 34 (2013), 505-516.  doi: 10.1002/oca.2033.

[11]

J. Y. Ishihara and M. H. Terra, On the Lyapunov theorem for singular systems, IEEE Transactions on Automatic Control, 47 (2002), 1926-1930.  doi: 10.1109/TAC.2002.804463.

[12]

R. E. Kalman, Contribution to the theory of optimal control, Boletin Sociedad Matem$\acute{a}$tica Mexicana, 5 (1960), 102–119.

[13]

Y. Kang and Y. Zhu, Bang-bang optimal control for multi-stage uncertain systems, International Journal on Information, 15 (2012), 3229-3237. 

[14]

D. E. Kirk, Optimal Control Theory: An Introduction, Courier Corporation, 2012.

[15]

N. Kumaresan and P. Balasubramaniam, Optimal control for stochastic nonlinear singular system using neural networks, Computers and Mathematics with Applications, 56 (2008), 2145-2154.  doi: 10.1016/j.camwa.2008.03.041.

[16]

P. Kunkel and V. Mehrmann, The linear quadratic optimal control problem for linear descriptor systems with variable coefficients, Mathematics of Control, Signals, and Systems, 10 (1997), 247-264.  doi: 10.1007/BF01211506.

[17]

B. Li and and Y. Zhu, The piecewise parametric optimal control of uncertain linear quadratic models, International Journal of Systems Science, 50 (2019), 961-969.  doi: 10.1080/00207721.2019.1586003.

[18]

F. Liao, Z. Ren, M. Tomizuka and J. Wu, Preview control for impulse-free continuous-time descriptor systems, International Journal of Control, 88 (2015), 1142-1149.  doi: 10.1080/00207179.2014.996769.

[19]

J. Lian, C. Li and D. Liu, Input-to-state stability for discrete-time nonlinear switched singular systems, IET Control Theory Appl., 11 (2017), 2893–2899. doi: 10.1049/iet-cta.2017.0028.

[20]

W. Liu, X. Liang, Y. Ma and W. Liu, Aircraft trajectory optimization for collision avoidance using stochastic optimal control, Asian Journal of Control, 21 (2019), 2308-2320.  doi: 10.1002/asjc.1855.

[21]

D. G. Luenberger, Dynamic equations in descriptor form, IEEE Transactions on Automatic Control, 22 (1977), 312-321.  doi: 10.1109/tac.1977.1101502.

[22]

D. G. Luenberger and A. Arbel, Singular dynamic Leontief systems, Econometrica, 45 (1977), 991-995.  doi: 10.2307/1912686.

[23]

G. P. Mantas and N. J. Krikelis, LQ optimal control for discrete descriptor systems with a complete set of boundary information, International Journal of Control, 47 (1988), 1467-1477.  doi: 10.1080/00207178808906108.

[24]

D. S. Naidu, Singular perturbations and time scales in control theory and applications: An overview., Dynamics of Continuous Discrete & Impulsive Systems Ser. B Appl. Algorithms, 9 (2002), 233-278. 

[25]

H. Peng, Fei. Li, J. Liu and Z. Ju, A symplectic instantaneous optimal control for robot trajectory tracking with differential-algebraic equation models, IEEE Transactions on Industrial Electronics, 67 (2020), 3819-3829.  doi: 10.1109/TIE.2019.2916390.

[26]

Y. Shu and Y. Zhu, Optimistic value based optimal control for uncertain linear singular systems and application to a dynamic input-output model, ISA Transactions, 71 (2017), 235-251.  doi: 10.1016/j.isatra.2017.08.007.

[27]

Y. Shu and Y. Zhu, Stability analysis of uncertain singular systems, Soft Computing, 22 (2018), 5671-5681.  doi: 10.1007/s00500-017-2599-2.

[28]

K. Wang, L. Xie and Y. Liu, Characteristics of self-excited oscillation in a curved diffuser., Journal of Propulsion Technology, 39 (2018), 1955-1964. 

[29]

W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM Journal on Control, 6 (1968), 681-697.  doi: 10.1137/0306044.

[30]

H. Yan, Y. Sun and Y. Zhu, A linear-quadratic control problem of uncertain discrete-time switched systems, Journal of Industrial and Management Optimization, 13 (2017), 267-282.  doi: 10.3934/jimo.2016016.

[31]

W. Zhang, J. Hu and A. Abate, On the value function of the discrete-time switched LQR problem, IEEE Transactions on Automatic Control, 54 (2009), 2669-2674.  doi: 10.1109/TAC.2009.2031574.

[32]

W. Zhang, Y. Lin and L. Xue, Linear quadratic Pareto optimal control problem of stochastic singular systems, Journal of the Franklin Institute, 354 (2017), 1220-1238.  doi: 10.1016/j.jfranklin.2016.11.021.

Figure 1.  Trajectories of optimal controls $ u^*(k) $ for problem (11)
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2.  Trajectories of state vectors $ x(k) $ for problem (11)
Figure Options

Download full-size image

Download as PowerPoint slide

Table 1.  The optimal results of problem (11)
Stage $ (u^{*}(k))^T $ $ \mu_k $ $ x_1^T(k) $ $ x^T(k) $ $ J\big(k, x_1(k), x_2(7)\big) $
0 $ (3.567, -2.323, -5.406) $ $ 2.517 $ $ (1, 2) $ $ (1.000, -5.434, 2.000, -4.600) $ $ -112.722 $
1 $ (0.338, -2.541, -0.797) $ $ 1.743 $ $ (-4.756, 0.145) $ $ (-4.756, -2.699, 0.145, -4.756) $ $ -55.708 $
2 $ (1.796, -2.599, -0.303) $ $ 2.098 $ $ (-1.416, -4.148) $ $ (-1.416, -5.019, -4.148, -1.218) $ $ 20.114 $
3 $ (0.167, -1.651, 0.469) $ $ 0.202 $ $ (-1.429, -1.614) $ $ (-1.429, -1.769, -1.614, 0.490) $ $ 41.636 $
4 $ (0.635, -1.194, 0.582) $ $ 1.165 $ $ (-0.931, -2.843) $ $ (-0.931, -2.743, -2.843, -0.320) $ $ 39.171 $
5 $ (0.862, -0.338, -0.251) $ $ 0.872 $ $ (-0.362, -0.430) $ $ (-0.362, -1.988, -0.430, -1.147) $ $ 32.059 $
6 $ (0.390, -0.274, -0.403) $ $ 1.394 $ $ (-0.505, 0.182) $ $ (-0.505, -2.598, 0.182, -2.027) $ $ 46.328 $
7 $ u^{*}(7)\in R^3 $ $ (-1.217, 1.619) $ $ (-1.217, 0.598, 1.619, -0.217) $ $ 7.363 $
Table Options

Download as excel

Stage $ (u^{*}(k))^T $ $ \mu_k $ $ x_1^T(k) $ $ x^T(k) $ $ J\big(k, x_1(k), x_2(7)\big) $
0 $ (3.567, -2.323, -5.406) $ $ 2.517 $ $ (1, 2) $ $ (1.000, -5.434, 2.000, -4.600) $ $ -112.722 $
1 $ (0.338, -2.541, -0.797) $ $ 1.743 $ $ (-4.756, 0.145) $ $ (-4.756, -2.699, 0.145, -4.756) $ $ -55.708 $
2 $ (1.796, -2.599, -0.303) $ $ 2.098 $ $ (-1.416, -4.148) $ $ (-1.416, -5.019, -4.148, -1.218) $ $ 20.114 $
3 $ (0.167, -1.651, 0.469) $ $ 0.202 $ $ (-1.429, -1.614) $ $ (-1.429, -1.769, -1.614, 0.490) $ $ 41.636 $
4 $ (0.635, -1.194, 0.582) $ $ 1.165 $ $ (-0.931, -2.843) $ $ (-0.931, -2.743, -2.843, -0.320) $ $ 39.171 $
5 $ (0.862, -0.338, -0.251) $ $ 0.872 $ $ (-0.362, -0.430) $ $ (-0.362, -1.988, -0.430, -1.147) $ $ 32.059 $
6 $ (0.390, -0.274, -0.403) $ $ 1.394 $ $ (-0.505, 0.182) $ $ (-0.505, -2.598, 0.182, -2.027) $ $ 46.328 $
7 $ u^{*}(7)\in R^3 $ $ (-1.217, 1.619) $ $ (-1.217, 0.598, 1.619, -0.217) $ $ 7.363 $
[1]

Mohamed Aliane, Mohand Bentobache, Nacima Moussouni, Philippe Marthon. Direct method to solve linear-quadratic optimal control problems. Numerical Algebra, Control and Optimization, 2021, 11 (4) : 645-663. doi: 10.3934/naco.2021002
[2]

Galina Kurina, Sahlar Meherrem. Decomposition of discrete linear-quadratic optimal control problems for switching systems. Conference Publications, 2015, 2015 (special) : 764-774. doi: 10.3934/proc.2015.0764
[3]

Nguyen Thi Hoai. Asymptotic approximation to a solution of a singularly perturbed linear-quadratic optimal control problem with second-order linear ordinary differential equation of state variable. Numerical Algebra, Control and Optimization, 2021, 11 (4) : 495-512. doi: 10.3934/naco.2020040
[4]

Russell Johnson, Carmen Núñez. Remarks on linear-quadratic dissipative control systems. Discrete and Continuous Dynamical Systems - B, 2015, 20 (3) : 889-914. doi: 10.3934/dcdsb.2015.20.889
[5]

Qi Lü, Tianxiao Wang, Xu Zhang. Characterization of optimal feedback for stochastic linear quadratic control problems. Probability, Uncertainty and Quantitative Risk, 2017, 2 (0) : 11-. doi: 10.1186/s41546-017-0022-7
[6]

Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control and Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97
[7]

Hanxiao Wang, Jingrui Sun, Jiongmin Yong. Weak closed-loop solvability of stochastic linear-quadratic optimal control problems. Discrete and Continuous Dynamical Systems, 2019, 39 (5) : 2785-2805. doi: 10.3934/dcds.2019117
[8]

Jingrui Sun, Hanxiao Wang. Mean-field stochastic linear-quadratic optimal control problems: Weak closed-loop solvability. Mathematical Control and Related Fields, 2021, 11 (1) : 47-71. doi: 10.3934/mcrf.2020026
[9]

Ying Hu, Shanjian Tang. Mixed deterministic and random optimal control of linear stochastic systems with quadratic costs. Probability, Uncertainty and Quantitative Risk, 2019, 4 (0) : 1-. doi: 10.1186/s41546-018-0035-x
[10]

Rein Luus. Optimal control of oscillatory systems by iterative dynamic programming. Journal of Industrial and Management Optimization, 2008, 4 (1) : 1-15. doi: 10.3934/jimo.2008.4.1
[11]

Shigeaki Koike, Hiroaki Morimoto, Shigeru Sakaguchi. A linear-quadratic control problem with discretionary stopping. Discrete and Continuous Dynamical Systems - B, 2007, 8 (2) : 261-277. doi: 10.3934/dcdsb.2007.8.261
[12]

Haibo Jin, Long Hai, Xiaoliang Tang. An optimal maintenance strategy for multi-state systems based on a system linear integral equation and dynamic programming. Journal of Industrial and Management Optimization, 2020, 16 (2) : 965-990. doi: 10.3934/jimo.2018188
[13]

Georg Vossen, Stefan Volkwein. Model reduction techniques with a-posteriori error analysis for linear-quadratic optimal control problems. Numerical Algebra, Control and Optimization, 2012, 2 (3) : 465-485. doi: 10.3934/naco.2012.2.465
[14]

Changzhi Wu, Kok Lay Teo, Volker Rehbock. Optimal control of piecewise affine systems with piecewise affine state feedback. Journal of Industrial and Management Optimization, 2009, 5 (4) : 737-747. doi: 10.3934/jimo.2009.5.737
[15]

Hongyan Yan, Yun Sun, Yuanguo Zhu. A linear-quadratic control problem of uncertain discrete-time switched systems. Journal of Industrial and Management Optimization, 2017, 13 (1) : 267-282. doi: 10.3934/jimo.2016016
[16]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437
[17]

Roberta Fabbri, Russell Johnson, Sylvia Novo, Carmen Núñez. On linear-quadratic dissipative control processes with time-varying coefficients. Discrete and Continuous Dynamical Systems, 2013, 33 (1) : 193-210. doi: 10.3934/dcds.2013.33.193
[18]

Rohit Gupta, Farhad Jafari, Robert J. Kipka, Boris S. Mordukhovich. Linear openness and feedback stabilization of nonlinear control systems. Discrete and Continuous Dynamical Systems - S, 2018, 11 (6) : 1103-1119. doi: 10.3934/dcdss.2018063
[19]

Tijana Levajković, Hermann Mena, Amjad Tuffaha. The stochastic linear quadratic optimal control problem in Hilbert spaces: A polynomial chaos approach. Evolution Equations and Control Theory, 2016, 5 (1) : 105-134. doi: 10.3934/eect.2016.5.105
[20]

Xun Li, Jingrui Sun, Jiongmin Yong. Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability. Probability, Uncertainty and Quantitative Risk, 2016, 1 (0) : 2-. doi: 10.1186/s41546-016-0002-3

2021 Impact Factor: 1.411

Tools

Metrics

  • PDF downloads (479)
  • HTML views (538)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2022 American Institute of Mathematical Sciences | Privacy Policy