American Institute of Mathematical Sciences

Advanced
doi: 10.3934/jimo.2021034

Linear-quadratic optimal control for discrete-time stochastic descriptor systems

Yadong Shu 1,2,, and  Bo Li 1,2,

1. 

School of Mathematics and Statistics, Nanjing University of Information Science and Technology, Nanjing 210044, China
2. 

School of Applied Mathematics, Nanjing University of Finance and Economics, Nanjing 210023, China

* Corresponding author

Received  October 2020 Revised  December 2020 Published  February 2021

In this paper, an optimal control model ruled by a class of linear discrete-time stochastic descriptor systems is considered under quadratic index performance. Employing dynamic programming method, a recurrence equation to simplify the optimal control problem is presented provided that the descriptor systems are both regular and impulse-free. When the objective function is quadratic, according to the recurrence equation, a discrete-time linear-quadratic optimal control problem is completely settled, that is, optimal controls and optimal values of the problem are both obtained through analytical expressions. At last, a numerical example about linear-quadratic optimal control for a discrete-time stochastic descriptor system is provided to illustrate the validness of the results derived.

Keywords: Linear-quadratic optimal control, stochastic descriptor systems, dynamic programming method, recurrence equation, sub-state feedback control.
Mathematics Subject Classification: 49N10, 93E20.
Citation: Yadong Shu, Bo Li. Linear-quadratic optimal control for discrete-time stochastic descriptor systems. Journal of Industrial & Management Optimization, doi: 10.3934/jimo.2021034
References:
[1]

K. Avrachenkov, O. Habachi, A. Piunovskiy and Y. Zhang, Infinite horizon optimal impulsive control with applications to Internet congestion control, International Journal of Control, 88 (2015), 703-716.  doi: 10.1080/00207179.2014.971436.  Google Scholar

[2]

A. V. Balakrishnan, On stochastic bang-bang control, Applied Mathematics and Optimization, 6 (1980), 91-96.  doi: 10.1007/BF01442885.  Google Scholar

[3]

M. Basin and and J. Rodriguez-Gonzalez, Optimal control for linear systems with time delay in control input, IEEE Transactions on Automatic Control, 51 (2006), 91-97.  doi: 10.1109/TAC.2005.861718.  Google Scholar

[4]

A. Cairns, Some notes on the dynamics and optimal control of stochastic pension fund models in continuous time, Astin Bulletin, 30 (2000), 19-55.  doi: 10.2143/AST.30.1.504625.  Google Scholar

[5]

S. L. Campbell, Singular Systems of Differential Equations II, Pitman, Boston, Mass.-London, 1982.  Google Scholar

[6]

M. Chadli, H. R. Karimi and P. Shi, On stability and stabilization of singular uncertain Takagi-Sugeno fuzzy systems, Journal of the Franklin Institute, 351 (2014), 1453-1463.  doi: 10.1016/j.jfranklin.2013.11.008.  Google Scholar

[7]

L. Chen and Z. Wu, Maximum principle for the stochastic optimal control problem with delay and application, Automatica J. IFAC, 46 (2010), 1074-1080.  doi: 10.1016/j.automatica.2010.03.005.  Google Scholar

[8]

D. Cobb, Controllability, observability, and duality in singular systems, IEEE Transactions on Automatic Control, 29 (1984), 1076-1082.  doi: 10.1109/TAC.1984.1103451.  Google Scholar

[9]

L. Dai, Singular Control Systems, Springer-Verlag, Berlin, Germany, 1989. doi: 10.1007/BFb0002475.  Google Scholar

[10]

J.-e Feng, P. Cui and Z. Hou, Singular linear quadratic optimal control for singular stochastic discrete-time systems, Optimal Control Applications and Methods, 34 (2013), 505-516.  doi: 10.1002/oca.2033.  Google Scholar

[11]

J. Y. Ishihara and M. H. Terra, On the Lyapunov theorem for singular systems, IEEE Transactions on Automatic Control, 47 (2002), 1926-1930.  doi: 10.1109/TAC.2002.804463.  Google Scholar

[12]

R. E. Kalman, Contribution to the theory of optimal control, Boletin Sociedad Matem$\acute{a}$tica Mexicana, 5 (1960), 102–119.  Google Scholar

[13]

Y. Kang and Y. Zhu, Bang-bang optimal control for multi-stage uncertain systems, International Journal on Information, 15 (2012), 3229-3237.   Google Scholar

[14]

D. E. Kirk, Optimal Control Theory: An Introduction, Courier Corporation, 2012. Google Scholar

[15]

N. Kumaresan and P. Balasubramaniam, Optimal control for stochastic nonlinear singular system using neural networks, Computers and Mathematics with Applications, 56 (2008), 2145-2154.  doi: 10.1016/j.camwa.2008.03.041.  Google Scholar

[16]

P. Kunkel and V. Mehrmann, The linear quadratic optimal control problem for linear descriptor systems with variable coefficients, Mathematics of Control, Signals, and Systems, 10 (1997), 247-264.  doi: 10.1007/BF01211506.  Google Scholar

[17]

B. Li and and Y. Zhu, The piecewise parametric optimal control of uncertain linear quadratic models, International Journal of Systems Science, 50 (2019), 961-969.  doi: 10.1080/00207721.2019.1586003.  Google Scholar

[18]

F. Liao, Z. Ren, M. Tomizuka and J. Wu, Preview control for impulse-free continuous-time descriptor systems, International Journal of Control, 88 (2015), 1142-1149.  doi: 10.1080/00207179.2014.996769.  Google Scholar

[19]

J. Lian, C. Li and D. Liu, Input-to-state stability for discrete-time nonlinear switched singular systems, IET Control Theory Appl., 11 (2017), 2893–2899. doi: 10.1049/iet-cta.2017.0028.  Google Scholar

[20]

W. Liu, X. Liang, Y. Ma and W. Liu, Aircraft trajectory optimization for collision avoidance using stochastic optimal control, Asian Journal of Control, 21 (2019), 2308-2320.  doi: 10.1002/asjc.1855.  Google Scholar

[21]

D. G. Luenberger, Dynamic equations in descriptor form, IEEE Transactions on Automatic Control, 22 (1977), 312-321.  doi: 10.1109/tac.1977.1101502.  Google Scholar

[22]

D. G. Luenberger and A. Arbel, Singular dynamic Leontief systems, Econometrica, 45 (1977), 991-995.  doi: 10.2307/1912686.  Google Scholar

[23]

G. P. Mantas and N. J. Krikelis, LQ optimal control for discrete descriptor systems with a complete set of boundary information, International Journal of Control, 47 (1988), 1467-1477.  doi: 10.1080/00207178808906108.  Google Scholar

[24]

D. S. Naidu, Singular perturbations and time scales in control theory and applications: An overview., Dynamics of Continuous Discrete & Impulsive Systems Ser. B Appl. Algorithms, 9 (2002), 233-278.   Google Scholar

[25]

H. Peng, Fei. Li, J. Liu and Z. Ju, A symplectic instantaneous optimal control for robot trajectory tracking with differential-algebraic equation models, IEEE Transactions on Industrial Electronics, 67 (2020), 3819-3829.  doi: 10.1109/TIE.2019.2916390.  Google Scholar

[26]

Y. Shu and Y. Zhu, Optimistic value based optimal control for uncertain linear singular systems and application to a dynamic input-output model, ISA Transactions, 71 (2017), 235-251.  doi: 10.1016/j.isatra.2017.08.007.  Google Scholar

[27]

Y. Shu and Y. Zhu, Stability analysis of uncertain singular systems, Soft Computing, 22 (2018), 5671-5681.  doi: 10.1007/s00500-017-2599-2.  Google Scholar

[28]

K. Wang, L. Xie and Y. Liu, Characteristics of self-excited oscillation in a curved diffuser., Journal of Propulsion Technology, 39 (2018), 1955-1964.   Google Scholar

[29]

W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM Journal on Control, 6 (1968), 681-697.  doi: 10.1137/0306044.  Google Scholar

[30]

H. Yan, Y. Sun and Y. Zhu, A linear-quadratic control problem of uncertain discrete-time switched systems, Journal of Industrial and Management Optimization, 13 (2017), 267-282.  doi: 10.3934/jimo.2016016.  Google Scholar

[31]

W. Zhang, J. Hu and A. Abate, On the value function of the discrete-time switched LQR problem, IEEE Transactions on Automatic Control, 54 (2009), 2669-2674.  doi: 10.1109/TAC.2009.2031574.  Google Scholar

[32]

W. Zhang, Y. Lin and L. Xue, Linear quadratic Pareto optimal control problem of stochastic singular systems, Journal of the Franklin Institute, 354 (2017), 1220-1238.  doi: 10.1016/j.jfranklin.2016.11.021.  Google Scholar

show all references

References:
[1]

K. Avrachenkov, O. Habachi, A. Piunovskiy and Y. Zhang, Infinite horizon optimal impulsive control with applications to Internet congestion control, International Journal of Control, 88 (2015), 703-716.  doi: 10.1080/00207179.2014.971436.  Google Scholar

[2]

A. V. Balakrishnan, On stochastic bang-bang control, Applied Mathematics and Optimization, 6 (1980), 91-96.  doi: 10.1007/BF01442885.  Google Scholar

[3]

M. Basin and and J. Rodriguez-Gonzalez, Optimal control for linear systems with time delay in control input, IEEE Transactions on Automatic Control, 51 (2006), 91-97.  doi: 10.1109/TAC.2005.861718.  Google Scholar

[4]

A. Cairns, Some notes on the dynamics and optimal control of stochastic pension fund models in continuous time, Astin Bulletin, 30 (2000), 19-55.  doi: 10.2143/AST.30.1.504625.  Google Scholar

[5]

S. L. Campbell, Singular Systems of Differential Equations II, Pitman, Boston, Mass.-London, 1982.  Google Scholar

[6]

M. Chadli, H. R. Karimi and P. Shi, On stability and stabilization of singular uncertain Takagi-Sugeno fuzzy systems, Journal of the Franklin Institute, 351 (2014), 1453-1463.  doi: 10.1016/j.jfranklin.2013.11.008.  Google Scholar

[7]

L. Chen and Z. Wu, Maximum principle for the stochastic optimal control problem with delay and application, Automatica J. IFAC, 46 (2010), 1074-1080.  doi: 10.1016/j.automatica.2010.03.005.  Google Scholar

[8]

D. Cobb, Controllability, observability, and duality in singular systems, IEEE Transactions on Automatic Control, 29 (1984), 1076-1082.  doi: 10.1109/TAC.1984.1103451.  Google Scholar

[9]

L. Dai, Singular Control Systems, Springer-Verlag, Berlin, Germany, 1989. doi: 10.1007/BFb0002475.  Google Scholar

[10]

J.-e Feng, P. Cui and Z. Hou, Singular linear quadratic optimal control for singular stochastic discrete-time systems, Optimal Control Applications and Methods, 34 (2013), 505-516.  doi: 10.1002/oca.2033.  Google Scholar

[11]

J. Y. Ishihara and M. H. Terra, On the Lyapunov theorem for singular systems, IEEE Transactions on Automatic Control, 47 (2002), 1926-1930.  doi: 10.1109/TAC.2002.804463.  Google Scholar

[12]

R. E. Kalman, Contribution to the theory of optimal control, Boletin Sociedad Matem$\acute{a}$tica Mexicana, 5 (1960), 102–119.  Google Scholar

[13]

Y. Kang and Y. Zhu, Bang-bang optimal control for multi-stage uncertain systems, International Journal on Information, 15 (2012), 3229-3237.   Google Scholar

[14]

D. E. Kirk, Optimal Control Theory: An Introduction, Courier Corporation, 2012. Google Scholar

[15]

N. Kumaresan and P. Balasubramaniam, Optimal control for stochastic nonlinear singular system using neural networks, Computers and Mathematics with Applications, 56 (2008), 2145-2154.  doi: 10.1016/j.camwa.2008.03.041.  Google Scholar

[16]

P. Kunkel and V. Mehrmann, The linear quadratic optimal control problem for linear descriptor systems with variable coefficients, Mathematics of Control, Signals, and Systems, 10 (1997), 247-264.  doi: 10.1007/BF01211506.  Google Scholar

[17]

B. Li and and Y. Zhu, The piecewise parametric optimal control of uncertain linear quadratic models, International Journal of Systems Science, 50 (2019), 961-969.  doi: 10.1080/00207721.2019.1586003.  Google Scholar

[18]

F. Liao, Z. Ren, M. Tomizuka and J. Wu, Preview control for impulse-free continuous-time descriptor systems, International Journal of Control, 88 (2015), 1142-1149.  doi: 10.1080/00207179.2014.996769.  Google Scholar

[19]

J. Lian, C. Li and D. Liu, Input-to-state stability for discrete-time nonlinear switched singular systems, IET Control Theory Appl., 11 (2017), 2893–2899. doi: 10.1049/iet-cta.2017.0028.  Google Scholar

[20]

W. Liu, X. Liang, Y. Ma and W. Liu, Aircraft trajectory optimization for collision avoidance using stochastic optimal control, Asian Journal of Control, 21 (2019), 2308-2320.  doi: 10.1002/asjc.1855.  Google Scholar

[21]

D. G. Luenberger, Dynamic equations in descriptor form, IEEE Transactions on Automatic Control, 22 (1977), 312-321.  doi: 10.1109/tac.1977.1101502.  Google Scholar

[22]

D. G. Luenberger and A. Arbel, Singular dynamic Leontief systems, Econometrica, 45 (1977), 991-995.  doi: 10.2307/1912686.  Google Scholar

[23]

G. P. Mantas and N. J. Krikelis, LQ optimal control for discrete descriptor systems with a complete set of boundary information, International Journal of Control, 47 (1988), 1467-1477.  doi: 10.1080/00207178808906108.  Google Scholar

[24]

D. S. Naidu, Singular perturbations and time scales in control theory and applications: An overview., Dynamics of Continuous Discrete & Impulsive Systems Ser. B Appl. Algorithms, 9 (2002), 233-278.   Google Scholar

[25]

H. Peng, Fei. Li, J. Liu and Z. Ju, A symplectic instantaneous optimal control for robot trajectory tracking with differential-algebraic equation models, IEEE Transactions on Industrial Electronics, 67 (2020), 3819-3829.  doi: 10.1109/TIE.2019.2916390.  Google Scholar

[26]

Y. Shu and Y. Zhu, Optimistic value based optimal control for uncertain linear singular systems and application to a dynamic input-output model, ISA Transactions, 71 (2017), 235-251.  doi: 10.1016/j.isatra.2017.08.007.  Google Scholar

[27]

Y. Shu and Y. Zhu, Stability analysis of uncertain singular systems, Soft Computing, 22 (2018), 5671-5681.  doi: 10.1007/s00500-017-2599-2.  Google Scholar

[28]

K. Wang, L. Xie and Y. Liu, Characteristics of self-excited oscillation in a curved diffuser., Journal of Propulsion Technology, 39 (2018), 1955-1964.   Google Scholar

[29]

W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM Journal on Control, 6 (1968), 681-697.  doi: 10.1137/0306044.  Google Scholar

[30]

H. Yan, Y. Sun and Y. Zhu, A linear-quadratic control problem of uncertain discrete-time switched systems, Journal of Industrial and Management Optimization, 13 (2017), 267-282.  doi: 10.3934/jimo.2016016.  Google Scholar

[31]

W. Zhang, J. Hu and A. Abate, On the value function of the discrete-time switched LQR problem, IEEE Transactions on Automatic Control, 54 (2009), 2669-2674.  doi: 10.1109/TAC.2009.2031574.  Google Scholar

[32]

W. Zhang, Y. Lin and L. Xue, Linear quadratic Pareto optimal control problem of stochastic singular systems, Journal of the Franklin Institute, 354 (2017), 1220-1238.  doi: 10.1016/j.jfranklin.2016.11.021.  Google Scholar

Figure 1.  Trajectories of optimal controls $ u^*(k) $ for problem (11)
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2.  Trajectories of state vectors $ x(k) $ for problem (11)
Figure Options

Download full-size image

Download as PowerPoint slide

Table 1.  The optimal results of problem (11)
Stage $ (u^{*}(k))^T $ $ \mu_k $ $ x_1^T(k) $ $ x^T(k) $ $ J\big(k, x_1(k), x_2(7)\big) $
0 $ (3.567, -2.323, -5.406) $ $ 2.517 $ $ (1, 2) $ $ (1.000, -5.434, 2.000, -4.600) $ $ -112.722 $
1 $ (0.338, -2.541, -0.797) $ $ 1.743 $ $ (-4.756, 0.145) $ $ (-4.756, -2.699, 0.145, -4.756) $ $ -55.708 $
2 $ (1.796, -2.599, -0.303) $ $ 2.098 $ $ (-1.416, -4.148) $ $ (-1.416, -5.019, -4.148, -1.218) $ $ 20.114 $
3 $ (0.167, -1.651, 0.469) $ $ 0.202 $ $ (-1.429, -1.614) $ $ (-1.429, -1.769, -1.614, 0.490) $ $ 41.636 $
4 $ (0.635, -1.194, 0.582) $ $ 1.165 $ $ (-0.931, -2.843) $ $ (-0.931, -2.743, -2.843, -0.320) $ $ 39.171 $
5 $ (0.862, -0.338, -0.251) $ $ 0.872 $ $ (-0.362, -0.430) $ $ (-0.362, -1.988, -0.430, -1.147) $ $ 32.059 $
6 $ (0.390, -0.274, -0.403) $ $ 1.394 $ $ (-0.505, 0.182) $ $ (-0.505, -2.598, 0.182, -2.027) $ $ 46.328 $
7 $ u^{*}(7)\in R^3 $ $ (-1.217, 1.619) $ $ (-1.217, 0.598, 1.619, -0.217) $ $ 7.363 $
Table Options

Download as excel

Stage $ (u^{*}(k))^T $ $ \mu_k $ $ x_1^T(k) $ $ x^T(k) $ $ J\big(k, x_1(k), x_2(7)\big) $
0 $ (3.567, -2.323, -5.406) $ $ 2.517 $ $ (1, 2) $ $ (1.000, -5.434, 2.000, -4.600) $ $ -112.722 $
1 $ (0.338, -2.541, -0.797) $ $ 1.743 $ $ (-4.756, 0.145) $ $ (-4.756, -2.699, 0.145, -4.756) $ $ -55.708 $
2 $ (1.796, -2.599, -0.303) $ $ 2.098 $ $ (-1.416, -4.148) $ $ (-1.416, -5.019, -4.148, -1.218) $ $ 20.114 $
3 $ (0.167, -1.651, 0.469) $ $ 0.202 $ $ (-1.429, -1.614) $ $ (-1.429, -1.769, -1.614, 0.490) $ $ 41.636 $
4 $ (0.635, -1.194, 0.582) $ $ 1.165 $ $ (-0.931, -2.843) $ $ (-0.931, -2.743, -2.843, -0.320) $ $ 39.171 $
5 $ (0.862, -0.338, -0.251) $ $ 0.872 $ $ (-0.362, -0.430) $ $ (-0.362, -1.988, -0.430, -1.147) $ $ 32.059 $
6 $ (0.390, -0.274, -0.403) $ $ 1.394 $ $ (-0.505, 0.182) $ $ (-0.505, -2.598, 0.182, -2.027) $ $ 46.328 $
7 $ u^{*}(7)\in R^3 $ $ (-1.217, 1.619) $ $ (-1.217, 0.598, 1.619, -0.217) $ $ 7.363 $
[1]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437
[2]

Luke Finlay, Vladimir Gaitsgory, Ivan Lebedev. Linear programming solutions of periodic optimization problems: approximation of the optimal control. Journal of Industrial & Management Optimization, 2007, 3 (2) : 399-413. doi: 10.3934/jimo.2007.3.399
[3]

Guirong Jiang, Qishao Lu. The dynamics of a Prey-Predator model with impulsive state feedback control. Discrete & Continuous Dynamical Systems - B, 2006, 6 (6) : 1301-1320. doi: 10.3934/dcdsb.2006.6.1301
[4]

Peter Benner, Jens Saak, M. Monir Uddin. Balancing based model reduction for structured index-2 unstable descriptor systems with application to flow control. Numerical Algebra, Control & Optimization, 2016, 6 (1) : 1-20. doi: 10.3934/naco.2016.6.1
[5]

Shanjian Tang, Fu Zhang. Path-dependent optimal stochastic control and viscosity solution of associated Bellman equations. Discrete & Continuous Dynamical Systems - A, 2015, 35 (11) : 5521-5553. doi: 10.3934/dcds.2015.35.5521
[6]

J. Frédéric Bonnans, Justina Gianatti, Francisco J. Silva. On the convergence of the Sakawa-Shindo algorithm in stochastic control. Mathematical Control & Related Fields, 2016, 6 (3) : 391-406. doi: 10.3934/mcrf.2016008
[7]

Alberto Bressan, Ke Han, Franco Rampazzo. On the control of non holonomic systems by active constraints. Discrete & Continuous Dynamical Systems - A, 2013, 33 (8) : 3329-3353. doi: 10.3934/dcds.2013.33.3329
[8]

Paula A. González-Parra, Sunmi Lee, Leticia Velázquez, Carlos Castillo-Chavez. A note on the use of optimal control on a discrete time model of influenza dynamics. Mathematical Biosciences & Engineering, 2011, 8 (1) : 183-197. doi: 10.3934/mbe.2011.8.183
[9]

Yves Dumont, Frederic Chiroleu. Vector control for the Chikungunya disease. Mathematical Biosciences & Engineering, 2010, 7 (2) : 313-345. doi: 10.3934/mbe.2010.7.313
[10]

Simone Cacace, Maurizio Falcone. A dynamic domain decomposition for the eikonal-diffusion equation. Discrete & Continuous Dynamical Systems - S, 2016, 9 (1) : 109-123. doi: 10.3934/dcdss.2016.9.109
[11]

Ardeshir Ahmadi, Hamed Davari-Ardakani. A multistage stochastic programming framework for cardinality constrained portfolio optimization. Numerical Algebra, Control & Optimization, 2017, 7 (3) : 359-377. doi: 10.3934/naco.2017023
[12]

Marcelo Messias. Periodic perturbation of quadratic systems with two infinite heteroclinic cycles. Discrete & Continuous Dynamical Systems - A, 2012, 32 (5) : 1881-1899. doi: 10.3934/dcds.2012.32.1881
[13]

A. K. Misra, Anupama Sharma, Jia Li. A mathematical model for control of vector borne diseases through media campaigns. Discrete & Continuous Dynamical Systems - B, 2013, 18 (7) : 1909-1927. doi: 10.3934/dcdsb.2013.18.1909
[14]

Nhu N. Nguyen, George Yin. Stochastic partial differential equation models for spatially dependent predator-prey equations. Discrete & Continuous Dynamical Systems - B, 2020, 25 (1) : 117-139. doi: 10.3934/dcdsb.2019175
[15]

Longxiang Fang, Narayanaswamy Balakrishnan, Wenyu Huang. Stochastic comparisons of parallel systems with scale proportional hazards components equipped with starting devices. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2021004
[16]

Alexandr Mikhaylov, Victor Mikhaylov. Dynamic inverse problem for Jacobi matrices. Inverse Problems & Imaging, 2019, 13 (3) : 431-447. doi: 10.3934/ipi.2019021
[17]

Jean-François Biasse. Improvements in the computation of ideal class groups of imaginary quadratic number fields. Advances in Mathematics of Communications, 2010, 4 (2) : 141-154. doi: 10.3934/amc.2010.4.141
[18]

Yunfei Lv, Rong Yuan, Yuan He. Wavefronts of a stage structured model with state--dependent delay. Discrete & Continuous Dynamical Systems - A, 2015, 35 (10) : 4931-4954. doi: 10.3934/dcds.2015.35.4931
[19]

Daoyuan Fang, Ting Zhang. Compressible Navier-Stokes equations with vacuum state in one dimension. Communications on Pure & Applied Analysis, 2004, 3 (4) : 675-694. doi: 10.3934/cpaa.2004.3.675
[20]

Y. Latushkin, B. Layton. The optimal gap condition for invariant manifolds. Discrete & Continuous Dynamical Systems - A, 1999, 5 (2) : 233-268. doi: 10.3934/dcds.1999.5.233

2019 Impact Factor: 1.366

Tools

Metrics

  • PDF downloads (18)
  • HTML views (45)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences