# American Institute of Mathematical Sciences

• Previous Article
Effects of disruption risk on a supply chain with a risk-averse retailer
• JIMO Home
• This Issue
• Next Article
First passage problems of refracted jump diffusion processes and their applications in valuing equity-linked death benefits
doi: 10.3934/jimo.2021034
Online First

Online First articles are published articles within a journal that have not yet been assigned to a formal issue. This means they do not yet have a volume number, issue number, or page numbers assigned to them, however, they can still be found and cited using their DOI (Digital Object Identifier). Online First publication benefits the research community by making new scientific discoveries known as quickly as possible.

Readers can access Online First articles via the “Online First” tab for the selected journal.

## Linear-quadratic optimal control for discrete-time stochastic descriptor systems

 1 School of Mathematics and Statistics, Nanjing University of Information Science and Technology, Nanjing 210044, China 2 School of Applied Mathematics, Nanjing University of Finance and Economics, Nanjing 210023, China

* Corresponding author

Received  October 2020 Revised  December 2020 Early access February 2021

In this paper, an optimal control model ruled by a class of linear discrete-time stochastic descriptor systems is considered under quadratic index performance. Employing dynamic programming method, a recurrence equation to simplify the optimal control problem is presented provided that the descriptor systems are both regular and impulse-free. When the objective function is quadratic, according to the recurrence equation, a discrete-time linear-quadratic optimal control problem is completely settled, that is, optimal controls and optimal values of the problem are both obtained through analytical expressions. At last, a numerical example about linear-quadratic optimal control for a discrete-time stochastic descriptor system is provided to illustrate the validness of the results derived.

Citation: Yadong Shu, Bo Li. Linear-quadratic optimal control for discrete-time stochastic descriptor systems. Journal of Industrial & Management Optimization, doi: 10.3934/jimo.2021034
##### References:
 [1] K. Avrachenkov, O. Habachi, A. Piunovskiy and Y. Zhang, Infinite horizon optimal impulsive control with applications to Internet congestion control, International Journal of Control, 88 (2015), 703-716.  doi: 10.1080/00207179.2014.971436.  Google Scholar [2] A. V. Balakrishnan, On stochastic bang-bang control, Applied Mathematics and Optimization, 6 (1980), 91-96.  doi: 10.1007/BF01442885.  Google Scholar [3] M. Basin and and J. Rodriguez-Gonzalez, Optimal control for linear systems with time delay in control input, IEEE Transactions on Automatic Control, 51 (2006), 91-97.  doi: 10.1109/TAC.2005.861718.  Google Scholar [4] A. Cairns, Some notes on the dynamics and optimal control of stochastic pension fund models in continuous time, Astin Bulletin, 30 (2000), 19-55.  doi: 10.2143/AST.30.1.504625.  Google Scholar [5] S. L. Campbell, Singular Systems of Differential Equations II, Pitman, Boston, Mass.-London, 1982.  Google Scholar [6] M. Chadli, H. R. Karimi and P. Shi, On stability and stabilization of singular uncertain Takagi-Sugeno fuzzy systems, Journal of the Franklin Institute, 351 (2014), 1453-1463.  doi: 10.1016/j.jfranklin.2013.11.008.  Google Scholar [7] L. Chen and Z. Wu, Maximum principle for the stochastic optimal control problem with delay and application, Automatica J. IFAC, 46 (2010), 1074-1080.  doi: 10.1016/j.automatica.2010.03.005.  Google Scholar [8] D. Cobb, Controllability, observability, and duality in singular systems, IEEE Transactions on Automatic Control, 29 (1984), 1076-1082.  doi: 10.1109/TAC.1984.1103451.  Google Scholar [9] L. Dai, Singular Control Systems, Springer-Verlag, Berlin, Germany, 1989. doi: 10.1007/BFb0002475.  Google Scholar [10] J.-e Feng, P. Cui and Z. Hou, Singular linear quadratic optimal control for singular stochastic discrete-time systems, Optimal Control Applications and Methods, 34 (2013), 505-516.  doi: 10.1002/oca.2033.  Google Scholar [11] J. Y. Ishihara and M. H. Terra, On the Lyapunov theorem for singular systems, IEEE Transactions on Automatic Control, 47 (2002), 1926-1930.  doi: 10.1109/TAC.2002.804463.  Google Scholar [12] R. E. Kalman, Contribution to the theory of optimal control, Boletin Sociedad Matem$\acute{a}$tica Mexicana, 5 (1960), 102–119.  Google Scholar [13] Y. Kang and Y. Zhu, Bang-bang optimal control for multi-stage uncertain systems, International Journal on Information, 15 (2012), 3229-3237.   Google Scholar [14] D. E. Kirk, Optimal Control Theory: An Introduction, Courier Corporation, 2012. Google Scholar [15] N. Kumaresan and P. Balasubramaniam, Optimal control for stochastic nonlinear singular system using neural networks, Computers and Mathematics with Applications, 56 (2008), 2145-2154.  doi: 10.1016/j.camwa.2008.03.041.  Google Scholar [16] P. Kunkel and V. Mehrmann, The linear quadratic optimal control problem for linear descriptor systems with variable coefficients, Mathematics of Control, Signals, and Systems, 10 (1997), 247-264.  doi: 10.1007/BF01211506.  Google Scholar [17] B. Li and and Y. Zhu, The piecewise parametric optimal control of uncertain linear quadratic models, International Journal of Systems Science, 50 (2019), 961-969.  doi: 10.1080/00207721.2019.1586003.  Google Scholar [18] F. Liao, Z. Ren, M. Tomizuka and J. Wu, Preview control for impulse-free continuous-time descriptor systems, International Journal of Control, 88 (2015), 1142-1149.  doi: 10.1080/00207179.2014.996769.  Google Scholar [19] J. Lian, C. Li and D. Liu, Input-to-state stability for discrete-time nonlinear switched singular systems, IET Control Theory Appl., 11 (2017), 2893–2899. doi: 10.1049/iet-cta.2017.0028.  Google Scholar [20] W. Liu, X. Liang, Y. Ma and W. Liu, Aircraft trajectory optimization for collision avoidance using stochastic optimal control, Asian Journal of Control, 21 (2019), 2308-2320.  doi: 10.1002/asjc.1855.  Google Scholar [21] D. G. Luenberger, Dynamic equations in descriptor form, IEEE Transactions on Automatic Control, 22 (1977), 312-321.  doi: 10.1109/tac.1977.1101502.  Google Scholar [22] D. G. Luenberger and A. Arbel, Singular dynamic Leontief systems, Econometrica, 45 (1977), 991-995.  doi: 10.2307/1912686.  Google Scholar [23] G. P. Mantas and N. J. Krikelis, LQ optimal control for discrete descriptor systems with a complete set of boundary information, International Journal of Control, 47 (1988), 1467-1477.  doi: 10.1080/00207178808906108.  Google Scholar [24] D. S. Naidu, Singular perturbations and time scales in control theory and applications: An overview., Dynamics of Continuous Discrete & Impulsive Systems Ser. B Appl. Algorithms, 9 (2002), 233-278.   Google Scholar [25] H. Peng, Fei. Li, J. Liu and Z. Ju, A symplectic instantaneous optimal control for robot trajectory tracking with differential-algebraic equation models, IEEE Transactions on Industrial Electronics, 67 (2020), 3819-3829.  doi: 10.1109/TIE.2019.2916390.  Google Scholar [26] Y. Shu and Y. Zhu, Optimistic value based optimal control for uncertain linear singular systems and application to a dynamic input-output model, ISA Transactions, 71 (2017), 235-251.  doi: 10.1016/j.isatra.2017.08.007.  Google Scholar [27] Y. Shu and Y. Zhu, Stability analysis of uncertain singular systems, Soft Computing, 22 (2018), 5671-5681.  doi: 10.1007/s00500-017-2599-2.  Google Scholar [28] K. Wang, L. Xie and Y. Liu, Characteristics of self-excited oscillation in a curved diffuser., Journal of Propulsion Technology, 39 (2018), 1955-1964.   Google Scholar [29] W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM Journal on Control, 6 (1968), 681-697.  doi: 10.1137/0306044.  Google Scholar [30] H. Yan, Y. Sun and Y. Zhu, A linear-quadratic control problem of uncertain discrete-time switched systems, Journal of Industrial and Management Optimization, 13 (2017), 267-282.  doi: 10.3934/jimo.2016016.  Google Scholar [31] W. Zhang, J. Hu and A. Abate, On the value function of the discrete-time switched LQR problem, IEEE Transactions on Automatic Control, 54 (2009), 2669-2674.  doi: 10.1109/TAC.2009.2031574.  Google Scholar [32] W. Zhang, Y. Lin and L. Xue, Linear quadratic Pareto optimal control problem of stochastic singular systems, Journal of the Franklin Institute, 354 (2017), 1220-1238.  doi: 10.1016/j.jfranklin.2016.11.021.  Google Scholar

show all references

##### References:
 [1] K. Avrachenkov, O. Habachi, A. Piunovskiy and Y. Zhang, Infinite horizon optimal impulsive control with applications to Internet congestion control, International Journal of Control, 88 (2015), 703-716.  doi: 10.1080/00207179.2014.971436.  Google Scholar [2] A. V. Balakrishnan, On stochastic bang-bang control, Applied Mathematics and Optimization, 6 (1980), 91-96.  doi: 10.1007/BF01442885.  Google Scholar [3] M. Basin and and J. Rodriguez-Gonzalez, Optimal control for linear systems with time delay in control input, IEEE Transactions on Automatic Control, 51 (2006), 91-97.  doi: 10.1109/TAC.2005.861718.  Google Scholar [4] A. Cairns, Some notes on the dynamics and optimal control of stochastic pension fund models in continuous time, Astin Bulletin, 30 (2000), 19-55.  doi: 10.2143/AST.30.1.504625.  Google Scholar [5] S. L. Campbell, Singular Systems of Differential Equations II, Pitman, Boston, Mass.-London, 1982.  Google Scholar [6] M. Chadli, H. R. Karimi and P. Shi, On stability and stabilization of singular uncertain Takagi-Sugeno fuzzy systems, Journal of the Franklin Institute, 351 (2014), 1453-1463.  doi: 10.1016/j.jfranklin.2013.11.008.  Google Scholar [7] L. Chen and Z. Wu, Maximum principle for the stochastic optimal control problem with delay and application, Automatica J. IFAC, 46 (2010), 1074-1080.  doi: 10.1016/j.automatica.2010.03.005.  Google Scholar [8] D. Cobb, Controllability, observability, and duality in singular systems, IEEE Transactions on Automatic Control, 29 (1984), 1076-1082.  doi: 10.1109/TAC.1984.1103451.  Google Scholar [9] L. Dai, Singular Control Systems, Springer-Verlag, Berlin, Germany, 1989. doi: 10.1007/BFb0002475.  Google Scholar [10] J.-e Feng, P. Cui and Z. Hou, Singular linear quadratic optimal control for singular stochastic discrete-time systems, Optimal Control Applications and Methods, 34 (2013), 505-516.  doi: 10.1002/oca.2033.  Google Scholar [11] J. Y. Ishihara and M. H. Terra, On the Lyapunov theorem for singular systems, IEEE Transactions on Automatic Control, 47 (2002), 1926-1930.  doi: 10.1109/TAC.2002.804463.  Google Scholar [12] R. E. Kalman, Contribution to the theory of optimal control, Boletin Sociedad Matem$\acute{a}$tica Mexicana, 5 (1960), 102–119.  Google Scholar [13] Y. Kang and Y. Zhu, Bang-bang optimal control for multi-stage uncertain systems, International Journal on Information, 15 (2012), 3229-3237.   Google Scholar [14] D. E. Kirk, Optimal Control Theory: An Introduction, Courier Corporation, 2012. Google Scholar [15] N. Kumaresan and P. Balasubramaniam, Optimal control for stochastic nonlinear singular system using neural networks, Computers and Mathematics with Applications, 56 (2008), 2145-2154.  doi: 10.1016/j.camwa.2008.03.041.  Google Scholar [16] P. Kunkel and V. Mehrmann, The linear quadratic optimal control problem for linear descriptor systems with variable coefficients, Mathematics of Control, Signals, and Systems, 10 (1997), 247-264.  doi: 10.1007/BF01211506.  Google Scholar [17] B. Li and and Y. Zhu, The piecewise parametric optimal control of uncertain linear quadratic models, International Journal of Systems Science, 50 (2019), 961-969.  doi: 10.1080/00207721.2019.1586003.  Google Scholar [18] F. Liao, Z. Ren, M. Tomizuka and J. Wu, Preview control for impulse-free continuous-time descriptor systems, International Journal of Control, 88 (2015), 1142-1149.  doi: 10.1080/00207179.2014.996769.  Google Scholar [19] J. Lian, C. Li and D. Liu, Input-to-state stability for discrete-time nonlinear switched singular systems, IET Control Theory Appl., 11 (2017), 2893–2899. doi: 10.1049/iet-cta.2017.0028.  Google Scholar [20] W. Liu, X. Liang, Y. Ma and W. Liu, Aircraft trajectory optimization for collision avoidance using stochastic optimal control, Asian Journal of Control, 21 (2019), 2308-2320.  doi: 10.1002/asjc.1855.  Google Scholar [21] D. G. Luenberger, Dynamic equations in descriptor form, IEEE Transactions on Automatic Control, 22 (1977), 312-321.  doi: 10.1109/tac.1977.1101502.  Google Scholar [22] D. G. Luenberger and A. Arbel, Singular dynamic Leontief systems, Econometrica, 45 (1977), 991-995.  doi: 10.2307/1912686.  Google Scholar [23] G. P. Mantas and N. J. Krikelis, LQ optimal control for discrete descriptor systems with a complete set of boundary information, International Journal of Control, 47 (1988), 1467-1477.  doi: 10.1080/00207178808906108.  Google Scholar [24] D. S. Naidu, Singular perturbations and time scales in control theory and applications: An overview., Dynamics of Continuous Discrete & Impulsive Systems Ser. B Appl. Algorithms, 9 (2002), 233-278.   Google Scholar [25] H. Peng, Fei. Li, J. Liu and Z. Ju, A symplectic instantaneous optimal control for robot trajectory tracking with differential-algebraic equation models, IEEE Transactions on Industrial Electronics, 67 (2020), 3819-3829.  doi: 10.1109/TIE.2019.2916390.  Google Scholar [26] Y. Shu and Y. Zhu, Optimistic value based optimal control for uncertain linear singular systems and application to a dynamic input-output model, ISA Transactions, 71 (2017), 235-251.  doi: 10.1016/j.isatra.2017.08.007.  Google Scholar [27] Y. Shu and Y. Zhu, Stability analysis of uncertain singular systems, Soft Computing, 22 (2018), 5671-5681.  doi: 10.1007/s00500-017-2599-2.  Google Scholar [28] K. Wang, L. Xie and Y. Liu, Characteristics of self-excited oscillation in a curved diffuser., Journal of Propulsion Technology, 39 (2018), 1955-1964.   Google Scholar [29] W. M. Wonham, On a matrix Riccati equation of stochastic control, SIAM Journal on Control, 6 (1968), 681-697.  doi: 10.1137/0306044.  Google Scholar [30] H. Yan, Y. Sun and Y. Zhu, A linear-quadratic control problem of uncertain discrete-time switched systems, Journal of Industrial and Management Optimization, 13 (2017), 267-282.  doi: 10.3934/jimo.2016016.  Google Scholar [31] W. Zhang, J. Hu and A. Abate, On the value function of the discrete-time switched LQR problem, IEEE Transactions on Automatic Control, 54 (2009), 2669-2674.  doi: 10.1109/TAC.2009.2031574.  Google Scholar [32] W. Zhang, Y. Lin and L. Xue, Linear quadratic Pareto optimal control problem of stochastic singular systems, Journal of the Franklin Institute, 354 (2017), 1220-1238.  doi: 10.1016/j.jfranklin.2016.11.021.  Google Scholar
Trajectories of optimal controls $u^*(k)$ for problem (11)
Trajectories of state vectors $x(k)$ for problem (11)
The optimal results of problem (11)
 Stage $(u^{*}(k))^T$ $\mu_k$ $x_1^T(k)$ $x^T(k)$ $J\big(k, x_1(k), x_2(7)\big)$ 0 $(3.567, -2.323, -5.406)$ $2.517$ $(1, 2)$ $(1.000, -5.434, 2.000, -4.600)$ $-112.722$ 1 $(0.338, -2.541, -0.797)$ $1.743$ $(-4.756, 0.145)$ $(-4.756, -2.699, 0.145, -4.756)$ $-55.708$ 2 $(1.796, -2.599, -0.303)$ $2.098$ $(-1.416, -4.148)$ $(-1.416, -5.019, -4.148, -1.218)$ $20.114$ 3 $(0.167, -1.651, 0.469)$ $0.202$ $(-1.429, -1.614)$ $(-1.429, -1.769, -1.614, 0.490)$ $41.636$ 4 $(0.635, -1.194, 0.582)$ $1.165$ $(-0.931, -2.843)$ $(-0.931, -2.743, -2.843, -0.320)$ $39.171$ 5 $(0.862, -0.338, -0.251)$ $0.872$ $(-0.362, -0.430)$ $(-0.362, -1.988, -0.430, -1.147)$ $32.059$ 6 $(0.390, -0.274, -0.403)$ $1.394$ $(-0.505, 0.182)$ $(-0.505, -2.598, 0.182, -2.027)$ $46.328$ 7 $u^{*}(7)\in R^3$ $(-1.217, 1.619)$ $(-1.217, 0.598, 1.619, -0.217)$ $7.363$
 Stage $(u^{*}(k))^T$ $\mu_k$ $x_1^T(k)$ $x^T(k)$ $J\big(k, x_1(k), x_2(7)\big)$ 0 $(3.567, -2.323, -5.406)$ $2.517$ $(1, 2)$ $(1.000, -5.434, 2.000, -4.600)$ $-112.722$ 1 $(0.338, -2.541, -0.797)$ $1.743$ $(-4.756, 0.145)$ $(-4.756, -2.699, 0.145, -4.756)$ $-55.708$ 2 $(1.796, -2.599, -0.303)$ $2.098$ $(-1.416, -4.148)$ $(-1.416, -5.019, -4.148, -1.218)$ $20.114$ 3 $(0.167, -1.651, 0.469)$ $0.202$ $(-1.429, -1.614)$ $(-1.429, -1.769, -1.614, 0.490)$ $41.636$ 4 $(0.635, -1.194, 0.582)$ $1.165$ $(-0.931, -2.843)$ $(-0.931, -2.743, -2.843, -0.320)$ $39.171$ 5 $(0.862, -0.338, -0.251)$ $0.872$ $(-0.362, -0.430)$ $(-0.362, -1.988, -0.430, -1.147)$ $32.059$ 6 $(0.390, -0.274, -0.403)$ $1.394$ $(-0.505, 0.182)$ $(-0.505, -2.598, 0.182, -2.027)$ $46.328$ 7 $u^{*}(7)\in R^3$ $(-1.217, 1.619)$ $(-1.217, 0.598, 1.619, -0.217)$ $7.363$
 [1] Mohamed Aliane, Mohand Bentobache, Nacima Moussouni, Philippe Marthon. Direct method to solve linear-quadratic optimal control problems. Numerical Algebra, Control & Optimization, 2021, 11 (4) : 645-663. doi: 10.3934/naco.2021002 [2] Galina Kurina, Sahlar Meherrem. Decomposition of discrete linear-quadratic optimal control problems for switching systems. Conference Publications, 2015, 2015 (special) : 764-774. doi: 10.3934/proc.2015.0764 [3] Nguyen Thi Hoai. Asymptotic approximation to a solution of a singularly perturbed linear-quadratic optimal control problem with second-order linear ordinary differential equation of state variable. Numerical Algebra, Control & Optimization, 2021, 11 (4) : 495-512. doi: 10.3934/naco.2020040 [4] Russell Johnson, Carmen Núñez. Remarks on linear-quadratic dissipative control systems. Discrete & Continuous Dynamical Systems - B, 2015, 20 (3) : 889-914. doi: 10.3934/dcdsb.2015.20.889 [5] Qi Lü, Tianxiao Wang, Xu Zhang. Characterization of optimal feedback for stochastic linear quadratic control problems. Probability, Uncertainty and Quantitative Risk, 2017, 2 (0) : 11-. doi: 10.1186/s41546-017-0022-7 [6] Jianhui Huang, Xun Li, Jiongmin Yong. A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Mathematical Control & Related Fields, 2015, 5 (1) : 97-139. doi: 10.3934/mcrf.2015.5.97 [7] Hanxiao Wang, Jingrui Sun, Jiongmin Yong. Weak closed-loop solvability of stochastic linear-quadratic optimal control problems. Discrete & Continuous Dynamical Systems, 2019, 39 (5) : 2785-2805. doi: 10.3934/dcds.2019117 [8] Jingrui Sun, Hanxiao Wang. Mean-field stochastic linear-quadratic optimal control problems: Weak closed-loop solvability. Mathematical Control & Related Fields, 2021, 11 (1) : 47-71. doi: 10.3934/mcrf.2020026 [9] Ying Hu, Shanjian Tang. Mixed deterministic and random optimal control of linear stochastic systems with quadratic costs. Probability, Uncertainty and Quantitative Risk, 2019, 4 (0) : 1-. doi: 10.1186/s41546-018-0035-x [10] Rein Luus. Optimal control of oscillatory systems by iterative dynamic programming. Journal of Industrial & Management Optimization, 2008, 4 (1) : 1-15. doi: 10.3934/jimo.2008.4.1 [11] Shigeaki Koike, Hiroaki Morimoto, Shigeru Sakaguchi. A linear-quadratic control problem with discretionary stopping. Discrete & Continuous Dynamical Systems - B, 2007, 8 (2) : 261-277. doi: 10.3934/dcdsb.2007.8.261 [12] Haibo Jin, Long Hai, Xiaoliang Tang. An optimal maintenance strategy for multi-state systems based on a system linear integral equation and dynamic programming. Journal of Industrial & Management Optimization, 2020, 16 (2) : 965-990. doi: 10.3934/jimo.2018188 [13] Georg Vossen, Stefan Volkwein. Model reduction techniques with a-posteriori error analysis for linear-quadratic optimal control problems. Numerical Algebra, Control & Optimization, 2012, 2 (3) : 465-485. doi: 10.3934/naco.2012.2.465 [14] Changzhi Wu, Kok Lay Teo, Volker Rehbock. Optimal control of piecewise affine systems with piecewise affine state feedback. Journal of Industrial & Management Optimization, 2009, 5 (4) : 737-747. doi: 10.3934/jimo.2009.5.737 [15] Hongyan Yan, Yun Sun, Yuanguo Zhu. A linear-quadratic control problem of uncertain discrete-time switched systems. Journal of Industrial & Management Optimization, 2017, 13 (1) : 267-282. doi: 10.3934/jimo.2016016 [16] Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437 [17] Roberta Fabbri, Russell Johnson, Sylvia Novo, Carmen Núñez. On linear-quadratic dissipative control processes with time-varying coefficients. Discrete & Continuous Dynamical Systems, 2013, 33 (1) : 193-210. doi: 10.3934/dcds.2013.33.193 [18] Rohit Gupta, Farhad Jafari, Robert J. Kipka, Boris S. Mordukhovich. Linear openness and feedback stabilization of nonlinear control systems. Discrete & Continuous Dynamical Systems - S, 2018, 11 (6) : 1103-1119. doi: 10.3934/dcdss.2018063 [19] Tijana Levajković, Hermann Mena, Amjad Tuffaha. The stochastic linear quadratic optimal control problem in Hilbert spaces: A polynomial chaos approach. Evolution Equations & Control Theory, 2016, 5 (1) : 105-134. doi: 10.3934/eect.2016.5.105 [20] Xun Li, Jingrui Sun, Jiongmin Yong. Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability. Probability, Uncertainty and Quantitative Risk, 2016, 1 (0) : 2-. doi: 10.1186/s41546-016-0002-3

2020 Impact Factor: 1.801

## Tools

Article outline

Figures and Tables