Output regulation for discrete-time nonlinear stochastic optimal control problems with model-reality differences

Previous Article
A survey on rank and inertia optimization problems of the matrix-valued function $A + BXB^{*}$
NACO Home
This Issue
Next Article
Matrix group monotonicity using a dominance notion

2015, 5(3): 275-288. doi: 10.3934/naco.2015.5.275

Output regulation for discrete-time nonlinear stochastic optimal control problems with model-reality differences

Sie Long Kek ^1, and Mohd Ismail Abd Aziz ^2,

1.	Department of Mathematics, Universiti Tun Hussein Onn Malaysia, 86400 Parit Raja, Malaysia
2.	Department of Mathematics, Universiti Teknologi Malaysia, 81310 UTM, Skudai, Malaysia

Received May 2014 Revised March 2015 Published August 2015

Abstract
Related Papers

In this paper, we propose an output regulation approach, which is based on principle of model-reality differences, to obtain the optimal output measurement of a discrete-time nonlinear stochastic optimal control problem. In our approach, a model-based optimal control problem with adding the adjustable parameters is considered. We aim to regulate the optimal output trajectory of the model used as closely as possible to the output measurement of the original optimal control problem. In doing so, an expanded optimal control problem is introduced, where system optimization and parameter estimation are integrated. During the computation procedure, the differences between the real plant and the model used are measured repeatedly. In such a way, the optimal solution of the model is updated. At the end of iteration, the converged solution approaches closely to the true optimal solution of the original optimal control problem in spite of model-reality differences. It is important to notice that the resulting algorithm could give the output residual that is superior to those obtained from Kalman filtering theory. The accuracy of the output regulation is therefore highly recommended. For illustration, a continuous stirred-tank reactor problem is studied. The results obtained show the efficiency of the approach proposed.

Keywords: stochastic optimal control, Output regulation, adjustable parameters., iterative algorithm, model-reality differences.

Mathematics Subject Classification: Primary: 93E20, 93E10; Secondary: 93C1.

Citation: Sie Long Kek, Mohd Ismail Abd Aziz. Output regulation for discrete-time nonlinear stochastic optimal control problems with model-reality differences. Numerical Algebra, Control and Optimization, 2015, 5 (3) : 275-288. doi: 10.3934/naco.2015.5.275

References:

[1]	V. M. Becerra and P. D. Roberts, Dynamic integrated system optimization and parameter estimation for discrete time optimal control of nonlinear systems, Int. J. Control, 63 (1996), 257-281. doi: 10.1080/00207179608921843.
[2]	A. E. Bryson and Y. C. Ho, Applied Optimal Control, Hemisphere Publishing Company, New York, 1975.
[3]	S. L. Kek and A. A. Mohd Ismail, Optimal control of discrete-time linear stochastic dynamic system with model-reality differences, in Proceeding of International Conference on Machine Learning and Computing (ICML 2009), 10-12 July, 2009, Perth, Australia, 573-578.
[4]	S. L. Kek, K. L. Teo and A. A. Mohd Ismail, An integrated optimal control algorithm for discrete-time nonlinear stochastic system, International Journal of Control, 83 (2010), 2536-2545. doi: 10.1080/00207179.2010.531766.
[5]	S. L. Kek, K. L. Teo and A. A. Mohd Ismail, Filtering solution of nonlinear stochastic optimal control problem in discrete-time with model-reality differences, Numerical Algebra, Control and Optimization, 2 (2012), 207-222. doi: 10.3934/naco.2012.2.207.
[6]	S. L. Kek, A. A. Mohd Ismail, K. L. Teo and A. Rohanin, An iterative algorithm based on model-reality differences for discrete-time nonlinear stochastic optimal control problems, Numerical Algebra, Control and Optimization, 3 (2013), 109-125. doi: 10.3934/naco.2013.3.109.
[7]	D. E. Kirk, Optimal Control Theory: An Introduction, Mineola, NY: Dover Publications, 2004.
[8]	F. L. Lewis and V. L. Syrmos, Optimal Control, 2nd ed, John Wiley & Sons 1995.
[9]	A. A. Mohd Ismail and S. L. Kek, Optimal control of nonlinear discrete-time stochastic system with model-reality differences, in 2009 IEEE International Conference on Control and Automation, 9-11 December, 2009, Christchurch, New Zealand, 722-726.
[10]	A. A. Mohd Ismail, A. Rohanin, S. L. Kek and K. L. Teo, Computational integrated optimal control and estimation with model information for discrete-time nonlinear stochastic dynamic system, in Proceeding of the 2010 IRAST Internation Congress on Computer Applications and Computational Science (CACS 2010), 4-6 December, 2010, Singapore, 899-902.
[11]	P. D. Roberts and T. W. C. Williams, On an algorithm for combined system optimization and parameter estimation, Automatica, 17 (1981), 199-209. doi: 10.1016/0005-1098(81)90095-9.
[12]	P. D. Roberts, Optimal control of nonlinear systems with model-reality differences, Proceedings of the 31st IEEE Conference on Decision and Control, 1 (1992), 257-258.

show all references

References:

[1]	V. M. Becerra and P. D. Roberts, Dynamic integrated system optimization and parameter estimation for discrete time optimal control of nonlinear systems, Int. J. Control, 63 (1996), 257-281. doi: 10.1080/00207179608921843.
[2]	A. E. Bryson and Y. C. Ho, Applied Optimal Control, Hemisphere Publishing Company, New York, 1975.
[3]	S. L. Kek and A. A. Mohd Ismail, Optimal control of discrete-time linear stochastic dynamic system with model-reality differences, in Proceeding of International Conference on Machine Learning and Computing (ICML 2009), 10-12 July, 2009, Perth, Australia, 573-578.
[4]	S. L. Kek, K. L. Teo and A. A. Mohd Ismail, An integrated optimal control algorithm for discrete-time nonlinear stochastic system, International Journal of Control, 83 (2010), 2536-2545. doi: 10.1080/00207179.2010.531766.
[5]	S. L. Kek, K. L. Teo and A. A. Mohd Ismail, Filtering solution of nonlinear stochastic optimal control problem in discrete-time with model-reality differences, Numerical Algebra, Control and Optimization, 2 (2012), 207-222. doi: 10.3934/naco.2012.2.207.
[6]	S. L. Kek, A. A. Mohd Ismail, K. L. Teo and A. Rohanin, An iterative algorithm based on model-reality differences for discrete-time nonlinear stochastic optimal control problems, Numerical Algebra, Control and Optimization, 3 (2013), 109-125. doi: 10.3934/naco.2013.3.109.
[7]	D. E. Kirk, Optimal Control Theory: An Introduction, Mineola, NY: Dover Publications, 2004.
[8]	F. L. Lewis and V. L. Syrmos, Optimal Control, 2nd ed, John Wiley & Sons 1995.
[9]	A. A. Mohd Ismail and S. L. Kek, Optimal control of nonlinear discrete-time stochastic system with model-reality differences, in 2009 IEEE International Conference on Control and Automation, 9-11 December, 2009, Christchurch, New Zealand, 722-726.
[10]	A. A. Mohd Ismail, A. Rohanin, S. L. Kek and K. L. Teo, Computational integrated optimal control and estimation with model information for discrete-time nonlinear stochastic dynamic system, in Proceeding of the 2010 IRAST Internation Congress on Computer Applications and Computational Science (CACS 2010), 4-6 December, 2010, Singapore, 899-902.
[11]	P. D. Roberts and T. W. C. Williams, On an algorithm for combined system optimization and parameter estimation, Automatica, 17 (1981), 199-209. doi: 10.1016/0005-1098(81)90095-9.
[12]	P. D. Roberts, Optimal control of nonlinear systems with model-reality differences, Proceedings of the 31st IEEE Conference on Decision and Control, 1 (1992), 257-258.

[1]	Sie Long Kek, Mohd Ismail Abd Aziz, Kok Lay Teo, Rohanin Ahmad. An iterative algorithm based on model-reality differences for discrete-time nonlinear stochastic optimal control problems. Numerical Algebra, Control and Optimization, 2013, 3 (1) : 109-125. doi: 10.3934/naco.2013.3.109
[2]	Sie Long Kek, Mohd Ismail Abd Aziz, Kok Lay Teo. A gradient algorithm for optimal control problems with model-reality differences. Numerical Algebra, Control and Optimization, 2015, 5 (3) : 251-266. doi: 10.3934/naco.2015.5.251
[3]	Sie Long Kek, Kok Lay Teo, Mohd Ismail Abd Aziz. Filtering solution of nonlinear stochastic optimal control problem in discrete-time with model-reality differences. Numerical Algebra, Control and Optimization, 2012, 2 (1) : 207-222. doi: 10.3934/naco.2012.2.207
[4]	N. U. Ahmed. Existence of optimal output feedback control law for a class of uncertain infinite dimensional stochastic systems: A direct approach. Evolution Equations and Control Theory, 2012, 1 (2) : 235-250. doi: 10.3934/eect.2012.1.235
[5]	Toufik Bakir, Bernard Bonnard, Jérémy Rouot. A case study of optimal input-output system with sampled-data control: Ding et al. force and fatigue muscular control model. Networks and Heterogeneous Media, 2019, 14 (1) : 79-100. doi: 10.3934/nhm.2019005
[6]	Rein Luus. Optimal control of oscillatory systems by iterative dynamic programming. Journal of Industrial and Management Optimization, 2008, 4 (1) : 1-15. doi: 10.3934/jimo.2008.4.1
[7]	Kangbo Bao, Libin Rong, Qimin Zhang. Analysis of a stochastic SIRS model with interval parameters. Discrete and Continuous Dynamical Systems - B, 2019, 24 (9) : 4827-4849. doi: 10.3934/dcdsb.2019033
[8]	James P. Nelson, Mark J. Balas. Direct model reference adaptive control of linear systems with input/output delays. Numerical Algebra, Control and Optimization, 2013, 3 (3) : 445-462. doi: 10.3934/naco.2013.3.445
[9]	Margherita Carletti, Matteo Montani, Valentina Meschini, Marzia Bianchi, Lucia Radici. Stochastic modelling of PTEN regulation in brain tumors: A model for glioblastoma multiforme. Mathematical Biosciences & Engineering, 2015, 12 (5) : 965-981. doi: 10.3934/mbe.2015.12.965
[10]	Francesco Cordoni, Luca Di Persio. Optimal control for the stochastic FitzHugh-Nagumo model with recovery variable. Evolution Equations and Control Theory, 2018, 7 (4) : 571-585. doi: 10.3934/eect.2018027
[11]	Guangzhou Chen, Guijian Liu, Jiaquan Wang, Ruzhong Li. Identification of water quality model parameters using artificial bee colony algorithm. Numerical Algebra, Control and Optimization, 2012, 2 (1) : 157-165. doi: 10.3934/naco.2012.2.157
[12]	Ugur G. Abdulla. On the optimal control of the free boundary problems for the second order parabolic equations. II. Convergence of the method of finite differences. Inverse Problems and Imaging, 2016, 10 (4) : 869-898. doi: 10.3934/ipi.2016025
[13]	J. Frédéric Bonnans, Justina Gianatti, Francisco J. Silva. On the convergence of the Sakawa-Shindo algorithm in stochastic control. Mathematical Control and Related Fields, 2016, 6 (3) : 391-406. doi: 10.3934/mcrf.2016008
[14]	Jiongmin Yong. Stochastic optimal control — A concise introduction. Mathematical Control and Related Fields, 2020 doi: 10.3934/mcrf.2020027
[15]	Yuan Zhao, Shunfu Jin, Wuyi Yue. Adjustable admission control with threshold in centralized CR networks: Analysis and optimization. Journal of Industrial and Management Optimization, 2015, 11 (4) : 1393-1408. doi: 10.3934/jimo.2015.11.1393
[16]	Yanqin Bai, Yudan Wei, Qian Li. An optimal trade-off model for portfolio selection with sensitivity of parameters. Journal of Industrial and Management Optimization, 2017, 13 (2) : 947-965. doi: 10.3934/jimo.2016055
[17]	Yi Zhang, Xiao-Li Ma. Research on image digital watermarking optimization algorithm under virtual reality technology. Discrete and Continuous Dynamical Systems - S, 2019, 12 (4&5) : 1427-1440. doi: 10.3934/dcdss.2019098
[18]	Liangquan Zhang, Qing Zhou, Juan Yang. Necessary condition for optimal control of doubly stochastic systems. Mathematical Control and Related Fields, 2020, 10 (2) : 379-403. doi: 10.3934/mcrf.2020002
[19]	Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437
[20]	Fulvia Confortola, Elisa Mastrogiacomo. Optimal control for stochastic heat equation with memory. Evolution Equations and Control Theory, 2014, 3 (1) : 35-58. doi: 10.3934/eect.2014.3.35

Impact Factor:

Tools

Metrics

Other articles
by authors

on AIMS
Sie Long Kek Mohd Ismail Abd Aziz
on Google Scholar
Sie Long Kek Mohd Ismail Abd Aziz

[Back to Top]