2015, 5(3): 275-288. doi: 10.3934/naco.2015.5.275

Output regulation for discrete-time nonlinear stochastic optimal control problems with model-reality differences

1. 

Department of Mathematics, Universiti Tun Hussein Onn Malaysia, 86400 Parit Raja, Malaysia

2. 

Department of Mathematics, Universiti Teknologi Malaysia, 81310 UTM, Skudai, Malaysia

Received  May 2014 Revised  March 2015 Published  August 2015

In this paper, we propose an output regulation approach, which is based on principle of model-reality differences, to obtain the optimal output measurement of a discrete-time nonlinear stochastic optimal control problem. In our approach, a model-based optimal control problem with adding the adjustable parameters is considered. We aim to regulate the optimal output trajectory of the model used as closely as possible to the output measurement of the original optimal control problem. In doing so, an expanded optimal control problem is introduced, where system optimization and parameter estimation are integrated. During the computation procedure, the differences between the real plant and the model used are measured repeatedly. In such a way, the optimal solution of the model is updated. At the end of iteration, the converged solution approaches closely to the true optimal solution of the original optimal control problem in spite of model-reality differences. It is important to notice that the resulting algorithm could give the output residual that is superior to those obtained from Kalman filtering theory. The accuracy of the output regulation is therefore highly recommended. For illustration, a continuous stirred-tank reactor problem is studied. The results obtained show the efficiency of the approach proposed.
Citation: Sie Long Kek, Mohd Ismail Abd Aziz. Output regulation for discrete-time nonlinear stochastic optimal control problems with model-reality differences. Numerical Algebra, Control & Optimization, 2015, 5 (3) : 275-288. doi: 10.3934/naco.2015.5.275
References:
[1]

V. M. Becerra and P. D. Roberts, Dynamic integrated system optimization and parameter estimation for discrete time optimal control of nonlinear systems,, \emph{Int. J. Control}, 63 (1996), 257. doi: 10.1080/00207179608921843.

[2]

A. E. Bryson and Y. C. Ho, Applied Optimal Control,, Hemisphere Publishing Company, (1975).

[3]

S. L. Kek and A. A. Mohd Ismail, Optimal control of discrete-time linear stochastic dynamic system with model-reality differences,, in \emph{Proceeding of International Conference on Machine Learning and Computing (ICML 2009)}, (2009), 10.

[4]

S. L. Kek, K. L. Teo and A. A. Mohd Ismail, An integrated optimal control algorithm for discrete-time nonlinear stochastic system,, \emph{International Journal of Control}, 83 (2010), 2536. doi: 10.1080/00207179.2010.531766.

[5]

S. L. Kek, K. L. Teo and A. A. Mohd Ismail, Filtering solution of nonlinear stochastic optimal control problem in discrete-time with model-reality differences,, \emph{Numerical Algebra, 2 (2012), 207. doi: 10.3934/naco.2012.2.207.

[6]

S. L. Kek, A. A. Mohd Ismail, K. L. Teo and A. Rohanin, An iterative algorithm based on model-reality differences for discrete-time nonlinear stochastic optimal control problems,, \emph{Numerical Algebra, 3 (2013), 109. doi: 10.3934/naco.2013.3.109.

[7]

D. E. Kirk, Optimal Control Theory: An Introduction,, Mineola, (2004).

[8]

F. L. Lewis and V. L. Syrmos, Optimal Control,, 2nd ed, (1995).

[9]

A. A. Mohd Ismail and S. L. Kek, Optimal control of nonlinear discrete-time stochastic system with model-reality differences,, in \emph{2009 IEEE International Conference on Control and Automation}, (2009), 9.

[10]

A. A. Mohd Ismail, A. Rohanin, S. L. Kek and K. L. Teo, Computational integrated optimal control and estimation with model information for discrete-time nonlinear stochastic dynamic system,, in \emph{Proceeding of the 2010 IRAST Internation Congress on Computer Applications and Computational Science (CACS 2010)}, (2010), 4.

[11]

P. D. Roberts and T. W. C. Williams, On an algorithm for combined system optimization and parameter estimation,, \emph{Automatica}, 17 (1981), 199. doi: 10.1016/0005-1098(81)90095-9.

[12]

P. D. Roberts, Optimal control of nonlinear systems with model-reality differences,, \emph{Proceedings of the 31st IEEE Conference on Decision and Control}, 1 (1992), 257.

show all references

References:
[1]

V. M. Becerra and P. D. Roberts, Dynamic integrated system optimization and parameter estimation for discrete time optimal control of nonlinear systems,, \emph{Int. J. Control}, 63 (1996), 257. doi: 10.1080/00207179608921843.

[2]

A. E. Bryson and Y. C. Ho, Applied Optimal Control,, Hemisphere Publishing Company, (1975).

[3]

S. L. Kek and A. A. Mohd Ismail, Optimal control of discrete-time linear stochastic dynamic system with model-reality differences,, in \emph{Proceeding of International Conference on Machine Learning and Computing (ICML 2009)}, (2009), 10.

[4]

S. L. Kek, K. L. Teo and A. A. Mohd Ismail, An integrated optimal control algorithm for discrete-time nonlinear stochastic system,, \emph{International Journal of Control}, 83 (2010), 2536. doi: 10.1080/00207179.2010.531766.

[5]

S. L. Kek, K. L. Teo and A. A. Mohd Ismail, Filtering solution of nonlinear stochastic optimal control problem in discrete-time with model-reality differences,, \emph{Numerical Algebra, 2 (2012), 207. doi: 10.3934/naco.2012.2.207.

[6]

S. L. Kek, A. A. Mohd Ismail, K. L. Teo and A. Rohanin, An iterative algorithm based on model-reality differences for discrete-time nonlinear stochastic optimal control problems,, \emph{Numerical Algebra, 3 (2013), 109. doi: 10.3934/naco.2013.3.109.

[7]

D. E. Kirk, Optimal Control Theory: An Introduction,, Mineola, (2004).

[8]

F. L. Lewis and V. L. Syrmos, Optimal Control,, 2nd ed, (1995).

[9]

A. A. Mohd Ismail and S. L. Kek, Optimal control of nonlinear discrete-time stochastic system with model-reality differences,, in \emph{2009 IEEE International Conference on Control and Automation}, (2009), 9.

[10]

A. A. Mohd Ismail, A. Rohanin, S. L. Kek and K. L. Teo, Computational integrated optimal control and estimation with model information for discrete-time nonlinear stochastic dynamic system,, in \emph{Proceeding of the 2010 IRAST Internation Congress on Computer Applications and Computational Science (CACS 2010)}, (2010), 4.

[11]

P. D. Roberts and T. W. C. Williams, On an algorithm for combined system optimization and parameter estimation,, \emph{Automatica}, 17 (1981), 199. doi: 10.1016/0005-1098(81)90095-9.

[12]

P. D. Roberts, Optimal control of nonlinear systems with model-reality differences,, \emph{Proceedings of the 31st IEEE Conference on Decision and Control}, 1 (1992), 257.

[1]

Sie Long Kek, Mohd Ismail Abd Aziz, Kok Lay Teo, Rohanin Ahmad. An iterative algorithm based on model-reality differences for discrete-time nonlinear stochastic optimal control problems. Numerical Algebra, Control & Optimization, 2013, 3 (1) : 109-125. doi: 10.3934/naco.2013.3.109

[2]

Sie Long Kek, Mohd Ismail Abd Aziz, Kok Lay Teo. A gradient algorithm for optimal control problems with model-reality differences. Numerical Algebra, Control & Optimization, 2015, 5 (3) : 251-266. doi: 10.3934/naco.2015.5.251

[3]

Sie Long Kek, Kok Lay Teo, Mohd Ismail Abd Aziz. Filtering solution of nonlinear stochastic optimal control problem in discrete-time with model-reality differences. Numerical Algebra, Control & Optimization, 2012, 2 (1) : 207-222. doi: 10.3934/naco.2012.2.207

[4]

N. U. Ahmed. Existence of optimal output feedback control law for a class of uncertain infinite dimensional stochastic systems: A direct approach. Evolution Equations & Control Theory, 2012, 1 (2) : 235-250. doi: 10.3934/eect.2012.1.235

[5]

Rein Luus. Optimal control of oscillatory systems by iterative dynamic programming. Journal of Industrial & Management Optimization, 2008, 4 (1) : 1-15. doi: 10.3934/jimo.2008.4.1

[6]

Francesco Cordoni, Luca Di Persio. Optimal control for the stochastic FitzHugh-Nagumo model with recovery variable. Evolution Equations & Control Theory, 2018, 7 (4) : 571-585. doi: 10.3934/eect.2018027

[7]

Margherita Carletti, Matteo Montani, Valentina Meschini, Marzia Bianchi, Lucia Radici. Stochastic modelling of PTEN regulation in brain tumors: A model for glioblastoma multiforme. Mathematical Biosciences & Engineering, 2015, 12 (5) : 965-981. doi: 10.3934/mbe.2015.12.965

[8]

Guangzhou Chen, Guijian Liu, Jiaquan Wang, Ruzhong Li. Identification of water quality model parameters using artificial bee colony algorithm. Numerical Algebra, Control & Optimization, 2012, 2 (1) : 157-165. doi: 10.3934/naco.2012.2.157

[9]

James P. Nelson, Mark J. Balas. Direct model reference adaptive control of linear systems with input/output delays. Numerical Algebra, Control & Optimization, 2013, 3 (3) : 445-462. doi: 10.3934/naco.2013.3.445

[10]

J. Frédéric Bonnans, Justina Gianatti, Francisco J. Silva. On the convergence of the Sakawa-Shindo algorithm in stochastic control. Mathematical Control & Related Fields, 2016, 6 (3) : 391-406. doi: 10.3934/mcrf.2016008

[11]

Ugur G. Abdulla. On the optimal control of the free boundary problems for the second order parabolic equations. II. Convergence of the method of finite differences. Inverse Problems & Imaging, 2016, 10 (4) : 869-898. doi: 10.3934/ipi.2016025

[12]

Yanqin Bai, Yudan Wei, Qian Li. An optimal trade-off model for portfolio selection with sensitivity of parameters. Journal of Industrial & Management Optimization, 2017, 13 (2) : 947-965. doi: 10.3934/jimo.2016055

[13]

Yuan Zhao, Shunfu Jin, Wuyi Yue. Adjustable admission control with threshold in centralized CR networks: Analysis and optimization. Journal of Industrial & Management Optimization, 2015, 11 (4) : 1393-1408. doi: 10.3934/jimo.2015.11.1393

[14]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437

[15]

Fulvia Confortola, Elisa Mastrogiacomo. Optimal control for stochastic heat equation with memory. Evolution Equations & Control Theory, 2014, 3 (1) : 35-58. doi: 10.3934/eect.2014.3.35

[16]

Filipe Rodrigues, Cristiana J. Silva, Delfim F. M. Torres, Helmut Maurer. Optimal control of a delayed HIV model. Discrete & Continuous Dynamical Systems - B, 2018, 23 (1) : 443-458. doi: 10.3934/dcdsb.2018030

[17]

Simai He, Min Li, Shuzhong Zhang, Zhi-Quan Luo. A nonconvergent example for the iterative water-filling algorithm. Numerical Algebra, Control & Optimization, 2011, 1 (1) : 147-150. doi: 10.3934/naco.2011.1.147

[18]

Lingling Lv, Zhe Zhang, Lei Zhang, Weishu Wang. An iterative algorithm for periodic sylvester matrix equations. Journal of Industrial & Management Optimization, 2018, 14 (1) : 413-425. doi: 10.3934/jimo.2017053

[19]

Cristiana J. Silva, Helmut Maurer, Delfim F. M. Torres. Optimal control of a Tuberculosis model with state and control delays. Mathematical Biosciences & Engineering, 2017, 14 (1) : 321-337. doi: 10.3934/mbe.2017021

[20]

Mahadevan Ganesh, Brandon C. Reyes, Avi Purkayastha. An FEM-MLMC algorithm for a moving shutter diffraction in time stochastic model. Discrete & Continuous Dynamical Systems - B, 2018, 22 (11) : 1-16. doi: 10.3934/dcdsb.2018107

 Impact Factor: 

Metrics

  • PDF downloads (3)
  • HTML views (0)
  • Cited by (0)

Other articles
by authors

[Back to Top]