American Institute of Mathematical Sciences

Advanced
2015, 5(3): 275-288. doi: 10.3934/naco.2015.5.275

Output regulation for discrete-time nonlinear stochastic optimal control problems with model-reality differences

Sie Long Kek 1, and  Mohd Ismail Abd Aziz 2,

1. 

Department of Mathematics, Universiti Tun Hussein Onn Malaysia, 86400 Parit Raja, Malaysia
2. 

Department of Mathematics, Universiti Teknologi Malaysia, 81310 UTM, Skudai, Malaysia

Received  May 2014 Revised  March 2015 Published  August 2015

In this paper, we propose an output regulation approach, which is based on principle of model-reality differences, to obtain the optimal output measurement of a discrete-time nonlinear stochastic optimal control problem. In our approach, a model-based optimal control problem with adding the adjustable parameters is considered. We aim to regulate the optimal output trajectory of the model used as closely as possible to the output measurement of the original optimal control problem. In doing so, an expanded optimal control problem is introduced, where system optimization and parameter estimation are integrated. During the computation procedure, the differences between the real plant and the model used are measured repeatedly. In such a way, the optimal solution of the model is updated. At the end of iteration, the converged solution approaches closely to the true optimal solution of the original optimal control problem in spite of model-reality differences. It is important to notice that the resulting algorithm could give the output residual that is superior to those obtained from Kalman filtering theory. The accuracy of the output regulation is therefore highly recommended. For illustration, a continuous stirred-tank reactor problem is studied. The results obtained show the efficiency of the approach proposed.
Keywords: stochastic optimal control, Output regulation, adjustable parameters., iterative algorithm, model-reality differences.
Mathematics Subject Classification: Primary: 93E20, 93E10; Secondary: 93C1.
Citation: Sie Long Kek, Mohd Ismail Abd Aziz. Output regulation for discrete-time nonlinear stochastic optimal control problems with model-reality differences. Numerical Algebra, Control & Optimization, 2015, 5 (3) : 275-288. doi: 10.3934/naco.2015.5.275
References:
[1]

V. M. Becerra and P. D. Roberts, Dynamic integrated system optimization and parameter estimation for discrete time optimal control of nonlinear systems,, \emph{Int. J. Control}, 63 (1996), 257.  doi: 10.1080/00207179608921843.  Google Scholar

[2]

A. E. Bryson and Y. C. Ho, Applied Optimal Control,, Hemisphere Publishing Company, (1975).   Google Scholar

[3]

S. L. Kek and A. A. Mohd Ismail, Optimal control of discrete-time linear stochastic dynamic system with model-reality differences,, in \emph{Proceeding of International Conference on Machine Learning and Computing (ICML 2009)}, (2009), 10.   Google Scholar

[4]

S. L. Kek, K. L. Teo and A. A. Mohd Ismail, An integrated optimal control algorithm for discrete-time nonlinear stochastic system,, \emph{International Journal of Control}, 83 (2010), 2536.  doi: 10.1080/00207179.2010.531766.  Google Scholar

[5]

S. L. Kek, K. L. Teo and A. A. Mohd Ismail, Filtering solution of nonlinear stochastic optimal control problem in discrete-time with model-reality differences,, \emph{Numerical Algebra, 2 (2012), 207.  doi: 10.3934/naco.2012.2.207.  Google Scholar

[6]

S. L. Kek, A. A. Mohd Ismail, K. L. Teo and A. Rohanin, An iterative algorithm based on model-reality differences for discrete-time nonlinear stochastic optimal control problems,, \emph{Numerical Algebra, 3 (2013), 109.  doi: 10.3934/naco.2013.3.109.  Google Scholar

[7]

D. E. Kirk, Optimal Control Theory: An Introduction,, Mineola, (2004).   Google Scholar

[8]

F. L. Lewis and V. L. Syrmos, Optimal Control,, 2nd ed, (1995).   Google Scholar

[9]

A. A. Mohd Ismail and S. L. Kek, Optimal control of nonlinear discrete-time stochastic system with model-reality differences,, in \emph{2009 IEEE International Conference on Control and Automation}, (2009), 9.   Google Scholar

[10]

A. A. Mohd Ismail, A. Rohanin, S. L. Kek and K. L. Teo, Computational integrated optimal control and estimation with model information for discrete-time nonlinear stochastic dynamic system,, in \emph{Proceeding of the 2010 IRAST Internation Congress on Computer Applications and Computational Science (CACS 2010)}, (2010), 4.   Google Scholar

[11]

P. D. Roberts and T. W. C. Williams, On an algorithm for combined system optimization and parameter estimation,, \emph{Automatica}, 17 (1981), 199.  doi: 10.1016/0005-1098(81)90095-9.  Google Scholar

[12]

P. D. Roberts, Optimal control of nonlinear systems with model-reality differences,, \emph{Proceedings of the 31st IEEE Conference on Decision and Control}, 1 (1992), 257.   Google Scholar

show all references

References:
[1]

V. M. Becerra and P. D. Roberts, Dynamic integrated system optimization and parameter estimation for discrete time optimal control of nonlinear systems,, \emph{Int. J. Control}, 63 (1996), 257.  doi: 10.1080/00207179608921843.  Google Scholar

[2]

A. E. Bryson and Y. C. Ho, Applied Optimal Control,, Hemisphere Publishing Company, (1975).   Google Scholar

[3]

S. L. Kek and A. A. Mohd Ismail, Optimal control of discrete-time linear stochastic dynamic system with model-reality differences,, in \emph{Proceeding of International Conference on Machine Learning and Computing (ICML 2009)}, (2009), 10.   Google Scholar

[4]

S. L. Kek, K. L. Teo and A. A. Mohd Ismail, An integrated optimal control algorithm for discrete-time nonlinear stochastic system,, \emph{International Journal of Control}, 83 (2010), 2536.  doi: 10.1080/00207179.2010.531766.  Google Scholar

[5]

S. L. Kek, K. L. Teo and A. A. Mohd Ismail, Filtering solution of nonlinear stochastic optimal control problem in discrete-time with model-reality differences,, \emph{Numerical Algebra, 2 (2012), 207.  doi: 10.3934/naco.2012.2.207.  Google Scholar

[6]

S. L. Kek, A. A. Mohd Ismail, K. L. Teo and A. Rohanin, An iterative algorithm based on model-reality differences for discrete-time nonlinear stochastic optimal control problems,, \emph{Numerical Algebra, 3 (2013), 109.  doi: 10.3934/naco.2013.3.109.  Google Scholar

[7]

D. E. Kirk, Optimal Control Theory: An Introduction,, Mineola, (2004).   Google Scholar

[8]

F. L. Lewis and V. L. Syrmos, Optimal Control,, 2nd ed, (1995).   Google Scholar

[9]

A. A. Mohd Ismail and S. L. Kek, Optimal control of nonlinear discrete-time stochastic system with model-reality differences,, in \emph{2009 IEEE International Conference on Control and Automation}, (2009), 9.   Google Scholar

[10]

A. A. Mohd Ismail, A. Rohanin, S. L. Kek and K. L. Teo, Computational integrated optimal control and estimation with model information for discrete-time nonlinear stochastic dynamic system,, in \emph{Proceeding of the 2010 IRAST Internation Congress on Computer Applications and Computational Science (CACS 2010)}, (2010), 4.   Google Scholar

[11]

P. D. Roberts and T. W. C. Williams, On an algorithm for combined system optimization and parameter estimation,, \emph{Automatica}, 17 (1981), 199.  doi: 10.1016/0005-1098(81)90095-9.  Google Scholar

[12]

P. D. Roberts, Optimal control of nonlinear systems with model-reality differences,, \emph{Proceedings of the 31st IEEE Conference on Decision and Control}, 1 (1992), 257.   Google Scholar

[1]

Sie Long Kek, Mohd Ismail Abd Aziz, Kok Lay Teo, Rohanin Ahmad. An iterative algorithm based on model-reality differences for discrete-time nonlinear stochastic optimal control problems. Numerical Algebra, Control & Optimization, 2013, 3 (1) : 109-125. doi: 10.3934/naco.2013.3.109
[2]

Sie Long Kek, Mohd Ismail Abd Aziz, Kok Lay Teo. A gradient algorithm for optimal control problems with model-reality differences. Numerical Algebra, Control & Optimization, 2015, 5 (3) : 251-266. doi: 10.3934/naco.2015.5.251
[3]

Sie Long Kek, Kok Lay Teo, Mohd Ismail Abd Aziz. Filtering solution of nonlinear stochastic optimal control problem in discrete-time with model-reality differences. Numerical Algebra, Control & Optimization, 2012, 2 (1) : 207-222. doi: 10.3934/naco.2012.2.207
[4]

N. U. Ahmed. Existence of optimal output feedback control law for a class of uncertain infinite dimensional stochastic systems: A direct approach. Evolution Equations & Control Theory, 2012, 1 (2) : 235-250. doi: 10.3934/eect.2012.1.235
[5]

Toufik Bakir, Bernard Bonnard, Jérémy Rouot. A case study of optimal input-output system with sampled-data control: Ding et al. force and fatigue muscular control model. Networks & Heterogeneous Media, 2019, 14 (1) : 79-100. doi: 10.3934/nhm.2019005
[6]

Rein Luus. Optimal control of oscillatory systems by iterative dynamic programming. Journal of Industrial & Management Optimization, 2008, 4 (1) : 1-15. doi: 10.3934/jimo.2008.4.1
[7]

Kangbo Bao, Libin Rong, Qimin Zhang. Analysis of a stochastic SIRS model with interval parameters. Discrete & Continuous Dynamical Systems - B, 2019, 24 (9) : 4827-4849. doi: 10.3934/dcdsb.2019033
[8]

Francesco Cordoni, Luca Di Persio. Optimal control for the stochastic FitzHugh-Nagumo model with recovery variable. Evolution Equations & Control Theory, 2018, 7 (4) : 571-585. doi: 10.3934/eect.2018027
[9]

James P. Nelson, Mark J. Balas. Direct model reference adaptive control of linear systems with input/output delays. Numerical Algebra, Control & Optimization, 2013, 3 (3) : 445-462. doi: 10.3934/naco.2013.3.445
[10]

Margherita Carletti, Matteo Montani, Valentina Meschini, Marzia Bianchi, Lucia Radici. Stochastic modelling of PTEN regulation in brain tumors: A model for glioblastoma multiforme. Mathematical Biosciences & Engineering, 2015, 12 (5) : 965-981. doi: 10.3934/mbe.2015.12.965
[11]

Ugur G. Abdulla. On the optimal control of the free boundary problems for the second order parabolic equations. II. Convergence of the method of finite differences. Inverse Problems & Imaging, 2016, 10 (4) : 869-898. doi: 10.3934/ipi.2016025
[12]

Guangzhou Chen, Guijian Liu, Jiaquan Wang, Ruzhong Li. Identification of water quality model parameters using artificial bee colony algorithm. Numerical Algebra, Control & Optimization, 2012, 2 (1) : 157-165. doi: 10.3934/naco.2012.2.157
[13]

J. Frédéric Bonnans, Justina Gianatti, Francisco J. Silva. On the convergence of the Sakawa-Shindo algorithm in stochastic control. Mathematical Control & Related Fields, 2016, 6 (3) : 391-406. doi: 10.3934/mcrf.2016008
[14]

Jiongmin Yong. Stochastic optimal control — A concise introduction. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020027
[15]

Yanqin Bai, Yudan Wei, Qian Li. An optimal trade-off model for portfolio selection with sensitivity of parameters. Journal of Industrial & Management Optimization, 2017, 13 (2) : 947-965. doi: 10.3934/jimo.2016055
[16]

Yuan Zhao, Shunfu Jin, Wuyi Yue. Adjustable admission control with threshold in centralized CR networks: Analysis and optimization. Journal of Industrial & Management Optimization, 2015, 11 (4) : 1393-1408. doi: 10.3934/jimo.2015.11.1393
[17]

Yi Zhang, Xiao-Li Ma. Research on image digital watermarking optimization algorithm under virtual reality technology. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 1427-1440. doi: 10.3934/dcdss.2019098
[18]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437
[19]

Fulvia Confortola, Elisa Mastrogiacomo. Optimal control for stochastic heat equation with memory. Evolution Equations & Control Theory, 2014, 3 (1) : 35-58. doi: 10.3934/eect.2014.3.35
[20]

Liangquan Zhang, Qing Zhou, Juan Yang. Necessary condition for optimal control of doubly stochastic systems. Mathematical Control & Related Fields, 2020, 10 (2) : 379-403. doi: 10.3934/mcrf.2020002

 Impact Factor: 

Tools

Metrics

  • PDF downloads (21)
  • HTML views (0)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2020 American Institute of Mathematical Sciences