2012, 2(1): 207-222. doi: 10.3934/naco.2012.2.207

Filtering solution of nonlinear stochastic optimal control problem in discrete-time with model-reality differences

1. 

Department of Mathematics, Universiti Tun Hussein Onn Malaysia, 86400 Parit Raja, Malaysia

2. 

Department of Mathematics and Statistics, Curtin University, G.P.O. Box U1987, Perth, WA 6845

3. 

Department of Mathematics, Universiti Teknologi Malaysia, 81310 UTM, Skudai, Malaysia

Received  March 2011 Revised  July 2011 Published  March 2012

In this paper, we propose an efficient algorithm for solving a nonlinear stochastic optimal control problem in discrete-time, where the true filtered solution of the original optimal control problem is obtained through solving a linear model-based optimal control problem with adjustable parameters iteratively. The adjustments of these parameters are based on the differences between the real plant and the linear model that are measured. The main feature of the algorithm proposed is the integration of system optimization and parameter estimation in an interactive way so that the correct filtered solution of the original optimal control problem is obtained when the convergence is achieved. For illustration, a nonlinear continuous stirred reactor tank problem is studied. The simulation results obtained demonstrate the efficiency of the algorithm proposed.
Citation: Sie Long Kek, Kok Lay Teo, Mohd Ismail Abd Aziz. Filtering solution of nonlinear stochastic optimal control problem in discrete-time with model-reality differences. Numerical Algebra, Control & Optimization, 2012, 2 (1) : 207-222. doi: 10.3934/naco.2012.2.207
References:
[1]

V. M. Becerra, "Development and Applications of Novel Optimal Control Algorithms,", Ph.D. thesis, (1994). Google Scholar

[2]

V. M. Becerra and P. D. Roberts, Dynamic integrated system optimization and parameter estimation for discrete time optimal control of nonlinear systems,, Int. J. Control, 63 (1996), 257. doi: 10.1080/00207179608921843. Google Scholar

[3]

V. M. Becerra and P. D. Roberts, Application of a novel optimal control algorithm to a benchmark fed-batch fermentation process,, Trans. Inst. Measurement Control, 20 (1998), 11. doi: 10.1177/014233129802000103. Google Scholar

[4]

A. E. Bryson and Y. C. Ho, "Applied Optimal Control,", Hemisphere Publishing Company, (1975). Google Scholar

[5]

A. E. Bryson, "Applied Linear Optimal Control, Examples and Algorithms,", Cambridge University Press, (2002). Google Scholar

[6]

Y. Y. Haimes and D. A. Wismer, A computational approach to the combined problem of optimization and parameter estimation,, Automatica, 8 (1972), 337. doi: 10.1016/0005-1098(72)90052-0. Google Scholar

[7]

M. H. Hu, Q. Gao and H. H. Shao, Optimal control of a class of non-linear discrete-continuous hybrid systems,, in, (2006), 835. Google Scholar

[8]

M. H. Hu, Y. S. Wang and H. H Shao, Costate prediction based optimal control for non-linear hybrid systems,, ISA Transactions, 47 (2008), 113. doi: 10.1016/j.isatra.2007.06.001. Google Scholar

[9]

S. L. Kek and A. A. Mohd Ismail, Optimal control of discrete-time linear stochastic dynamic system with model-reality differences,, in, (2009), 10. Google Scholar

[10]

J. S. Kong and B. W. Wan, The study of integrated optimal control approach for complex system under network environment,, Computing Technology and Automation, 22 (2003), 23. Google Scholar

[11]

F. L. Lewis, "Optimal Control,", John Wiley and Sons, (1986). Google Scholar

[12]

F. L. Lewis, "Applied Optimal Control and Estimation: Digital Design and Implementation,", Prentice Hall, (1992). Google Scholar

[13]

J. M. Li, B. W. Wan and Z. L. Huang, Optimal control of nonlinear discrete systems with model-reality differences,, Control Theory and Applications, 16 (1999), 32. Google Scholar

[14]

A. A. Mohd Ismail and S. L. Kek, Optimal control of nonlinear discrete-time stochastic system with model-reality differences,, in, (2009), 9. Google Scholar

[15]

A. A. Mohd Ismail, A. Rohanin, S. L. Kek and K. L. Teo, Computational integrated optimal control and estimation with model information for discrete-time nonlinear stochastic dynamic system,, in, (2010), 4. Google Scholar

[16]

W. H. Ray, "Advanced Process Control,", McGraw-Hill, (1989). Google Scholar

[17]

P. D. Roberts, An algorithm for steady-state system optimization and parameter estimation,, Int. J. Systems Science, 10 (1979), 719. doi: 10.1080/00207727908941614. Google Scholar

[18]

P. D. Roberts and T. W. C. Williams, On an algorithm for combined system optimization and parameter estimation,, Automatica, 17 (1981), 199. doi: 10.1016/0005-1098(81)90095-9. Google Scholar

[19]

P. D. Roberts, Optimal control of nonlinear systems with model-reality differences,, Proceedings of the 31st IEEE Conference on Decision and Control, 1 (1992), 257. Google Scholar

[20]

P. D. Roberts and V. M. Becerra, Optimal control of a class of discrete-continuous non-linear systems decomposition and hierarchical structure,, Automatica, 37 (2001), 1757. doi: 10.1016/S0005-1098(01)00141-8. Google Scholar

[21]

Y. Zhang and S. Y. Li, DISOPE distributed model predictive control of cascade systems with network communication,, Journal of Control Theory and Applications, 2 (2005), 131. doi: 10.1007/s11768-005-0005-6. Google Scholar

show all references

References:
[1]

V. M. Becerra, "Development and Applications of Novel Optimal Control Algorithms,", Ph.D. thesis, (1994). Google Scholar

[2]

V. M. Becerra and P. D. Roberts, Dynamic integrated system optimization and parameter estimation for discrete time optimal control of nonlinear systems,, Int. J. Control, 63 (1996), 257. doi: 10.1080/00207179608921843. Google Scholar

[3]

V. M. Becerra and P. D. Roberts, Application of a novel optimal control algorithm to a benchmark fed-batch fermentation process,, Trans. Inst. Measurement Control, 20 (1998), 11. doi: 10.1177/014233129802000103. Google Scholar

[4]

A. E. Bryson and Y. C. Ho, "Applied Optimal Control,", Hemisphere Publishing Company, (1975). Google Scholar

[5]

A. E. Bryson, "Applied Linear Optimal Control, Examples and Algorithms,", Cambridge University Press, (2002). Google Scholar

[6]

Y. Y. Haimes and D. A. Wismer, A computational approach to the combined problem of optimization and parameter estimation,, Automatica, 8 (1972), 337. doi: 10.1016/0005-1098(72)90052-0. Google Scholar

[7]

M. H. Hu, Q. Gao and H. H. Shao, Optimal control of a class of non-linear discrete-continuous hybrid systems,, in, (2006), 835. Google Scholar

[8]

M. H. Hu, Y. S. Wang and H. H Shao, Costate prediction based optimal control for non-linear hybrid systems,, ISA Transactions, 47 (2008), 113. doi: 10.1016/j.isatra.2007.06.001. Google Scholar

[9]

S. L. Kek and A. A. Mohd Ismail, Optimal control of discrete-time linear stochastic dynamic system with model-reality differences,, in, (2009), 10. Google Scholar

[10]

J. S. Kong and B. W. Wan, The study of integrated optimal control approach for complex system under network environment,, Computing Technology and Automation, 22 (2003), 23. Google Scholar

[11]

F. L. Lewis, "Optimal Control,", John Wiley and Sons, (1986). Google Scholar

[12]

F. L. Lewis, "Applied Optimal Control and Estimation: Digital Design and Implementation,", Prentice Hall, (1992). Google Scholar

[13]

J. M. Li, B. W. Wan and Z. L. Huang, Optimal control of nonlinear discrete systems with model-reality differences,, Control Theory and Applications, 16 (1999), 32. Google Scholar

[14]

A. A. Mohd Ismail and S. L. Kek, Optimal control of nonlinear discrete-time stochastic system with model-reality differences,, in, (2009), 9. Google Scholar

[15]

A. A. Mohd Ismail, A. Rohanin, S. L. Kek and K. L. Teo, Computational integrated optimal control and estimation with model information for discrete-time nonlinear stochastic dynamic system,, in, (2010), 4. Google Scholar

[16]

W. H. Ray, "Advanced Process Control,", McGraw-Hill, (1989). Google Scholar

[17]

P. D. Roberts, An algorithm for steady-state system optimization and parameter estimation,, Int. J. Systems Science, 10 (1979), 719. doi: 10.1080/00207727908941614. Google Scholar

[18]

P. D. Roberts and T. W. C. Williams, On an algorithm for combined system optimization and parameter estimation,, Automatica, 17 (1981), 199. doi: 10.1016/0005-1098(81)90095-9. Google Scholar

[19]

P. D. Roberts, Optimal control of nonlinear systems with model-reality differences,, Proceedings of the 31st IEEE Conference on Decision and Control, 1 (1992), 257. Google Scholar

[20]

P. D. Roberts and V. M. Becerra, Optimal control of a class of discrete-continuous non-linear systems decomposition and hierarchical structure,, Automatica, 37 (2001), 1757. doi: 10.1016/S0005-1098(01)00141-8. Google Scholar

[21]

Y. Zhang and S. Y. Li, DISOPE distributed model predictive control of cascade systems with network communication,, Journal of Control Theory and Applications, 2 (2005), 131. doi: 10.1007/s11768-005-0005-6. Google Scholar

[1]

Sie Long Kek, Mohd Ismail Abd Aziz, Kok Lay Teo, Rohanin Ahmad. An iterative algorithm based on model-reality differences for discrete-time nonlinear stochastic optimal control problems. Numerical Algebra, Control & Optimization, 2013, 3 (1) : 109-125. doi: 10.3934/naco.2013.3.109

[2]

Sie Long Kek, Mohd Ismail Abd Aziz, Kok Lay Teo. A gradient algorithm for optimal control problems with model-reality differences. Numerical Algebra, Control & Optimization, 2015, 5 (3) : 251-266. doi: 10.3934/naco.2015.5.251

[3]

Sie Long Kek, Mohd Ismail Abd Aziz. Output regulation for discrete-time nonlinear stochastic optimal control problems with model-reality differences. Numerical Algebra, Control & Optimization, 2015, 5 (3) : 275-288. doi: 10.3934/naco.2015.5.275

[4]

Ellina Grigorieva, Evgenii Khailov. Optimal control of a nonlinear model of economic growth. Conference Publications, 2007, 2007 (Special) : 456-466. doi: 10.3934/proc.2007.2007.456

[5]

Francesco Cordoni, Luca Di Persio. Optimal control for the stochastic FitzHugh-Nagumo model with recovery variable. Evolution Equations & Control Theory, 2018, 7 (4) : 571-585. doi: 10.3934/eect.2018027

[6]

Filipe Rodrigues, Cristiana J. Silva, Delfim F. M. Torres, Helmut Maurer. Optimal control of a delayed HIV model. Discrete & Continuous Dynamical Systems - B, 2018, 23 (1) : 443-458. doi: 10.3934/dcdsb.2018030

[7]

Cristiana J. Silva, Helmut Maurer, Delfim F. M. Torres. Optimal control of a Tuberculosis model with state and control delays. Mathematical Biosciences & Engineering, 2017, 14 (1) : 321-337. doi: 10.3934/mbe.2017021

[8]

Chonghu Guan, Fahuai Yi, Xiaoshan Chen. A fully nonlinear free boundary problem arising from optimal dividend and risk control model. Mathematical Control & Related Fields, 2019, 9 (3) : 425-452. doi: 10.3934/mcrf.2019020

[9]

Rong Liu, Feng-Qin Zhang, Yuming Chen. Optimal contraception control for a nonlinear population model with size structure and a separable mortality. Discrete & Continuous Dynamical Systems - B, 2016, 21 (10) : 3603-3618. doi: 10.3934/dcdsb.2016112

[10]

Alexandr Golodnikov, Stan Uryasev, Grigoriy Zrazhevsky, Yevgeny Macheret, A. Alexandre Trindade. Optimization of composition and processing parameters for alloy development: a statistical model-based approach. Journal of Industrial & Management Optimization, 2007, 3 (3) : 489-501. doi: 10.3934/jimo.2007.3.489

[11]

Rein Luus. Optimal control of oscillatory systems by iterative dynamic programming. Journal of Industrial & Management Optimization, 2008, 4 (1) : 1-15. doi: 10.3934/jimo.2008.4.1

[12]

Hang-Chin Lai, Jin-Chirng Lee, Shuh-Jye Chern. A variational problem and optimal control. Journal of Industrial & Management Optimization, 2011, 7 (4) : 967-975. doi: 10.3934/jimo.2011.7.967

[13]

Enrique Fernández-Cara, Juan Límaco, Laurent Prouvée. Optimal control of a two-equation model of radiotherapy. Mathematical Control & Related Fields, 2018, 8 (1) : 117-133. doi: 10.3934/mcrf.2018005

[14]

Erika Asano, Louis J. Gross, Suzanne Lenhart, Leslie A. Real. Optimal control of vaccine distribution in a rabies metapopulation model. Mathematical Biosciences & Engineering, 2008, 5 (2) : 219-238. doi: 10.3934/mbe.2008.5.219

[15]

Huaying Guo, Jin Liang. An optimal control model of carbon reduction and trading. Mathematical Control & Related Fields, 2016, 6 (4) : 535-550. doi: 10.3934/mcrf.2016015

[16]

IvÁn Area, FaÏÇal NdaÏrou, Juan J. Nieto, Cristiana J. Silva, Delfim F. M. Torres. Ebola model and optimal control with vaccination constraints. Journal of Industrial & Management Optimization, 2018, 14 (2) : 427-446. doi: 10.3934/jimo.2017054

[17]

Qingkai Kong, Zhipeng Qiu, Zi Sang, Yun Zou. Optimal control of a vector-host epidemics model. Mathematical Control & Related Fields, 2011, 1 (4) : 493-508. doi: 10.3934/mcrf.2011.1.493

[18]

Erin N. Bodine, Louis J. Gross, Suzanne Lenhart. Optimal control applied to a model for species augmentation. Mathematical Biosciences & Engineering, 2008, 5 (4) : 669-680. doi: 10.3934/mbe.2008.5.669

[19]

Jingtao Shi, Juanjuan Xu, Huanshui Zhang. Stochastic recursive optimal control problem with time delay and applications. Mathematical Control & Related Fields, 2015, 5 (4) : 859-888. doi: 10.3934/mcrf.2015.5.859

[20]

Qun Lin, Ryan Loxton, Kok Lay Teo. The control parameterization method for nonlinear optimal control: A survey. Journal of Industrial & Management Optimization, 2014, 10 (1) : 275-309. doi: 10.3934/jimo.2014.10.275

 Impact Factor: 

Metrics

  • PDF downloads (4)
  • HTML views (0)
  • Cited by (4)

Other articles
by authors

[Back to Top]