2013, 3(1): 109-125. doi: 10.3934/naco.2013.3.109

An iterative algorithm based on model-reality differences for discrete-time nonlinear stochastic optimal control problems

1. 

Department of Mathematics, Universiti Tun Hussein Onn Malaysia, 86400 Parit Raja, Malaysia

2. 

Department of Mathematics, Universiti Teknologi Malaysia, 81310 UTM, Skudai, Malaysia, Malaysia

3. 

Department of Mathematics and Statistics, Curtin University, Perth, W.A. 6845

Received  November 2011 Revised  December 2012 Published  January 2013

An iterative algorithm, which is called the integrated optimal control and parameter estimation algorithm, is developed for solving a discrete time nonlinear stochastic control problem. It is based on the integration of the principle of model-reality differences and Kalman filtering theory, where the dynamic integrated system optimization and parameter estimation algorithm are used interactively. In this approach, the weighted least-square output residual is included in the cost function by appropriately monitoring the weighted matrix. An improved linear quadratic Gaussian optimal control model, rather than the original optimal control problem, is solved. Subsequently, the model optimum is updated using the adjusted parameters induced by the differences between the real plant and the model used. These updated solutions converge to the true optimum, despite model-reality differences. For illustration, the optimal control of a nonlinear continuous stirred tank reactor problem is considered and solved by using the method proposed.
Citation: Sie Long Kek, Mohd Ismail Abd Aziz, Kok Lay Teo, Rohanin Ahmad. An iterative algorithm based on model-reality differences for discrete-time nonlinear stochastic optimal control problems. Numerical Algebra, Control & Optimization, 2013, 3 (1) : 109-125. doi: 10.3934/naco.2013.3.109
References:
[1]

V. M. Becerra, "Development and Applications of Novel Optimal Control Algorithms,", Ph. D. thesis, (1994).   Google Scholar

[2]

V. M. Becerra and P. D. Roberts, Dynamic integrated system optimization and parameter estimation for discrete time optimal control of nonlinear systems,, Int. J. Control, 63 (1996), 257.  doi: 10.1080/00207179608921843.  Google Scholar

[3]

V. M. Becerra and P. D. Roberts, Application of a novel optimal control algorithm to a benchmark fed-batch fermentation process,, Trans. Inst. Measurement Control, 20 (1998), 11.  doi: 10.1177/014233129802000103.  Google Scholar

[4]

A. E. Bryson and Y. C. Ho, "Applied Optimal Control,", Hemisphere Publishing Company, (1975).   Google Scholar

[5]

A. E. Bryson, "Applied Linear Optimal Control, Examples and Algorithms,", Cambridge University Press, (2002).   Google Scholar

[6]

P. D. Christofides and N. H. El-Farra, "Control of Nonlinear and Hybrid Process Systems: Designs for Uncertainty, Constraints and Time-Delays,", Springer-Verlag, (2005).   Google Scholar

[7]

T. E. Dabbous, Adaptive control of nonlinear systems using fussy systems,, Journal of Industrial and Management Optimization, 6 (2010), 861.  doi: 10.3934/jimo.2010.6.861.  Google Scholar

[8]

Y. Y. Haimes and D. A. Wismer, A computational approach to the combined problem of optimization and parameter estimation,, Automatica, 8 (1972), 337.  doi: 10.1016/0005-1098(72)90052-0.  Google Scholar

[9]

M. H. Hu, Q. Gao and H. H. Shao, Optimal control of a class of non-linear discrete-continuous hybrid systems,, in, (2006), 21.   Google Scholar

[10]

M. H. Hu, Y. S. Wang and H. H Shao, Costate prediction based optimal control for non-linear hybrid systems,, ISA Transactions, 47 (2008), 113.  doi: 10.1016/j.isatra.2007.06.001.  Google Scholar

[11]

S. L. Kek, K. L. Teo and M. I. A. Aziz, An integrated optimal control algorithm for discrete-time nonlinear stochastic system,, International Journal of Control, 83 (2010), 2536.  doi: 10.1080/00207179.2010.531766.  Google Scholar

[12]

J. S. Kong and B. W. Wan, The study of integrated optimal control approach for complex system under network environment,, Computing Technology and Automation, 22 (2003), 23.   Google Scholar

[13]

F. L. Lewis, "Optimal Control,", John Wiley and Sons, (1986).   Google Scholar

[14]

F. L. Lewis, "Applied Optimal Control and Estimation: Digital Design and Implementation,", Prentice Hall, (1992).   Google Scholar

[15]

J. M. Li, B. W. Wan and Z. L. Huang, Optimal control of nonlinear discrete systems with model-reality differences,, Control Theory and Applications, 16 (1999), 32.   Google Scholar

[16]

R. Loxton, K. L. Teo and V. Rehbock, Computational method for a class of switched system optimal control problems,, IEEE Transactions on Automatic Control, 54 (2009), 2455.  doi: 10.1109/TAC.2009.2029310.  Google Scholar

[17]

R. Loxton, K. L. Teo, V. Rehbock and K. F. C. Yiu, Optimal control problems with a continuous inequality constraint on the state and the control,, Automatica, 45 (2009), 2250.  doi: 10.1016/j.automatica.2009.05.029.  Google Scholar

[18]

W. H. Ray, "Advanced Process Control,", McGraw-Hill, (1989).   Google Scholar

[19]

S. Richchardson and S. Wang, The viscosity approximation to Hamiltonian-Jacobi-Bellman equation in optimal feedback control: upper bounds for extended domains,, Journal of Industrial and Management Optimization, 6 (2010), 161.  doi: 10.3934/jimo.2010.6.161.  Google Scholar

[20]

P. D. Roberts, An algorithm for steady-state system optimization and parameter estimation,, Int. J. Systems Science, 10 (1979), 719.  doi: 10.1080/00207727908941614.  Google Scholar

[21]

P. D. Roberts and T. W. C. Williams, On an algorithm for combined system optimization and parameter estimation,, Automatica, 17 (1981), 199.  doi: 10.1016/0005-1098(81)90095-9.  Google Scholar

[22]

P. D. Roberts, Optimal control of nonlinear systems with model-reality differences,, Proceedings of the 31st IEEE Conference on Decision and Control, 1 (1992), 257.   Google Scholar

[23]

P. D. Roberts and V. M. Becerra, Optimal control of a class of discrete-continuous non-linear systems decomposition and hierarchical structure,, Automatica, 37 (2001), 1757.  doi: 10.1016/S0005-1098(01)00141-8.  Google Scholar

[24]

C. Z. Wu, K. L. Teo and V. Rehbock, Optimal control of piecewise affine systems with piecewise affine state feedback,, Journal of Industrial and Management Optimization, 5 (2009), 737.  doi: 10.3934/jimo.2009.5.737.  Google Scholar

[25]

Y. Zhang and S. Y. Li, DISOPE distributed model predictive control of cascade systems with network communication,, Journal of Control Theory and Applications, 2 (2005), 131.  doi: 10.1007/s11768-005-0005-6.  Google Scholar

show all references

References:
[1]

V. M. Becerra, "Development and Applications of Novel Optimal Control Algorithms,", Ph. D. thesis, (1994).   Google Scholar

[2]

V. M. Becerra and P. D. Roberts, Dynamic integrated system optimization and parameter estimation for discrete time optimal control of nonlinear systems,, Int. J. Control, 63 (1996), 257.  doi: 10.1080/00207179608921843.  Google Scholar

[3]

V. M. Becerra and P. D. Roberts, Application of a novel optimal control algorithm to a benchmark fed-batch fermentation process,, Trans. Inst. Measurement Control, 20 (1998), 11.  doi: 10.1177/014233129802000103.  Google Scholar

[4]

A. E. Bryson and Y. C. Ho, "Applied Optimal Control,", Hemisphere Publishing Company, (1975).   Google Scholar

[5]

A. E. Bryson, "Applied Linear Optimal Control, Examples and Algorithms,", Cambridge University Press, (2002).   Google Scholar

[6]

P. D. Christofides and N. H. El-Farra, "Control of Nonlinear and Hybrid Process Systems: Designs for Uncertainty, Constraints and Time-Delays,", Springer-Verlag, (2005).   Google Scholar

[7]

T. E. Dabbous, Adaptive control of nonlinear systems using fussy systems,, Journal of Industrial and Management Optimization, 6 (2010), 861.  doi: 10.3934/jimo.2010.6.861.  Google Scholar

[8]

Y. Y. Haimes and D. A. Wismer, A computational approach to the combined problem of optimization and parameter estimation,, Automatica, 8 (1972), 337.  doi: 10.1016/0005-1098(72)90052-0.  Google Scholar

[9]

M. H. Hu, Q. Gao and H. H. Shao, Optimal control of a class of non-linear discrete-continuous hybrid systems,, in, (2006), 21.   Google Scholar

[10]

M. H. Hu, Y. S. Wang and H. H Shao, Costate prediction based optimal control for non-linear hybrid systems,, ISA Transactions, 47 (2008), 113.  doi: 10.1016/j.isatra.2007.06.001.  Google Scholar

[11]

S. L. Kek, K. L. Teo and M. I. A. Aziz, An integrated optimal control algorithm for discrete-time nonlinear stochastic system,, International Journal of Control, 83 (2010), 2536.  doi: 10.1080/00207179.2010.531766.  Google Scholar

[12]

J. S. Kong and B. W. Wan, The study of integrated optimal control approach for complex system under network environment,, Computing Technology and Automation, 22 (2003), 23.   Google Scholar

[13]

F. L. Lewis, "Optimal Control,", John Wiley and Sons, (1986).   Google Scholar

[14]

F. L. Lewis, "Applied Optimal Control and Estimation: Digital Design and Implementation,", Prentice Hall, (1992).   Google Scholar

[15]

J. M. Li, B. W. Wan and Z. L. Huang, Optimal control of nonlinear discrete systems with model-reality differences,, Control Theory and Applications, 16 (1999), 32.   Google Scholar

[16]

R. Loxton, K. L. Teo and V. Rehbock, Computational method for a class of switched system optimal control problems,, IEEE Transactions on Automatic Control, 54 (2009), 2455.  doi: 10.1109/TAC.2009.2029310.  Google Scholar

[17]

R. Loxton, K. L. Teo, V. Rehbock and K. F. C. Yiu, Optimal control problems with a continuous inequality constraint on the state and the control,, Automatica, 45 (2009), 2250.  doi: 10.1016/j.automatica.2009.05.029.  Google Scholar

[18]

W. H. Ray, "Advanced Process Control,", McGraw-Hill, (1989).   Google Scholar

[19]

S. Richchardson and S. Wang, The viscosity approximation to Hamiltonian-Jacobi-Bellman equation in optimal feedback control: upper bounds for extended domains,, Journal of Industrial and Management Optimization, 6 (2010), 161.  doi: 10.3934/jimo.2010.6.161.  Google Scholar

[20]

P. D. Roberts, An algorithm for steady-state system optimization and parameter estimation,, Int. J. Systems Science, 10 (1979), 719.  doi: 10.1080/00207727908941614.  Google Scholar

[21]

P. D. Roberts and T. W. C. Williams, On an algorithm for combined system optimization and parameter estimation,, Automatica, 17 (1981), 199.  doi: 10.1016/0005-1098(81)90095-9.  Google Scholar

[22]

P. D. Roberts, Optimal control of nonlinear systems with model-reality differences,, Proceedings of the 31st IEEE Conference on Decision and Control, 1 (1992), 257.   Google Scholar

[23]

P. D. Roberts and V. M. Becerra, Optimal control of a class of discrete-continuous non-linear systems decomposition and hierarchical structure,, Automatica, 37 (2001), 1757.  doi: 10.1016/S0005-1098(01)00141-8.  Google Scholar

[24]

C. Z. Wu, K. L. Teo and V. Rehbock, Optimal control of piecewise affine systems with piecewise affine state feedback,, Journal of Industrial and Management Optimization, 5 (2009), 737.  doi: 10.3934/jimo.2009.5.737.  Google Scholar

[25]

Y. Zhang and S. Y. Li, DISOPE distributed model predictive control of cascade systems with network communication,, Journal of Control Theory and Applications, 2 (2005), 131.  doi: 10.1007/s11768-005-0005-6.  Google Scholar

[1]

Hong Niu, Zhijiang Feng, Qijin Xiao, Yajun Zhang. A PID control method based on optimal control strategy. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 117-126. doi: 10.3934/naco.2020019

[2]

Yuan Tan, Qingyuan Cao, Lan Li, Tianshi Hu, Min Su. A chance-constrained stochastic model predictive control problem with disturbance feedback. Journal of Industrial & Management Optimization, 2021, 17 (1) : 67-79. doi: 10.3934/jimo.2019099

[3]

Youming Guo, Tingting Li. Optimal control strategies for an online game addiction model with low and high risk exposure. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020347

[4]

Bernard Bonnard, Jérémy Rouot. Geometric optimal techniques to control the muscular force response to functional electrical stimulation using a non-isometric force-fatigue model. Journal of Geometric Mechanics, 2020  doi: 10.3934/jgm.2020032

[5]

Zuliang Lu, Fei Huang, Xiankui Wu, Lin Li, Shang Liu. Convergence and quasi-optimality of $ L^2- $norms based an adaptive finite element method for nonlinear optimal control problems. Electronic Research Archive, 2020, 28 (4) : 1459-1486. doi: 10.3934/era.2020077

[6]

Lars Grüne, Matthias A. Müller, Christopher M. Kellett, Steven R. Weller. Strict dissipativity for discrete time discounted optimal control problems. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020046

[7]

Hai Huang, Xianlong Fu. Optimal control problems for a neutral integro-differential system with infinite delay. Evolution Equations & Control Theory, 2020  doi: 10.3934/eect.2020107

[8]

Pierluigi Colli, Gianni Gilardi, Jürgen Sprekels. Deep quench approximation and optimal control of general Cahn–Hilliard systems with fractional operators and double obstacle potentials. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 243-271. doi: 10.3934/dcdss.2020213

[9]

Stefan Doboszczak, Manil T. Mohan, Sivaguru S. Sritharan. Pontryagin maximum principle for the optimal control of linearized compressible navier-stokes equations with state constraints. Evolution Equations & Control Theory, 2020  doi: 10.3934/eect.2020110

[10]

Tien-Yu Lin, Bhaba R. Sarker, Chien-Jui Lin. An optimal setup cost reduction and lot size for economic production quantity model with imperfect quality and quantity discounts. Journal of Industrial & Management Optimization, 2021, 17 (1) : 467-484. doi: 10.3934/jimo.2020043

[11]

Eduard Feireisl, Elisabetta Rocca, Giulio Schimperna, Arghir Zarnescu. Weak sequential stability for a nonlinear model of nematic electrolytes. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 219-241. doi: 10.3934/dcdss.2020366

[12]

Siyang Cai, Yongmei Cai, Xuerong Mao. A stochastic differential equation SIS epidemic model with regime switching. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020317

[13]

Wolfgang Riedl, Robert Baier, Matthias Gerdts. Optimization-based subdivision algorithm for reachable sets. Journal of Computational Dynamics, 2021, 8 (1) : 99-130. doi: 10.3934/jcd.2021005

[14]

José Madrid, João P. G. Ramos. On optimal autocorrelation inequalities on the real line. Communications on Pure & Applied Analysis, 2021, 20 (1) : 369-388. doi: 10.3934/cpaa.2020271

[15]

Omid Nikan, Seyedeh Mahboubeh Molavi-Arabshai, Hossein Jafari. Numerical simulation of the nonlinear fractional regularized long-wave model arising in ion acoustic plasma waves. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020466

[16]

Hui Lv, Xing'an Wang. Dissipative control for uncertain singular markovian jump systems via hybrid impulsive control. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 127-142. doi: 10.3934/naco.2020020

[17]

Cheng Peng, Zhaohui Tang, Weihua Gui, Qing Chen, Jing He. A bidirectional weighted boundary distance algorithm for time series similarity computation based on optimized sliding window size. Journal of Industrial & Management Optimization, 2021, 17 (1) : 205-220. doi: 10.3934/jimo.2019107

[18]

Sergio Conti, Georg Dolzmann. Optimal laminates in single-slip elastoplasticity. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 1-16. doi: 10.3934/dcdss.2020302

[19]

Haili Yuan, Yijun Hu. Optimal investment for an insurer under liquid reserves. Journal of Industrial & Management Optimization, 2021, 17 (1) : 339-355. doi: 10.3934/jimo.2019114

[20]

Awais Younus, Zoubia Dastgeer, Nudrat Ishaq, Abdul Ghaffar, Kottakkaran Sooppy Nisar, Devendra Kumar. On the observability of conformable linear time-invariant control systems. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020444

 Impact Factor: 

Metrics

  • PDF downloads (35)
  • HTML views (0)
  • Cited by (0)

[Back to Top]