Optimal synchronization control of multiple euler-lagrange systems via event-triggered reinforcement learning

Yuan Xu; Xin Jin; Saiwei Wang; Yang Tang

doi:10.3934/dcdss.2020377

Article Contents

2021, Volume 14, Issue 4: 1495-1518. Doi: 10.3934/dcdss.2020377

This issue Previous Article A simple digital spiking neural network: Synchronization and spike-train approximation Next Article Observability of switched Boolean control networks using algebraic forms

Optimal synchronization control of multiple euler-lagrange systems via event-triggered reinforcement learning

The Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai 200237, China

^* Corresponding author: Yang Tang
^* Corresponding author: Yang Tang

Received: January 2020

Revised: January 2020

Early access: May 2020

Published: April 2021

Abstract / Introduction Full Text(HTML) Figure(8) / Table(1) Related Papers Cited by

Abstract

In this paper, an event-triggered reinforcement learning-based met-hod is developed for model-based optimal synchronization control of multiple Euler-Lagrange systems (MELSs) under a directed graph. The strategy of event-triggered optimal control is deduced through the establishment of Hamilton-Jacobi-Bellman (HJB) equation and the triggering condition is then proposed. Event-triggered policy iteration (PI) algorithm is then borrowed from reinforcement learning algorithms to find the optimal solution. One neural network is used to represent the value function to find the analytical solution of the event-triggered HJB equation, weights of which are updated aperiodically. It is proved that both the synchronization error and the weight estimation error are uniformly ultimately bounded (UUB). The Zeno behavior is also excluded in this research. Finally, an example of multiple 2-DOF prototype manipulators is shown to validate the effectiveness of our method.

Keywords:

Mathematics Subject Classification: Primary: 93A16, 49L20; Secondary: 35F21.

Citation:

Full Text(HTML)

Figure 1. Communication graph of MELSs

Download: Full-size image PowerPoint slide

Figure 2. Triggering instants for all agents

Download: Full-size image PowerPoint slide

Figure 3. Position trajectories of the first and second component of each EL agent

Download: Full-size image PowerPoint slide

Figure 4. Velocity trajectories of the first and second component of each EL agent

Download: Full-size image PowerPoint slide

Figure 5. Synchronization errors of the first and second component of each EL agent

Download: Full-size image PowerPoint slide

Figure 6. Control policies of the first and second component of each EL agent under event-triggered mechanism

Download: Full-size image PowerPoint slide

Figure 7. Norm of estimated weights of the critic neural network

Download: Full-size image PowerPoint slide

Figure 8. Validation of Assumption 6 for agent 1

Download: Full-size image PowerPoint slide

Table 1. Notations, values and units of the according physical parameters

Notations	Values	Units
$ m_a $	1.2	$ kg $
$ m_b $	1	$ kg $
$ l_{ca} $	0.75	$ m $
$ l_{cb} $	0.75	$ m $
$ l_a $	0.26	$ m $
$ l_b $	0.5	$ m $
$ I_{ca} $	0.125	$ kg\cdot m^2 $
$ I_{cb} $	0.188	$ kg\cdot m^2 $
$ g $	9.81	$ m/s^2 $

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	A. Abdessameud, A. Tayebi and I. G. Polushin, Leader-follower synchronization of Euler-Lagrange systems with time-varying leader trajectory and constrained discrete-time communication, IEEE Trans. Autom. Control, 62 (2017), 2539-2545. doi: 10.1109/TAC.2016.2602326.
[2]	C. Amato, G. Konidaris, A. Anders, G. Cruz, J. P. How and L. P. Kaelbling, Policy search for multi-robot coordination under uncertainty, Int. J. Robot. Res., 35 (2016), 1760-1778. doi: 10.15607/RSS.2015.XI.007.
[3]	M. Bardi and I. Capuzzo-Dolcetta, Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations, Systems & Control: Foundations & Applications. Birkhäuser Boston, Inc., Boston, MA, 1997. doi: 10.1007/978-0-8176-4755-1.
[4]	D. P. Bertsekas, J. N. Tsitsiklis and A. Volgenant, Neuro-Dynamic Programming, Second edition. Athena Scientific Optimization and Computation Series. Athena Scientific, Belmont, MA, 1999.
[5]	D. P. Bertsekas, Dynamic Programming and Optimal Control, Athena scientific, Belmont, MA, 1995.
[6]	G. Chen, Y. Yue and Y. Song, Finite-time cooperative-tracking control for networked Euler-Lagrange systems, IET Control Theory Appl., 7 (2013), 1487-1497. doi: 10.1049/iet-cta.2013.0205.
[7]	S. J. Chung and J. J. E. Slotine, Cooperative robot control and concurrent synchronization of Lagrangian systems, IEEE Trans. Robot., 25 (2009), 686-700.
[8]	D. V. Dimarogonas, E. Frazzoli and K. H. Johansson, Distributed event-triggered control for multi-agent systems, IEEE Trans. Autom. Control, 57 (2012), 1291-1297. doi: 10.1109/TAC.2011.2174666.
[9]	F. Heppner and U. Grenander, A stochastic nonlinear model for coordinated bird flocks, Proc. Ubiquity Chaos, 233 (1990), 238.
[10]	W. Hu, L. Liu and G. Feng, Consensus of multi-agent systems by distributed event-triggered control, Proc. IFAC, 47 (2014), 9768-9773.
[11]	N. Huang, Z. Duan and Y. Zhao, Distributed consensus for multiple Euler-Lagrange systems: An event-triggered approach, Sci. China Technol. Sci., 59 (2016), 33-44. doi: 10.1007/s11431-015-5987-9.
[12]	B. Igelnik and Y. H. Pao, Stochastic choice of basis functions in adaptive function approximation and the functional-link net, IEEE Trans. Neural Netw., 6 (1995), 1320-1329. doi: 10.1109/72.471375.
[13]	X. Jin, D. Wei, W. He, L. Kocarev, Y. Tang and J. Kurths, Twisting-based finite-time consensus for Euler-Lagrange systems with an event-triggered strategy, IEEE Trans. Netw. Sci. Eng., (2019), 1–1. doi: 10.1109/TNSE.2019.2900264.
[14]	Y. Katz, K. Tunstrøm, C. C. Ioannou, C. Huepe and I. D. Couzin, Inferring the structure and dynamics of interactions in schooling fish, Proc. Natl Acad. Sci., 108 (2011), 18720-18725. doi: 10.1073/pnas.1107583108.
[15]	H. K. Khalil, Nonlinear Systems, Upper Saddle River, NJ: Prentice hall, 2002.
[16]	J. R. Klotz, Z. Kan, J. M. Shea, E. L. Pasiliao and W. E. Dixon, Asymptotic synchronization of a leader-follower network of uncertain Euler-Lagrange systems, IEEE Trans. Control Network Syst., 2 (2014), 174-182. doi: 10.1109/TCNS.2014.2378875.
[17]	J. R. Klotz, S. Obuz, Z. Kan and W. E. Dixon, Synchronization of uncertain Euler-Lagrange systems with uncertain time-varying communication delays, IEEE Trans. Cybern., 48 (2018), 807-817. doi: 10.1109/TCYB.2017.2657541.
[18]	F. L. Lewis, D. Vrabie and V. L. Syrmos, Optimal Control, John Wiley & Sons, New Jersey, 2012. doi: 10.1002/9781118122631.
[19]	X. Li, X. Yang and T. Huang, Persistence of delayed cooperative models: Impulsive control method, Appl. Math. Comput., 342 (2019), 130-146. doi: 10.1016/j.amc.2018.09.003.
[20]	J. Li, H. Modares, T. Chai, F. L. Lewis and L. Xie, Off-policy reinforcement learning for synchronization in multiagent graphical games, IEEE Trans. Neural Netw. Learn. Syst., 28 (2017), 2434-2445. doi: 10.1109/TNNLS.2016.2609500.
[21]	A. Loria and H. Nijmeijer, Bounded output feedback tracking control of fully actuated Euler-Lagrange systems, Syst. Control Lett., 33 (1998), 151-161. doi: 10.1016/S0167-6911(97)80170-3.
[22]	J. Mei, W. Ren and G. Ma, Distributed containment control for Lagrangian networks with parametric uncertainties under a directed graph, Automatica, 48 (2012), 653-659. doi: 10.1016/j.automatica.2012.01.020.
[23]	J. J. Murray, C. J. Cox, G. G. Lendaris and R. Saeks, Adaptive dynamic programming, IEEE Trans. Syst. Man Cybern., 32 (2002), 140-153. doi: 10.1109/TSMCC.2002.801727.
[24]	E. Nuno, R. Ortega, L. Basanez and D. Hill, Synchronization of networks of nonidentical Euler-Lagrange systems with uncertain parameters and communication delays, IEEE Trans. Autom. Control, 56 (2011), 935-941. doi: 10.1109/TAC.2010.2103415.
[25]	J. Qin, M. Li, Y. Shi, Q. Ma and W. X. Zheng, Optimal synchronization control of multiagent systems with input saturation via off-policy reinforcement learning, IEEE Trans. Neural Netw. Learn. Syst., 30 (2018), 85-96. doi: 10.1109/TNNLS.2018.2832025.
[26]	Z. Qiu, Y. Hong and L. Xie, Optimal consensus of Euler-Lagrangian systems with kinematic constraints, Proc. IFAC, 49 (2016), 327-332. doi: 10.1016/j.ifacol.2016.10.418.
[27]	J. Sarangapani, Neural Network Control of Nonlinear Discrete-Time Systems, CRC press, Boca Raton, 2006. doi: 10.1201/9781420015454.
[28]	R. S. Sutton and A. G. Barto, Introduction to Reinforcement Learning, Second edition. Adaptive Computation and Machine Learning. MIT Press, Cambridge, MA
[29]	Y. Tang, X. Wu, P. Shi and F. Qian, Input-to-state stability for nonlinear systems with stochastic impulses, Automatica, 113 (2020), 108766, 12pp. doi: 10.1016/j.automatica.2019.108766.
[30]	K. G. Vamvoudakis, F. L. Lewis and G. R. Hudas, Multi-agent differential graphical games: Online adaptive learning solution for synchronization with optimality, Automatica, 48 (2012), 1598-1611. doi: 10.1016/j.automatica.2012.05.074.
[31]	K. G. Vamvoudakis, Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems, IEEE/CAA J. Autom. Sinica, 1 (2014), 282-293.
[32]	X. F. Wang, Z. Deng, S. Ma and X. Du, Event-triggered design for multi-agent optimal consensus of Euler-Lagrangian systems, Kybernetika, 53 (2017), 179-194. doi: 10.14736/kyb-2017-1-0179.
[33]	C. Wei, J. Luo, H. Dai and J. Yuan, Adaptive model-free constrained control of postcapture flexible spacecraft: A Euler–Lagrange approach, J. Vib. Contr., 24 (2018), 4885-4903. doi: 10.1177/1077546317736965.
[34]	S. Weng, D. Yue and J. Shi, Distributed cooperative control for multiple photovoltaic generators in distribution power system under event-triggered mechanism, J. Franklin Inst., 353 (2016), 3407-3427. doi: 10.1016/j.jfranklin.2016.06.015.
[35]	D. Yang, X. Li and J. Qiu, Output tracking control of delayed switched systems via state-dependent switching and dynamic output feedback, Nonlinear Anal. Hybrid Syst., 32 (2019), 294-305. doi: 10.1016/j.nahs.2019.01.006.
[36]	H. Zhang, F. L. Lewis and A. Das, Optimal design for synchronization of cooperative systems: State feedback, observer and output feedback, IEEE Trans. Autom. Control, 56 (2011), 1948-1952. doi: 10.1109/TAC.2011.2139510.
[37]	W. Zhang, Y. Tang, T. Huang and A. V. Vasilakos, Consensus of networked Euler-Lagrange systems under time-varying sampled-data control, IEEE Trans. Ind. Inform., 14 (2018), 535-544. doi: 10.1109/TII.2017.2715843.
[38]	W. Zhang, Q. Han, Y. Tang and Y. Liu, Sampled-data control for a class of linear time-varying systems, Automatica, 103 (2019), 126-134. doi: 10.1016/j.automatica.2019.01.027.
[39]	W. Zhao and H. Zhang, Distributed optimal coordination control for nonlinear multi-agent systems using event-triggered adaptive dynamic programming method, ISA Trans., 91 (2019), 184-195. doi: 10.1016/j.isatra.2019.01.021.