# American Institute of Mathematical Sciences

June  2021, 29(2): 2187-2221. doi: 10.3934/era.2020112

## Pullback attractors for stochastic recurrent neural networks with discrete and distributed delays

 1 School of Mathematics and Statistics, Gansu Key Laboratory of Applied Mathematics and Complex Systems, Lanzhou University, Lanzhou 730000, China 2 Mathematisches Institut, Universität Tübingen, Tübingen 72076, Germany

* Corresponding author: Yejuan Wang

Received  April 2020 Revised  August 2020 Published  October 2020

Fund Project: This work was supported by NSF of China (Grant Nos. 41875084, 11571153), the Fundamental Research Funds for the Central Universities under Grant Nos. lzujbky-2018-it58 and lzujbky-2018-ot03

In this paper, we investigate a class of stochastic recurrent neural networks with discrete and distributed delays for both biological and mathematical interests. We do not assume any Lipschitz condition on the nonlinear term, just a continuity assumption together with growth conditions so that the uniqueness of the Cauchy problem fails to be true. Moreover, the existence of pullback attractors with or without periodicity is presented for the multi-valued noncompact random dynamical system. In particular, a new method for checking the asymptotical compactness of solutions to the class of nonautonomous stochastic lattice systems with infinite delay is used.

Citation: Meiyu Sui, Yejuan Wang, Peter E. Kloeden. Pullback attractors for stochastic recurrent neural networks with discrete and distributed delays. Electronic Research Archive, 2021, 29 (2) : 2187-2221. doi: 10.3934/era.2020112
##### References:
 [1] J.-P. Aubin and H. Franskowska, Set-Valued Analysis, Systems & Control: Foundations & Applications, 2. Birkhäuser Boston, Inc., Boston, MA, 1990.  Google Scholar [2] P. Balasubramaniam and R. Rakkiyappan, Global asymptotic stability of stochastic recurrent neural networks with multiple discrete delays and unbounded distributed delays, Appl. Math. Comput., 204 (2008), 680-686.  doi: 10.1016/j.amc.2008.05.001.  Google Scholar [3] T. Caraballo, M. J. Garrido-Atienza, B. Schmalfuss and J. Valero, Non-autonomous and random attractors for delay random semilinear equations without uniqueness, Discrete Contin. Dyn. Syst., 21 (2008), 415-443.  doi: 10.3934/dcds.2008.21.415.  Google Scholar [4] T. Caraballo, F. Morillas and J. Valero, Pullback attractors of stochastic lattice dynamical systems with a multiplicative noise and non-Lipschitz nonlinearities, Differential and Difference Equations with Applications, Springer Proc. Math. Stat., Springer, New York, 47 (2013), 341-349. doi: 10.1007/978-1-4614-7333-6_27.  Google Scholar [5] T. Caraballo, F. Morillas and J. Valero, On differential equations with delay in Banach spaces and attractors for retarded lattice dynamical systems, Discrete Contin. Dyn. Syst., 34 (2014), 51-77.  doi: 10.3934/dcds.2014.34.51.  Google Scholar [6] D. N. Cheban, Global Attractors of Non-Autonomous Dissipative Dynamical Systems: Interdisciplinary Mathematical Sciences, 1, World Scientific Publishing Co. Pte. Ltd., Hackensack, NJ, 2004. doi: 10.1142/9789812563088.  Google Scholar [7] T. Chen, Global exponential stability of delayed Hopfield neural networks, Neural Netw., 14 (2001), 977-980.   Google Scholar [8] G. Chen, D. Li, L. Shi, O. van Gaans and S. Verduyn Lunel, Stability results for stochastic delayed recurrent neural networks with discrete and distributed delays, J. Differential Equations, 264 (2018), 3864-3898.  doi: 10.1016/j.jde.2017.11.032.  Google Scholar [9] A. Cichocki and R. Unbehauen, Neural Networks for Optimization and Signal Processing, Wiley, Chichester, 1993. Google Scholar [10] H. Crauel and F. Flandoli, Attractors for random dynamical systems, Probab. Theory Related Fields, 100 (1994), 365-393.  doi: 10.1007/BF01193705.  Google Scholar [11] F. Flandoli and B. Schmalfuss, Random attractors for the $3$D stochastic Navier-Stokes equation with multiplicative white noise, Stochast. Stochast. Rep., 59 (1996), 21-45.  doi: 10.1080/17442509608834083.  Google Scholar [12] Y. Guo, Mean square global asymptotic stability of stochastic recurrent neural networks with distributed delays, Appl. Math. Comput., 215 (2009), 791-795.  doi: 10.1016/j.amc.2009.06.002.  Google Scholar [13] X. Han and P. E. Kloeden, Non-autonomous lattice systems with switching effects and delayed recovery, J. Differential Equations, 261 (2016), 2986-3009.  doi: 10.1016/j.jde.2016.05.015.  Google Scholar [14] S. Haykin, Neural Networks: A Comprehensive Foundation, Prentice-Hall, Englewood Cliffs, 1994. Google Scholar [15] J. Hu, S. Zhong and L. Liang, Exponential stability analysis of stochastic delayed cellular neural network, Chaos Solitons Fractals, 27 (2006), 1006-1010.  doi: 10.1016/j.chaos.2005.04.067.  Google Scholar [16] C. Huang, Y. He and H. Wang, Mean square exponential stability of stochastic recurrent neural networks with time-varying delays, Comput. Math. Appl., 56 (2008), 1773-1778.  doi: 10.1016/j.camwa.2008.04.004.  Google Scholar [17] P. E. Kloeden, Pullback attractors of nonautonomous semidynamical systems, Stoch. Dyn., 3 (2003), 101-112.  doi: 10.1142/S0219493703000632.  Google Scholar [18] X. Li and X. Fu, Stability analysis of stochastic functional differential equations with infinite delay and its application to recurrent neural networks, J. Comput. Appl. Math., 234 (2010), 407-417.  doi: 10.1016/j.cam.2009.12.033.  Google Scholar [19] X. Li, F. Li, X. Zhang, C. Yang and W. Gui, Exponential stability analysis for delayed semi-Markovian recurrent neural networks: A homogeneous polynomial approach, IEEE Trans. Neural Netw. Learn. Syst., 29 (2018), 6374-6384.  doi: 10.1109/TNNLS.2018.2830789.  Google Scholar [20] G. Nagamani, S. Ramasamy and P. Balasubramaniam, Robust dissipativity and passivity analysis for discrete-time stochastic neural networks with time-varying delay, Complexity, 21 (2015), 47-58.  doi: 10.1002/cplx.21614.  Google Scholar [21] G. Peng and L. Huang, Exponential stability of hybrid stochastic recurrent neural networks with time-varying delays, Nonlinear Anal. Hybrid Syst., 2 (2008), 1198-1204.  doi: 10.1016/j.nahs.2008.09.012.  Google Scholar [22] T. Roska and L. O. Chua, Cellular neural networks with nonlinear and delay-type template elements and non-uniform grids, Int. J. Circuit Theory Appl., 20 (1992), 469-481.   Google Scholar [23] R. Sakthivel, R. Samidurai and S. M. Anthoni, Asymptotic stability of stochastic delayed recurrent neural networks with impulsive effects, J. Optim. Theory Appl., 147 (2010), 583-596.  doi: 10.1007/s10957-010-9728-8.  Google Scholar [24] B. Schmalfuss, Measure Attractors of the Stochastic Navier-Stokes Equation, Report 258, Universität Bremen, Fachbereiche Mathematik/Informatik, Elektrotechnik/Physik, Forschungsschwerpunkt Dynamische Systeme, Bremen, 1991. Google Scholar [25] Y. Sun and J. Cao, $P$th moment exponential stability of stochastic recurrent neural networks with time-varying delays, Nonlinear Anal. Real World Appl., 8 (2007), 1171-1185.  doi: 10.1016/j.nonrwa.2006.06.009.  Google Scholar [26] M. Syed Ali and M. Marudai, Stochastic stability of discrete-time uncertain recurrent neural networks with Markovian jumping and time-varying delays, Math. Comput. Modelling, 54 (2011), 1979-1988.  doi: 10.1016/j.mcm.2011.05.004.  Google Scholar [27] M. Syed Ali, Stochastic stability of uncertain recurrent neural networks with Markovian jumping parameters, Acta Math. Sci. Ser. B, 35 (2015), 1122-1136.  doi: 10.1016/S0252-9602(15)30044-8.  Google Scholar [28] P. Venetianer and T. Roska, Image compression by delayed CNNs, IEEE Trans. Circuits Syst. I, 45 (1998), 205-215.   Google Scholar [29] C. Vidhya, S. Dharani and P. Balasubramaniam, Global asymptotic stability of stochastic reaction-diffusion recurrent neural networks with Markovian jumping parameters and mixed delays, J. Anal., 27 (2019), 277-292.  doi: 10.1007/s41478-018-0123-4.  Google Scholar [30] L. Wan and Q. Zhou, Almost sure exponential stability of stochastic recurrent neural networks with time-varying delays, Internat. J. Bifur. Chaos Appl. Sci. Engrg., 20 (2010), 539-544.  doi: 10.1142/S0218127410025594.  Google Scholar [31] B. Wang, Sufficient and necessary criteria for existence of pullback attractors for non-compact random dynamical systems, J. Differential Equations, 253 (2012), 1544-1583. doi: 10.1016/j.jde.2012.05.015.  Google Scholar [32] Y. Wang and M. Sui, Finite lattice approximation of infinite lattice systems with delays and non-Lipschitz nonlinearities, Asymptot. Anal., 106 (2018), 169-203.  doi: 10.3233/ASY-171444.  Google Scholar [33] Y. Wang and J. Wang, Pullback attractors for multi-valued non-compact random dynamical systems generated by reaction-diffusion equations on an unbounded domain, J. Differential Equations, 259 (2015), 728-776.  doi: 10.1016/j.jde.2015.02.026.  Google Scholar [34] J. Wang, Y. Wang and D. Zhao, Pullback attractors for multi-valued non-compact random dynamical systems generated by semi-linear degenerate parabolic equations with unbounded delays, Stoch. Dyn., 16 (2016), 1750001, 49 pp. doi: 10.1142/S0219493717500010.  Google Scholar [35] S. Zhu, W. Luo and Y. Shen, Robustness analysis for connection weight matrices of global exponential stability of stochastic delayed recurrent neural networks, Circuits Systems Signal Process, 33 (2014), 2065-2083.  doi: 10.1007/s00034-013-9735-8.  Google Scholar

show all references

##### References:
 [1] J.-P. Aubin and H. Franskowska, Set-Valued Analysis, Systems & Control: Foundations & Applications, 2. Birkhäuser Boston, Inc., Boston, MA, 1990.  Google Scholar [2] P. Balasubramaniam and R. Rakkiyappan, Global asymptotic stability of stochastic recurrent neural networks with multiple discrete delays and unbounded distributed delays, Appl. Math. Comput., 204 (2008), 680-686.  doi: 10.1016/j.amc.2008.05.001.  Google Scholar [3] T. Caraballo, M. J. Garrido-Atienza, B. Schmalfuss and J. Valero, Non-autonomous and random attractors for delay random semilinear equations without uniqueness, Discrete Contin. Dyn. Syst., 21 (2008), 415-443.  doi: 10.3934/dcds.2008.21.415.  Google Scholar [4] T. Caraballo, F. Morillas and J. Valero, Pullback attractors of stochastic lattice dynamical systems with a multiplicative noise and non-Lipschitz nonlinearities, Differential and Difference Equations with Applications, Springer Proc. Math. Stat., Springer, New York, 47 (2013), 341-349. doi: 10.1007/978-1-4614-7333-6_27.  Google Scholar [5] T. Caraballo, F. Morillas and J. Valero, On differential equations with delay in Banach spaces and attractors for retarded lattice dynamical systems, Discrete Contin. Dyn. Syst., 34 (2014), 51-77.  doi: 10.3934/dcds.2014.34.51.  Google Scholar [6] D. N. Cheban, Global Attractors of Non-Autonomous Dissipative Dynamical Systems: Interdisciplinary Mathematical Sciences, 1, World Scientific Publishing Co. Pte. Ltd., Hackensack, NJ, 2004. doi: 10.1142/9789812563088.  Google Scholar [7] T. Chen, Global exponential stability of delayed Hopfield neural networks, Neural Netw., 14 (2001), 977-980.   Google Scholar [8] G. Chen, D. Li, L. Shi, O. van Gaans and S. Verduyn Lunel, Stability results for stochastic delayed recurrent neural networks with discrete and distributed delays, J. Differential Equations, 264 (2018), 3864-3898.  doi: 10.1016/j.jde.2017.11.032.  Google Scholar [9] A. Cichocki and R. Unbehauen, Neural Networks for Optimization and Signal Processing, Wiley, Chichester, 1993. Google Scholar [10] H. Crauel and F. Flandoli, Attractors for random dynamical systems, Probab. Theory Related Fields, 100 (1994), 365-393.  doi: 10.1007/BF01193705.  Google Scholar [11] F. Flandoli and B. Schmalfuss, Random attractors for the $3$D stochastic Navier-Stokes equation with multiplicative white noise, Stochast. Stochast. Rep., 59 (1996), 21-45.  doi: 10.1080/17442509608834083.  Google Scholar [12] Y. Guo, Mean square global asymptotic stability of stochastic recurrent neural networks with distributed delays, Appl. Math. Comput., 215 (2009), 791-795.  doi: 10.1016/j.amc.2009.06.002.  Google Scholar [13] X. Han and P. E. Kloeden, Non-autonomous lattice systems with switching effects and delayed recovery, J. Differential Equations, 261 (2016), 2986-3009.  doi: 10.1016/j.jde.2016.05.015.  Google Scholar [14] S. Haykin, Neural Networks: A Comprehensive Foundation, Prentice-Hall, Englewood Cliffs, 1994. Google Scholar [15] J. Hu, S. Zhong and L. Liang, Exponential stability analysis of stochastic delayed cellular neural network, Chaos Solitons Fractals, 27 (2006), 1006-1010.  doi: 10.1016/j.chaos.2005.04.067.  Google Scholar [16] C. Huang, Y. He and H. Wang, Mean square exponential stability of stochastic recurrent neural networks with time-varying delays, Comput. Math. Appl., 56 (2008), 1773-1778.  doi: 10.1016/j.camwa.2008.04.004.  Google Scholar [17] P. E. Kloeden, Pullback attractors of nonautonomous semidynamical systems, Stoch. Dyn., 3 (2003), 101-112.  doi: 10.1142/S0219493703000632.  Google Scholar [18] X. Li and X. Fu, Stability analysis of stochastic functional differential equations with infinite delay and its application to recurrent neural networks, J. Comput. Appl. Math., 234 (2010), 407-417.  doi: 10.1016/j.cam.2009.12.033.  Google Scholar [19] X. Li, F. Li, X. Zhang, C. Yang and W. Gui, Exponential stability analysis for delayed semi-Markovian recurrent neural networks: A homogeneous polynomial approach, IEEE Trans. Neural Netw. Learn. Syst., 29 (2018), 6374-6384.  doi: 10.1109/TNNLS.2018.2830789.  Google Scholar [20] G. Nagamani, S. Ramasamy and P. Balasubramaniam, Robust dissipativity and passivity analysis for discrete-time stochastic neural networks with time-varying delay, Complexity, 21 (2015), 47-58.  doi: 10.1002/cplx.21614.  Google Scholar [21] G. Peng and L. Huang, Exponential stability of hybrid stochastic recurrent neural networks with time-varying delays, Nonlinear Anal. Hybrid Syst., 2 (2008), 1198-1204.  doi: 10.1016/j.nahs.2008.09.012.  Google Scholar [22] T. Roska and L. O. Chua, Cellular neural networks with nonlinear and delay-type template elements and non-uniform grids, Int. J. Circuit Theory Appl., 20 (1992), 469-481.   Google Scholar [23] R. Sakthivel, R. Samidurai and S. M. Anthoni, Asymptotic stability of stochastic delayed recurrent neural networks with impulsive effects, J. Optim. Theory Appl., 147 (2010), 583-596.  doi: 10.1007/s10957-010-9728-8.  Google Scholar [24] B. Schmalfuss, Measure Attractors of the Stochastic Navier-Stokes Equation, Report 258, Universität Bremen, Fachbereiche Mathematik/Informatik, Elektrotechnik/Physik, Forschungsschwerpunkt Dynamische Systeme, Bremen, 1991. Google Scholar [25] Y. Sun and J. Cao, $P$th moment exponential stability of stochastic recurrent neural networks with time-varying delays, Nonlinear Anal. Real World Appl., 8 (2007), 1171-1185.  doi: 10.1016/j.nonrwa.2006.06.009.  Google Scholar [26] M. Syed Ali and M. Marudai, Stochastic stability of discrete-time uncertain recurrent neural networks with Markovian jumping and time-varying delays, Math. Comput. Modelling, 54 (2011), 1979-1988.  doi: 10.1016/j.mcm.2011.05.004.  Google Scholar [27] M. Syed Ali, Stochastic stability of uncertain recurrent neural networks with Markovian jumping parameters, Acta Math. Sci. Ser. B, 35 (2015), 1122-1136.  doi: 10.1016/S0252-9602(15)30044-8.  Google Scholar [28] P. Venetianer and T. Roska, Image compression by delayed CNNs, IEEE Trans. Circuits Syst. I, 45 (1998), 205-215.   Google Scholar [29] C. Vidhya, S. Dharani and P. Balasubramaniam, Global asymptotic stability of stochastic reaction-diffusion recurrent neural networks with Markovian jumping parameters and mixed delays, J. Anal., 27 (2019), 277-292.  doi: 10.1007/s41478-018-0123-4.  Google Scholar [30] L. Wan and Q. Zhou, Almost sure exponential stability of stochastic recurrent neural networks with time-varying delays, Internat. J. Bifur. Chaos Appl. Sci. Engrg., 20 (2010), 539-544.  doi: 10.1142/S0218127410025594.  Google Scholar [31] B. Wang, Sufficient and necessary criteria for existence of pullback attractors for non-compact random dynamical systems, J. Differential Equations, 253 (2012), 1544-1583. doi: 10.1016/j.jde.2012.05.015.  Google Scholar [32] Y. Wang and M. Sui, Finite lattice approximation of infinite lattice systems with delays and non-Lipschitz nonlinearities, Asymptot. Anal., 106 (2018), 169-203.  doi: 10.3233/ASY-171444.  Google Scholar [33] Y. Wang and J. Wang, Pullback attractors for multi-valued non-compact random dynamical systems generated by reaction-diffusion equations on an unbounded domain, J. Differential Equations, 259 (2015), 728-776.  doi: 10.1016/j.jde.2015.02.026.  Google Scholar [34] J. Wang, Y. Wang and D. Zhao, Pullback attractors for multi-valued non-compact random dynamical systems generated by semi-linear degenerate parabolic equations with unbounded delays, Stoch. Dyn., 16 (2016), 1750001, 49 pp. doi: 10.1142/S0219493717500010.  Google Scholar [35] S. Zhu, W. Luo and Y. Shen, Robustness analysis for connection weight matrices of global exponential stability of stochastic delayed recurrent neural networks, Circuits Systems Signal Process, 33 (2014), 2065-2083.  doi: 10.1007/s00034-013-9735-8.  Google Scholar
 [1] John T. Betts, Stephen Campbell, Claire Digirolamo. Examination of solving optimal control problems with delays using GPOPS-Ⅱ. Numerical Algebra, Control & Optimization, 2021, 11 (2) : 283-305. doi: 10.3934/naco.2020026 [2] Akio Matsumot, Ferenc Szidarovszky. Stability switching and its directions in cournot duopoly game with three delays. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021069 [3] Dmitry Treschev. A locally integrable multi-dimensional billiard system. Discrete & Continuous Dynamical Systems - A, 2017, 37 (10) : 5271-5284. doi: 10.3934/dcds.2017228 [4] Zengyun Wang, Jinde Cao, Zuowei Cai, Lihong Huang. Finite-time stability of impulsive differential inclusion: Applications to discontinuous impulsive neural networks. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2677-2692. doi: 10.3934/dcdsb.2020200 [5] Xu Zhang, Xiang Li. Modeling and identification of dynamical system with Genetic Regulation in batch fermentation of glycerol. Numerical Algebra, Control & Optimization, 2015, 5 (4) : 393-403. doi: 10.3934/naco.2015.5.393 [6] Carlos Fresneda-Portillo, Sergey E. Mikhailov. Analysis of Boundary-Domain Integral Equations to the mixed BVP for a compressible stokes system with variable viscosity. Communications on Pure & Applied Analysis, 2019, 18 (6) : 3059-3088. doi: 10.3934/cpaa.2019137 [7] Juan Manuel Pastor, Javier García-Algarra, José M. Iriondo, José J. Ramasco, Javier Galeano. Dragging in mutualistic networks. Networks & Heterogeneous Media, 2015, 10 (1) : 37-52. doi: 10.3934/nhm.2015.10.37 [8] Gheorghe Craciun, Jiaxin Jin, Polly Y. Yu. Single-target networks. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021065 [9] Livia Betz, Irwin Yousept. Optimal control of elliptic variational inequalities with bounded and unbounded operators. Mathematical Control & Related Fields, 2021  doi: 10.3934/mcrf.2021009 [10] Rui Hu, Yuan Yuan. Stability, bifurcation analysis in a neural network model with delay and diffusion. Conference Publications, 2009, 2009 (Special) : 367-376. doi: 10.3934/proc.2009.2009.367 [11] Todd Hurst, Volker Rehbock. Optimizing micro-algae production in a raceway pond with variable depth. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021027 [12] Khosro Sayevand, Valeyollah Moradi. A robust computational framework for analyzing fractional dynamical systems. Discrete & Continuous Dynamical Systems - S, 2021  doi: 10.3934/dcdss.2021022 [13] Alessandro Gondolo, Fernando Guevara Vasquez. Characterization and synthesis of Rayleigh damped elastodynamic networks. Networks & Heterogeneous Media, 2014, 9 (2) : 299-314. doi: 10.3934/nhm.2014.9.299 [14] Juan Manuel Pastor, Javier García-Algarra, Javier Galeano, José María Iriondo, José J. Ramasco. A simple and bounded model of population dynamics for mutualistic networks. Networks & Heterogeneous Media, 2015, 10 (1) : 53-70. doi: 10.3934/nhm.2015.10.53 [15] Hirofumi Notsu, Masato Kimura. Symmetry and positive definiteness of the tensor-valued spring constant derived from P1-FEM for the equations of linear elasticity. Networks & Heterogeneous Media, 2014, 9 (4) : 617-634. doi: 10.3934/nhm.2014.9.617 [16] Samir Adly, Oanh Chau, Mohamed Rochdi. Solvability of a class of thermal dynamical contact problems with subdifferential conditions. Numerical Algebra, Control & Optimization, 2012, 2 (1) : 91-104. doi: 10.3934/naco.2012.2.91 [17] Xinyuan Liao, Caidi Zhao, Shengfan Zhou. Compact uniform attractors for dissipative non-autonomous lattice dynamical systems. Communications on Pure & Applied Analysis, 2007, 6 (4) : 1087-1111. doi: 10.3934/cpaa.2007.6.1087 [18] Hong Seng Sim, Wah June Leong, Chuei Yee Chen, Siti Nur Iqmal Ibrahim. Multi-step spectral gradient methods with modified weak secant relation for large scale unconstrained optimization. Numerical Algebra, Control & Optimization, 2018, 8 (3) : 377-387. doi: 10.3934/naco.2018024 [19] Namsu Ahn, Soochan Kim. Optimal and heuristic algorithms for the multi-objective vehicle routing problem with drones for military surveillance operations. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021037 [20] Dayalal Suthar, Sunil Dutt Purohit, Haile Habenom, Jagdev Singh. Class of integrals and applications of fractional kinetic equation with the generalized multi-index Bessel function. Discrete & Continuous Dynamical Systems - S, 2021  doi: 10.3934/dcdss.2021019

Impact Factor: 0.263