LONG TERM BEHAVIOR OF A RANDOM HOPFIELD NEURAL LATTICE MODEL

. A Hopﬁeld neural lattice model is developed as the inﬁnite dimensional extension of the classical ﬁnite dimensional Hopﬁeld model. In ad- dition, random external inputs are considered to incorporate environmental noise. The resulting random lattice dynamical system is ﬁrst formulated as a random ordinary diﬀerential equation on the space of square summable bi-inﬁnite sequences. Then the existence and uniqueness of solutions, as well as long term dynamics of solutions are investigated.


1.
Introduction. Neural networks have been gaining increasing attention of researchers, due to their wide range of applications such as character recognition (see, e.g., [17]), image compression (see, e.g., [9]), stock market prediction (see, e.g., [13,16]), traveling saleman's problem (see, e.g., [12]), etc. In particular, one of the most popular mathematical models for (artificial) neural network is the Hopfield neural model proposed by John Hopfield in 1984 [15], described by the following system of n ordinary differential equations (ODEs): λ i,j g j (u j (t)) + I i , i = 1, ..., n, where u i represents the voltage on the input of the ith neuron at time t; µ i > 0 and γ i > 0 represents the neuron amplifier input capacitance and resistance of the ith neuron, respectively; and I i is the constant external forcing on the ith neuron.
Here n is the total number of neurons coupled by an n × n matrix (λ i,j ) 1≤i,j≤n , where the entity λ i,j represents the connection strength between the ith and the jth the neuron. More precisely, for each pair of i, j = 1, . . . , n, λ i,j is the synapse efficacy between neurons i and j, and thus λ i,j > 0 (λ i,j < 0, resp.) means the output of neuron j excites (inhibits, resp.) neuron i. The term λ i,j g j (u j (t)) represents the electric current input to neuron i due to the present potential of neuron j, in which the function g j is neuron activation functions and assumed to be a sigmoid type function.
We are interested in studying dynamics of the above Hopfiled neural network model when its size becomes increasingly large, i.e., n → ∞. To this end, we extend the n dimensional ODE system (1) to an infinite dimensional lattice system, that models the dynamics of an infinite number of neurons indexed by i ∈ Z, in which each neuron is still connected with other neurons within its finite N neighborhood. More precisely, the ith neuron is connected to the (i − N )th, · · · , (i + N )th neurons through the strength matrix (λ i,j ) i−N ≤j≤i+N and the activation functions g j for j = i − N, · · · , i + N . In addition, to take into account random perturbations of the environment, we introduce a noise in the equations in (1) by replacing each constant input I i by a random forcing I i (θ t ω) represented by a measure-preserving dynamical system {θ t } t∈R acting on a probability space (Ω, F, P). System (1) then becomes the following random lattice dynamical system, namely the random Hopfield neural lattice model: Over the past two decades, extensive studies have been done on dynamics of lattice dynamical systems (see, e.g., [3,6,5,19,20,21,22,23] and references therein). However, most of the existing works consider a simple linear diffusion described by a linear operator such as u i−1 − 2u i + 2u i+1 . Very often the lattice dynamical system under consideration arises from discretization of a partial differential equation. Notice that the terms i+q j=i−q λ i,j g j (u j (t)) in system (2) model a nonlinear finite neighborhood connection structure, which is intrinsically discrete and does not arise from a continuous operator. Such models have not been studied in the literature of lattice dynamical systems.
The goal of this work is to investigate the long term dynamics of the random lattice dynamical system (2), in particular, the existence of random attractors. The main tool is the theory of random dynamical systems and random attractors (see, e.g., [1,14,8,7,18]). The paper is organized as follows. In Section 2 we provide necessary preliminaries on random dynamical systems and random attractors. In Section 3 we first reformulate system (2) as a random ordinary differential equation (RODE) on the space of bi-infinite sequences and then show that the resulting RODE has a unique solution that generates a random dynamical system (RDS). In Section 4 we investigate the existence of a random attractor for the RDS obtained in Section 3. Some closing remarks will be given in Section 5.
Definition 2.2. A set-valued mapping K : Ω → 2 X \∅ is said to be a random set if the mapping ω → dist X (x, K(ω)) is measurable for any x ∈ X. A random set K(ω) ⊂ X is said to be tempered with respect to Θ if for a.e. ω ∈ Ω lim t→∞ e −βt sup x∈K(θ−tω) x X = 0, ∀ β > 0.
A random variable ω → r(ω) ∈ R is said to be tempered if for a.e. ω ∈ Ω, Throughout this paper, denote by D(X) the set of all tempered random sets of X.
The existence of random attractors usually requires the existence of random absorbing set defined below.
Definition 2.4. A random set Γ(ω) ⊂ X is called a random absorbing set in D(X) if for any K ∈ D(X) and a.e ω ∈ Ω, there is T K (ω) > 0 such that The following theorem is one of the most widely used results for the existence of random attractors.
Proposition 1 ( [2,4,11]). Let Γ ∈ D(X) be a closed absorbing set for the continuous RDS {ϕ(t, ω)} t≥0,ω∈Ω and satisfies the asymptotic compactness condition for a.e. ω ∈ Ω, i.e., each sequence x n ∈ ϕ(t n , θ −tn , Γ(θ −tn ω)) has a convergent subsequence in X when t n → ∞. Then the RDS ϕ has a unique global random attractor with component subsets If the pullback absorbing set is positively invariant, i.e., ϕ(t, ω, Γ(ω)) ⊂ Γ(θ t ω)) for all t ≥ 0, then 3. Basic properties of solutions. In this section we study the existence and uniqueness of solutions to the lattice system (2), and show that the solutions generate a random dynamical system. To this end, we consider the separable Hilbert space of square summable bi-infinite sequences: For any u = (u i ) i∈Z ∈ 2 , define the operators Au = ((Au) i ) i∈Z and Λu = ((Λu) i ) i∈Z by Throughout this paper it is assumed that (A1) the efficacy among neurons is finite, i.e., there exists B λ > 0 such that In addition, it is assumed that , g i (0) = 0, and there exists a continuous nondecreasing function L(r) ∈ C(R + , R + ) such that It follows immediately from the assumption (A2) that Au ∈ 2 for every u ∈ 2 . In addition, notice that the assumption (A3) implies that given any Then by the assumptions (A1) and (A3) we have , and thus Λu ∈ 2 for every u ∈ 2 .
At last, define I(θ t ω) = Ii(θtω) µi i∈Z and assume that Then I(θ t ω) ∈ C 1 (R, 2 ) and the lattice system (2) can be rewritten as the random ordinary differential equation (RODE): Theorem 3.2 below states the existence and uniqueness of a global solution to the RODE (4). The proof needs the following general Gronwall type Lemma.
Lemma 3.1. Let a(t) be non-negative and non-decreasing, b(t) be non-negative and continuous, and c(t) be non-negative and integrable on [t 0 , T ], and assume that the function w : Proof. Since all functions and integrals are nonnegative, we have With w(t) ≤ 2a(t) + 2 t t0 b(s)w(s)ds and the assumption that a(t) is nondecreasing it follows directly from Gronwall's inequality that With w(t) ≤ 2 t t0 c(s) ln(w(s) + 1)w(s)ds and the assumption that c(t) is nonnegative we have Then by using a generalized Gronwall-like integral inequality [10], we obtain Inserting the inequalities (6) and (7) into (5) gives immediately the desired assertion. The proof is complete.
We next show that given any T > t 0 fixed the solution u(t) is bounded for all t ∈ [t 0 , T ]. To this end, multiplying both sides of the equation (2) by u i (t) and sum over all i ∈ Z to give First by Assumption (A2), we have Then by using the fact that xy ≤ 1 2 (ax 2 + 1 a y 2 ), for some a > 0 (to be determined later), it holds Then by assumptions (A1) and (A2), for some ξ j with |ξ j | ≤ |u j |. Thus by Assumption (A2), (A3) and the fact that Inserting estimations (11) -(13) into (10) we obtain Pick a > 0 such that − 1 BµBγ + a 2 < 0. In particular, let a = 1 BµBγ , and for simplicity denote Then the above inequality becomes Multiplying both sides of (14) by e B1t gives d e B1t u(t) Integrating the above inequality from t 0 to t results in and hence First notice that the assumption (A4) implies that t t0 M (θ s ω)e −B1(t−s) ds < ∞ for all t ≥ t 0 . Now using the assumption that L(r) ≤ κ 1 ln(r 2 + 1) + κ 2 in (15) we obtain Since M (θ s ω) ≥ 0 for all s ∈ R, t t0 M (θ s ω)e −B1(t−s) ds is non-negative and nondecreasing. It then follows directly from Lemma 3.1 that for all t ∈ [t 0 , T ], wherê We have just shown that the solution exists for all t ≥ t 0 . It remains to show the continuous dependence of solutions on initial data. To this end, let u o , v o ∈ 2 and consider two solutions of system (4) with initial value u(t 0 ) = u 0 and u(t 0 ) = v o , respectively, and write Multiplying the equation (18) by h i (t) and summing over i ∈ Z gives By Assumption (A3) again, we have where L T is a constant depends on T according to (16). Therefore, Integrating the above inequality gives which implies that sup t∈[t0,T ] The solution depends continuously on the initial data. The proof is now complete.
It is straightforward to check that and this allows us to define a continuous random dynamical system ϕ(t, ω, ·) by From now on, we will write u(t; ω, u o ) instead of u(t; 0, ω, u o ).
4. Existence of random attractors. In this section we investigate the existence of attractors for the random dynamical system ϕ(t, ω) defined by the solutions to the RODE (4). To this end, we first construct a closed and bounded absorbing set for ϕ(t, ω) and then prove the asymptotic compactness of the absorbing set. For simplification of exposition, throughout this section we assume that that L(r) ≡ L > 0 in Assumption (A3). In addition, assume that the functions g i satisfy the dissipative condition (A5) there exists α ≥ 0 and β = (β i ) i∈Z ∈ 2 such that Proof. We will start from the equation (10) and still use the inequalities (11) and (12). It then remains to estimate i∈Z 1 µi i+N j=i−N λ i,j u i g j (u j ). To this end first note that First by Assumption (A5), Then by Assumption (A3) with L(r) ≡ L and the fact that xy ≤ 1 2 x 2 a + ay 2 we get for some a > 0 (to be determined later). Therefore Pick a such that a 2 L − α < 0 in (22). In particular, let a = α L . Then multiplying (22) by λ i,j > 0 and summing over all j with λ i,j > 0 we have Using (21), (23) and (24) we obtain Multiplying both sides of (25) by 1 µi and summing over i ∈ Z we obtain It then follows from the assumption (19) and a shift of index that Now inserting the inequality (11), the inequality (12) with a = 1 BµBγ , and the inequality (26) into (10) results in where σ is defined as in (20) and M (θ t ω) = Then replacing ω by θ −t ω in the above inequality we finally obtain Note that 0 −t M (θ τ ω)e στ dτ is a tempered random variable due to Assuption (A4) and the integrability of e στ for τ < 0. Define (29) Then for any K(ω) ∈ D( 2 ) and given u o (ω) ∈ K(ω), there exists T K (ω) > 0 such that for all t ≥ T K (ω), i.e., ϕ(t, θ −t ω, K(θ −t ω)) ∈ Γ(ω). The proof is complete.
Remark 1. The dissipativity condition (A5) on g i 's is not necessary. In fact, even if α < 0, an absorbing set can still exist provided 1 BµBγ is large enough compare to L. But with g i 's being dissipative, the convergence rate of solutions toward the absorbing set is faster.
Following the techniques first introduced in [3], we next show that the absorbing set defined in (29) is asymptotically compact under the RDS ϕ(t, ω). This will be done by a tail estimate of solutions, presented in the lemma below. Note that as a particular case of Lemma 4.1, there exists T Γ (ω) such that Proof. Given k ∈ N fixed (to be determine later), let ρ k : R + → [0, 1] be a smooth, increasing and sub-additive function such that For any solution u(t; ω, Taking the inner product of v with (4) we get First by Assumption (A2), Second, similar to (12) there exists some a > 0 such that It remains to estimate the last term of (30). To this end, multiplying (25) by 1 µi ρ k (|i|) and summing over i ∈ Z to get i∈Z ρ k (|i|) Choose k > N . Notice that since for any j = i − N, · · · , i + N , |i − j| ≤ N , thus ρ k (|i − j|) = 0. Then using the increasingness and sub-additivity of ρ k we obtain For the first term of (34) we also need Apply the two above relations in the inequality (34) results in Then it follows from a shift of index that Inserting estimations (31), (32) with a = 1 BµBγ and (35) in (30) we get where σ is as defined in (20). Integrating the above inequality from 0 to t then replacing ω by θ −t ω gives Then for any > 0, there existsT ( , ω) > 0 such that Second, since β ∈ 2 , given any > 0 there exists N 1 ( ) > 0 such that .
Hence p n (the subsequence) have strongly converge in 2 , and therefore Γ(ω) is asymptotically compact. The proof is complete.
The following theorem follows directly from Lemma 4.1, Lemma 4.3 and Proposition 1. where Γ is defined as in (29). 5. Closing remarks. This work was motivated by the increasing size of a Hopfield neural network under the influence of random forcing. In the context of Hopfield models, there are two novelties in this work: (1) the underlying system is an infinite dimensional system of ODEs, i.e., lattice dynamical system; and (2) the forcing at each neuron is general random process. In the context of lattice dynamical systems, the major novelty as well as difficulty is the consideration of a nonlinear interconnection structure described by the nonlinear operator i+N j=i−N λ i,j g j (u j (t)). Such systems have never been studied in the past. The main contribution of this work is showing the existence of a global random attractor for the random Hopfield lattice system. Along with the construction of the random absorbing set, another interesting discovery is that inhibit relation among neurons contribute more to the convergence of solutions towards the attractor. These results serve as an important initial step towards the study on more interesting dynamical behavior of neural networks, such as upper semi continuity of the attractors and synchronization behavior of discrete neural networks under random perturbation.