Razumikhin-type theorems on moment exponential stability of functional differential equations involving two-time-scale Markovian switching

This work develops moment exponential stability of functional differential equations (FDEs) with Markovian switching, in which a two-time-scale (real time $t$ and fast time $t/\epsilon$ with a small parameter $\epsilon>0$) continuous-time and finite-state Markov chain 
 is used to represent the switching process. The essence is that there is a time-scale separation, which is motivated by the consideration of networked control systems and manufacturing systems. Under suitable conditions, we establish a Razumikhin-type theorem on the $p$th moment exponential $\epsilon$-stability for the small parameter $\epsilon$. By virtue of the Razumikhin-type theorem, we further deduce mean-square exponential stability results for delay differential equations (DDEs) and ordinary differential equations (ODEs) with two-time-scale Markovian switching. These stability results show that the overall system may be stabilized by the Markov switching even when some of the underlying subsystems are unstable. It is noted that in the presence of the Markovian switching, the stationary distribution of the fast changing part of the Markov chain plays an important role. Explicit conditions for the mean-square exponential stability of linear equations are derived and illustrative examples are provided to demonstrate our results.


1.
Introduction. Recently, Markovian switching systems driven by continuoustime Markovian chains have drawn growing attention in biological sciences [1,22,35], financial engineering [3, 5-7, 31, 39], communications and manufacturing [32,38], among others. Such systems have been used to model many practical scenarios in which abrupt changes are experienced in the structure and parameters caused by phenomena such as component failures, communication link interruption, packet loss, and power line contingency.

FUKE WU, GEORGE YIN AND LE YI WANG
In many applications, stability is one of the major concerns and a great challenge. The stochastic stability analysis was initiated more than fifty years ago [10,12,18,19] and has been substantially developed ever since. Comprehensive stability analysis, methodology exploration, algorithm development, and numerical solutions for random systems have emerged more recently [13,25,36,37]. Because time delays are often unavoidable in practical systems, significant efforts have also been devoted to delay differential systems and more generally functional differential systems, which are exemplified by chemical processes, biological systems, and communication systems, see for example [4,11,17,28]). It is well observed that delays have detrimental impact on stability and performance. Their presence adds substantial difficulties in stability analysis, mainly due to the fact that these systems are described by delay differential equations (DDEs) or functional differential equations (FDEs); see [9,15,16]. When Markovian switching systems with time delays are considered, the stability analysis becomes even more challenging due to the coexistence and interactions of time delays and abrupt changes in system structures and parameters.
Treating systems with delays, Razumikhin methods have been widely used for analyzing stability; see Haddock et al. [8] and Hale [9]. In essence, a Razumikhintype theorem states that if the derivative of a Liapunov function along all system trajectories is negative whenever the present value of the function dominates that of the past over the delay interval, then the Liapunov function along the system trajectories will converge to zero. In recent years, this theorem has been extensively applied to analyze input-to-state stability (ISS) and input-to-output stability (IOS) of time-delay systems and functional differential systems, as well as to design feedback stabilizers [11,23,28]. In a series of papers, Mao and his colleagues established the Razumikhin-type theorem on moment exponential stability for stochastic DDEs and stochastic FDEs with Markovian switching and Brownian motion [25][26][27].
When the Markovian switching evolves in a fast pace, certain "averaging" effect takes place [2,14,30,31,40]. Using such an averaging idea, Wu, Yin, and Wang [29] established the Razumikhin-type theorem on the moment exponential stability of systems with random delays modeled by a two-time-scale Markov Chain in which the driving Markov chain has both fast and slow motions and involves strong and weak interactions. It was shown that the "averaged" delay with respect to the stationary distribution of the fast motion plays an important role in asymptotic stability. In particular, the overall system may be stabilized by the Markov switching, even if some of the underlying subsystems are unstable. In [29], the Markovian switching systems with random delays were treated, in which a Markov chain is used to model the random delay. This paper substantially extends our previous results to general switching functional systems in which the Markov chain is embedded in a nonlinear and more general structure. To illustrate, consider a switching FDE of the forṁ where r(t) is a continuous-time Markov chain with state space S = {1, 2, . . . , m}, and f : C([−τ, 0]; R n ) × R + × S → R n is Borel measurable. This paper establishes a Razumikhin-type theorem of moment exponential stability for this class of switching systems. The main distinctive features from the existing literature (cf. [25][26][27]) is that the fast-switching motion of the Markov chain yields an "averaging" effect with respect to its stationary measure and potentially produces a stabilizing factor. The investigation on stability of stochastic delay equations goes back to Kushner [18]. Systems perturbed or driven by wide-band noises, possibly non-Markovian, were treated extensively in [20]; and later studied comprehensively for their related control and filtering problems [21]. This paper focuses on stability of functional differential diffusion systems under two-time-scale Markovian switching. Based on this formulation and employing certain technical results on two-time-scale Markov chains in [33], we obtain new Razumikhin-type stability results that characterize how the Markovian switching affects overall stability. Our findings can be used to guide subsystem design and network topology management to ensure robust stability under random network interruptions, failures, and structural changes.
The rest of the paper is arranged as follows. In the next section, we introduce notation and main assumptions. Section 3 establishes the Razumikhin-type theorem of the pth moment exponential ε-stability on FDEs with Markovian switching (1.1). A mean-square exponential ε-stability criterion is presented among others. Applying the Razumikhin-type theorem, Section 4 establishes criteria for the pth moment exponential stability of DDEs with two-time-scale Markovian switching. In Section 5, the moment exponential stability of ODEs with two-time-scale Markovian switching is derived using the Razumikhin-type theorem. Finally, Section 6 presents illustrative examples to demonstrate our results together with some additional remarks.
2. FDEs with Markovian switching: A two-time-scale formulation. Unless otherwise specified, we use the following notation throughout the paper. Let | · | be the Euclidean norm in R n . If A is a vector or matrix, its transpose is denoted by A . If A is a matrix, denote its trace norm by |A| = tr(A A). Let R + = [0, ∞). Denote by C([−τ, 0]; R n ) the family of continuous functions from [−τ, 0] to R n , which forms a Banach space with the norm ξ = sup −τ ≤θ≤0 |ξ(θ)|. λ min (A) and λ max (A) denote the smallest and largest eigenvalues of A, respectively. If a, b ∈ R, let a ∧ b = min{a, b} and a ∨ b = max{a, b}. Define a + = a ∨ 0 and a − = a ∧ 0. Let M denote all probability measures on [−τ, 0]. In this paper, these probability measures on M can be replaced by any right-continuous nondecreasing functions on [−τ, 0]. Let I A represent the indicator function of the set A.
Let (Ω, F, P) be a complete probability space with a filtration {F t } t≥0 satisfying the usual conditions, that is, it is right continuous and increasing while F 0 contains all P-null sets. Denoted by Let r(t), t ≥ 0 be an continuous-time Markov chain defined on the probability space (Ω, F, P) taking values in a finite state space S = {1, 2, . . . , m} with generator Q = ( q ij ) ∈ R m×m . Recall that the generator Q is weakly irreducible [33], if the system of equations has a unique solution satisfying ν i ≥ 0 for each i = 1, . . . , m. The solution ν = ( ν 1 , ν 2 , . . . , ν m ) is termed quasi-stationary distribution. In what follows, we use O(y) to denote the function of y satisfying sup y |O(y)|/|y| < ∞.
In our setup, the Markov chain r(t) has both fast and slow motions with strong and weak interactions. To reflect the fast and slow motions of the Markov chain, we introduce a small parameter ε > 0 and rewrite the Markov chain r(t) as r ε (t) and the generator Q as Q ε . Thus, the Markov chain displays two-time scales (a usual running time t and a stretched (fast) time t/ε). In the analysis to follow, we choose ε > 0 to be simply a small constant. The essence is that there is a time-scale separation motivated by the consideration of networked control systems. Suppose that the generator of the Markov chain is given by where both Q and Q 0 are generators of suitable continuous-time Markov chains. Throughout the paper, we assume that Q is weakly irreducible in the sense defined in (2.1). Denote the quasi-stationary distribution associated with Q by ν = (ν 1 , . . . , ν m ). As demonstrated in [33], using asymptotic expansions, one can show that the asymptotic properties of the probability vector p ε (t) = (P(r ε (t) = 1), . . . , P(r ε (t) = m)) and the transition probability matrix P ε (t) = (p ε ij (t)) = (P(r ε (t + s) = j|r ε (s) = i)) ∈ R m×m is dominated by Q/ε. Applying the asymptotic extensions of the probability vector p ε (t) (see [33]) yields where κ > 0 determined by the nonzero eigenvalues of Q.
We are interested in the asymptotic stability of the associated switching FDEs (1.1). To highlight the effect of the time-scale separation, let us rewrite system (1.1) asẋ ε (t) = f (x ε t , t, r ε (t)). (2.4) In this paper, assume that the initial data ξ ∈ C b F0 ([−τ, 0]; R n ) satisfying the Lipschitz condition, namely, there exists constant K such that |ξ(θ 1 )−ξ(θ 2 )| ≤ K|θ 1 −θ 2 | for all θ 1 , θ 2 ∈ [−τ, 0]. This system may be seen as a switching system among the following m functional differential subsystemṡ with the random switching governed by the Markov chain r ε (t). The switching functional differential system (2.4) may therefore be rewritten aṡ Let C 1,1 (R n × R + × S; R + ) denote the family of all nonnegative functions V on R n × R + × S which are continuously differentiable in x ∈ R n and t ∈ R + . For a given V ∈ C 1,1 (R n × R + × S; R + ), define an operator Remark 2.1. Let us emphasize that V is a function defined on R n × R + × S, but the operator L is a functional defined on C([−τ, 0]; R n ) × R + × S. Note that ϕ ∈ C([−τ, 0]; R n ) is from the coefficient of f , not from V . Since ϕ(0) is only one point of ϕ, for notational convenience, when L is applied to V , we write L(ϕ, t, i)V (ϕ(0), t, i) as LV (ϕ, t, i) in what follows. Thus, LV is only a notation; V on the other hand depends only on (ϕ(0), t, i).
Let x ε (t) be the solution of (2.4). According to the chain rule for differentiation, Let us impose the linear growth condition and the local Lipschitz condition on f : (2.9)

Assumption 2.3 (The local Lipschitz condition).
For each j = 1, 2, · · · , there exist constant K ij dependent on j and a probability measure µ ∈ M such that for every i ∈ S, According to the classical result (for example [9,27]), Assumptions 2.2 and 2.3 can guarantee that there exists a unique solution for switching FDE (2.4). Assumption 2.2 implies that f (0, t, i) = 0 for all i ∈ S. This implies that the switching FDE (2.4) admits a trivial solution (or zero solution). Let us now present the definition of the moment exponential ε-stability.
Definition 2.4. The trivial solution of the switching FDE (2.4), or simply (2.4) is said to be pth moment exponentially ε-stable if there exist positive constants C, γ and β such that for sufficiently small ε and any t ≥ 0, 3. Moment exponential stability. By the Razumkihin-type technique, this section examines the pth moment exponential stability for switching FDE with the two-time-scale Markovian switching (2.4).
Remark 3.2. In [26] or [27], the stochastic Razumikhin-type theorem required that condition (i) be satisfied and condition (iii) be replaced by Comparing with (iii ), condition (iii) shows that the asymptotic properties of the Markov chain, namely its stationary distribution, plays an important role to determine stability of the equation, which shows the "average" idea. In fact, (3.6) implies that for all i ∈ S, 3) only requires that the above inequality holds in the sense of "average" with respect to the stationary distribution of the driving Markov chain. This implies that even if some subsystems do not satisfy Razumikhin Theorem, the switching system may be stable, namely, the Markov chain becomes a stabilizing factor.

Remark 3.3.
Comparing with the result in [9] or [27], Theorem 3.1 requires that the additional condition (ii) is satisfied.
Proof. We rewrite (2.4) as which, together with the linear growth condition, implies that Applying the Gronwall inequality yields Choosing t 1 = T gives the desired assertion.
Let x ε (t) be the solution of the switching functional system (2.4). For i ∈ S and any t ∈ [0, T ], define a sequence of weighted occupation measures Z ε i (t) as follows: Let us establish the following lemma on convergence of Z ε i (t): Lemma 3.5. Let Assumptions 2.2 and 2.3 hold and condition (ii) in Theorem 3.1 be satisfied. For any ζ ∈ (0, 1) sufficiently small, This proof can be found in Appendix A.
Similar to Lemma 3.5, we can establish the following lemma on convergence of X ε i (t): Lemma 3.6. Let Assumptions 2.2 and 2.3 hold. Then there exists ζ ∈ (0, 1) such that This proof is similar to Lemma 3.5, so it is omitted.

FUKE WU, GEORGE YIN AND LE YI WANG
Proof of Theorem 3.1. If we can prove that for sufficiently small ε, Then for sufficiently small ε, we claim (3.11) holds for all t ≥ 0. Otherwise, for any given ε > 0 sufficiently small, by the continuity of W ε (t), there exists the smallest By the existence and uniqueness of the solution, x ε (t) = 0 a.s. for all t ≥ 0. This contradicts the definition of ρ ε . Therefore, we have which is equivalent to for any θ ∈ [−τ, 0]. Consequently, for any θ ∈ [−τ, 0], which implies that for sufficiently small δ > 0, for any t ∈ [ρ ε , ρ ε + δ), 14) It follows from this that (3.16) Letting ς = ζ/2 and applying Lemmas 3.5 and 3.6 lead to and Noting that ε is sufficiently small, we therefore have which contradicts the definition of ρ ε . Thus, (3.11) holds for all t ≥ 0. This completes this proof.
Clearly, in Theorem 3.1, condition (3.3) plays a crucial rule to determine the moment stability. Nevertheless, it is not convenient to check this condition because it is not related to the coefficient of f explicitly. In what follows, we impose some conditions on f to guarantee Theorem 3.1. These conditions show the roles of coefficient f as well as the Markov switching r ε (t) in the moment stability of switching FDE (2.4) for sufficiently small ε > 0.
Corollary 3.7. Under Assumptions 2.2 and 2.3, assume that there exist constants ζ i , σ i and a probability measure η ∈ M such that for any ϕ ∈ C([−τ, 0]; R n ) and t ≥ 0, (3.20) That is, the trivial solution of the system (2.4) is mean-square exponentially ε-stable.
Remark 3.8. The conditions in Theorem 3.1 are given in terms of the function V , so they are not easily verifiable. Condition (3.20) in Corollary 3.20 is given in terms of the function f . As a result, stability can be checked by using f . Comparing these two results, Theorem 3.1 is a general one, whereas Corollary 3.7 considers a special case.
In this corollary,ζ >σ ∨ 0 is a crucial condition. This condition does not require that every ζ i is sufficiently large to guarantee that the every subsystem is stable as traditional results (for example, [24]). Rather it only requires that its expectation ζ with respect to the stationary distribution measure of the fast-varying part of the Markov chain is sufficiently large, this implies that some ζ i may be positive (those systems are often unstable). Hence this corollary reveals that the averaging principle plays an important role and the Markov chain r ε (t) may serve as a stabilizing factor. 4. DDEs with Markovian switching. As a class of special cases of FDEs with Markovian switching, the Markovian switching DDEs have broad applications. By the moment stability theorem established above on switching FDEs, this section examines the moment exponential stability of switching DDEs with multiple delays. Let us consider the DDE with Markovian switching of the forṁ satisfying the Lipschitz condition, where τ = max{τ 1 , . . . , τ k }, F : R n × R n×k × R + × S → R n , is a Borel measurable function. Define µ 0 , µ 1 , . . . , µ k as the Dirac measures in 0, τ 1 , τ 2 , . . . , τ k . Let us choose It is clear that µ is a probability measure. For (4.1), Assumption 2.3 may be specialized as follows. for any x 0 , . . . , x k , y 0 , . . . , y k ∈ R n with |x 0 |∨|x 2 |∨· · ·∨|x k |∨|y 0 |∨|y 2 |∨· · ·∨|y k | ≤ j.
The linear growth condition may be rewritten in the following way.
then, for the solution of (4.1), Then we may present the following corollary, which may illustrate the "averaging" idea on the stability of stochastic switching equations with multiple delays. If there exist positive constants β i , i = 1, . . . , m such that then for any initial data ξ ∈ C([−τ, 0]; R n ) and sufficiently small ε > 0, where q ∈ (1,λ 0 /λ 0 ) is the unique root of the equationλ 0 − qλ = log q/τ , wherē That is, x(t), the solution of (4.1) is mean square exponentially ε-stable.
Remark 4.6. In this corollary, condition (4.11) is very important. This condition is expressed as the average value with respect to the stationary distribution of the fast-varying part of Markov chain r ε (t) (with weight β i ). This implies that both the switching Liapunov function and the stationary distribution will have crucial effect on the stability result.

5.
ODEs with Markovian switching. Letting τ = 0, Eq. (2.4) or (4.1) becomes the ordinary differential equation (ODE) with Markovian switching of the forṁ where f is a function from R n × R + × S to R n , and the initial value ξ becomes a bounded F 0 -measurable R n -valued random variable. Letting µ ∈ M be the Dirac measure in the origin, the local Lipschitz assumption (2.10) becomes for x, y ∈ R n with |x| ∨ |y| < j, where K ij is a constant dependent on j. Correspondingly, (2.7) becomes a function from R n × R + × S to R of the form 3) The following result follows directly from Theorem 3.1 or Theorem 4.3.
Theorem 5.1. Let λ, p, c 1 , c 2 be all positive numbers and q > 1. Under condition (5.2), if there exists a function V ∈ C 1,1 (R n × R + × S; R + ) such that conditions (i) and (ii) in Theorem 3.1 hold and the following condition is satisfied: then for all bounded initial value ξ ∈ R n , (5.1) is pth moment exponentially ε-stable and the pth moment Liapunov exponent is not greater than −λ.
Remark 5.2. In general, for λ > 0, LV (x, t, i) + λV (x, t, i) ≤ 0 for some i ∈ S may guarantee that the ith subsystem is stable and LV (x, t, i) − λV (x, t, i) > 0 implies that the ith system is unstable. condition (iii) in this theorem shows that, even if some subsystems are unstable under the condition LV (x, t, i) − λV (x, t, i) > 0, the switching system may still be stable if condition (iii) holds. That is, the Markov chain r ε (t) may serve as a stabilizing factor for sufficiently small ε > 0.
Let A i = A(i). It is well-known that the ith subsystem is exponentially stable if then condition (iii) of Theorem 5.1 is satisfied. Applying Theorem 5.1 gives the following corollary. Note that λ max (A i +A i ) < 0 shows that A i +A i is negative definite. By Corollary 5.3, even if A i + A i is positive definite for some i = 1, 2, . . . , m, the switching system (5.4) may still be mean-square exponentially stable if condition (5.5) holds. This shows that if even not all subsystems are stable, it is possible that the switched system may be stable. Namely, the Markov chain may be a stabilizing factor for small parameter ε.
6. Examples and remarks. The model for the Markov chain given in (2.2) can be generalized to where both Q and Q 0 are generators of suitable continuous-time Markov chains. Here, Q/ε represents the fast-varying part and Q 0 /ε γ0 represents the slow-changing part. All the results obtained before hold true in this system. To proceed, we present three examples next, in which we use (6.1) with γ 0 = 0.

6.2.
Remarks. This paper has focused on stability of functional differential equations with Markov switching. The Razumikhin-type techniques state that if the derivative of a Liapunov function along system trajectories is negative whenever the current value of the function dominates its recent history of the delay interval, then the Liapunov function along the trajectories will converge to zero. In this work, we have established the Razumikhin-type theorem on exponential ε-stability in the sense of the pth moment for functional and delay differential systems with two-time-scale Markov chain as the switching factor. This result implies that the Markov chain may serve as a stochastic stabilizing factor. In this stochastic stabilizing process, the rapid-switching part plays an important role.