Stochastic epidemic models driven by stochastic algorithms with constant step

In this paper we propose the stochastic epidemic model that relates directly to 
the deterministic counterpart and reveal close connections between these 
two models. Under the classic assumptions, the sample path of process eventually 
converges to the disease-free equilibrium, even though the corresponding deterministic 
flow converges to an endemic equilibrium. From the fact that disease can occur 
sporadically, we adjust the stochastic model slightly 
by introducing a stochastic incidence and establish precise connections between 
the long-run behavior of the discrete stochastic process 
and its deterministic flow approximation for large populations.


Introduction.
Recently both deterministic and stochastic models are used to model biological processes involving interacting populations.
In a deterministic SIS epidemic model, individuals in the population are classified according to disease status, either susceptible or infectious. A susceptible individual, after a successful contact with an infectious individual, becomes infectious, but does not confer immunity to the disease (e.g., sexually transmitted diseases) in the population. Hence, after recovery, infected individuals return to the susceptible class.
It is well-known that growth rate in population biology equals birth rate subtracted by death rate, which implies that both birth rate and death rate are the same when population number gains its maximum. It is reasonable to assume birth rate equals death rate when population number is around its maximum.
Let S(t) and I(t), respectively, denote the number of susceptible and infected individuals at time t. We assume that there are no disease-related deaths and all newborn infants are susceptible. The classic epidemic model of SIS type with vital dynamics (births and deaths) is given by the following standard incidence form: where N = S(t) + I(t) is the total population size, parameter b > 0 is the natural birth rate, γ > 0 is the recovery rate, β > 0 is the contact rate. This SIS epidemic model implies that there is an inflow of newborns into the susceptible class at rate bN . As mentioned before, we have assumed that the birth rate is equal to the death rate. Here (S, I) ∈ D N = {(S, I) : 0 ≤ S ≤ N, 0 ≤ I ≤ N, S + I ≤ N }. The dynamics of model (1) is determined by the basic reproduction number R 0 = β b+γ (see, e.g., [9]). Theorem 1.1. Let (S(t), I(t)) be a solution of (1).
(i) If R 0 ≤ 1, then P 0 = (N, 0) is the unique equilibrium in D N and it is globally asymptotically stable.
(ii) If R 0 > 1, then there is a unique positive equilibrium P * = (N 1 R0 , N (1 − 1 R0 )) and it is globally asymptotically stable with respect to positive solutions.
Let s(t) = S(t) N and i(t) = I(t) N be the susceptible and infectious fractions, respectively. Dividing the equations (1) by the constant total population size N yields   Analogously, we have Theorem 1.2. Let (s(t), i(t)) be a solution of (2). (i) If R 0 ≤ 1, then there is a unique equilibrium p 0 = (1, 0) in D 1 and it is globally asymptotically stable.
In fact, we can merely consider one independent variable i, because s + i is asymptotic to 1 as t → ∞. Hence, the model in (2) can be replaced by the following form: Recent works by Li and Xu [11] have shown the precise relationship between the deterministic and stochastic SIS epidemic models. The present paper is motivated by their works. Our aim in this paper is the same as theirs but with different approach, which relies on the theory of stochastic approximations, while Li and Xu's works rest on master equation.
The paper is organized as follows. Under the probabilistic assumptions on coefficients, the variant stochastic epidemic model is established in Section 2. The theme of this paper is presented in Section 3. We give a sufficient condition to guarantee a close connection between sample paths of the stochastic model and trajectories of the corresponding deterministic average differential equation. Section 4 gives more applications for our theoretic results.
The following notations and definitions will be used throughout. Let M be a given metric space. Let P(M ) and C b (M ) denote the set of probability measures and real-valued bounded continuous functions on M , respectively. A probability µ ∈ P(M ) is P invariant (or Markov chain X invariant) if it satisfies µP = µ, i.e., M P (x, B)µ(dx) = µ(B), ∀B ∈ B(M ), where P (x, B) is a Markov transition probability function of X and B(M ) is the Borel σ-algebra on M .
A family L ⊂ P(M ) is tight if for every η > 0 there exists a compact set K ⊂ M such that µ(K) ≥ 1 − η for all µ ∈ L . When M is a Polish space (i.e., a separable complete metric space), by the Prohorov theorem, a family L ⊂ P(M ) is relatively compact for the topology of weak * convergence if and only if L is tight.
2. Stochastic epidemic models. We are interested in the infectious fraction I N , when the total population N is large enough. So it is like the biochemistry's "large volume, small molecules" model. It is suitable to consider the discrete time stochastic model.
The total population N is fixed. Time is discrete. At any time k ∈ N, let S k and I k denote random variables for the number of susceptible and infected individuals at time k, respectively. Since S k = N − I k , there is only one independent random variable I k . The stochastic process {I k : k ∈ N} has the Markov property, that is, for any time k ∈ N. The Markov property says that the process at time k + 1 only depends on the process at current time k.
We make one more assumption: at any time k ∈ N, the number of infected individuals changes by at most one during time interval k → k + 1, that is, 1} denote the infectious fraction at time k and it is also a random variable. The assumptions of the process {I k : k ∈ N} imply that the process {X k : k ∈ N} enjoys the Markov property and the random variable The basic object of interest is the stochastic process {X k : k ∈ N} taking values in [0, 1], particulary in its long-run behavior.
Under the probabilistic assumptions, the stochastic model can be formulated as follows. Given the time k ∈ N and the state X k = x ∈ [0, 1].
The conditional probability of a new infection, x → x + 1 N equals the probability that an individual (called Q k ) is chosen from the susceptible class at time k, which is 1 − x, times the probability that Q k becomes infected at next time k + 1, which consists of the probability that Q k meets other individual, says A(x) ∈ [0, 1] (assumed to depend on x), conditioned on such a meeting, the probability that Q k meets the infective class, which is x, times the probability β 0 ∈ (0, 1] that this contact infects Q k . Hence, the conditional probability P( The conditional probability of a recovery or death, x → x − 1 N is the probability that an individual (called Q k ) is chosen from the infective class at time k, which is x, times the probability that Q k is susceptible or death at time k+1, i.e., it comes from the probability of recovery which is γ(x) (assumed to depend on x) or the probability of a death which is b ∈ (0, 1]. That is, . Finally, the conditional probability of no change (the only other possibility), For simplicity, we choose A(x) and γ(x) are both positive constants such that lies in the interval [0, 1]. This implies β > 0 and γ + b > 0 are both constants. These assumptions make the stochastic epidemic model be a useful approximation to the model (3). Without causing confusion, we write β, γ and b as β, γ and b, respectively.
This implies that the sample paths of X N approach the disease-free equilibrium (as N → ∞), regardless of the magnitude of the basic reproduction number. This is the most important difference between the deterministic and stochastic epidemic models.
As we know that the disease cases can occur sporadically in the host population with a small probability α due to environmental variability, e.g., transmission by another species, contaminated blood banks, foreign travel, etc. Therefore we make a small adjustment with a stochastic incidence into the model (see, e.g., [4,11]): the conditional probability of a new infection also comes from the susceptible class (which is 1 − x), times the small probability α. Because for large populations, we assume for simplicity that the conditional probability α 1 is constant. Assume that α > 0. Then the previous epidemic models are modified slightly, It is easy to see from the transition diagram in Figure 1 that the Markov chain {X k } k∈N is irreducible, that is to say, there is nonzero probability of eventually going from any state to any other state. Furthermore, it follows from positive recurrence that {X k } k∈N is a finite irreducible Markov chain. Therefore from Perron-Frobenius theorem it admits a unique invariant probability measure As total population N goes to ∞, the variable x is of course viewed as real number in [0, 1]. The conditional where Let F k denote the σ-algebra generated by the random variables X 0 , · · · , X k and . Manipulating the equation (4), we obtain the recursion formula By the assumption that the random variable Then a family {X k } k∈N of processes, parametrized by > 0, satisfies recursion of the form Recursive processes described by (6) arise frequently in a large variety of fields such as control theory, game theory, biological modeling and elsewhere under the name stochastic approximations, which are widely studied in [2,3,8].
3. Rigorous relationship between the stochastic and deterministic models. We note that E[U k+1 |F k ] = 0 implies EU k = 0 for k ≥ 1 . Therefore, one can think of (6) as a kind of Euler approximation scheme for numerically solving the following differential equation Here we give some conditions to make this intuition precisely.
The Hypothesis 1 implies that F generates a flow In order to analyse the behavior of {X k } k∈N in terms of the behavior of the flow Φ it is convenient to introduce the interpolated process X ,x : It follows from (5) that Hypothesis 2 holds.
The following proposition is a standard averaging result for the stochastic approximation processes.
Proposition 3.1. Assume that Hypotheses 1 and 2 hold. Let K ⊂ R m be a compact set. Then for any T > 0 and δ > 0, we have where [ s ] denotes the integral part of s . And since subtracting (9) from (8) and taking sup-norm, we obtain sup 0≤τ ≤t Our next goal is to estimate the order of L 2 norm on the left-hand side of the above inequality. So we consider each term's L 2 norm on the right-hand side. By Hypothesis 1, we obtain It is easy to see that where in the last inequality, we have used so-called "area principle", which is stated as follows: Therefore, the last inequality is a consequence of the "area principle" with i = 0, We continue to estimate the second term's L 2 norm on the right-hand side. A simple computation yields where we have used Burkholder-Davis-Gundy's inequality (see, [6]) and Hypothesis 2 in the last two inequalities, B 2 > 0 is a universal constant. Also, by the Lipschitz assumption of Hence, for all x ∈ K, we get In other words, for every sufficiently small, we obtain Then Gronwall's inequality yields that, ∀x ∈ K,  Proof. Fix t > 0 and let n = n(t, ) := [ t ]. Let X ,x denote the interpolated process with initial condition X 0 = x ∈ R m . For any g ∈ C b (R m ), since µ is (P ) n invariant, we have Let η > 0. Since the sequence {µ i } is tight, there exists a compact set K ⊂ R m such that inf By uniform continuity of g on a compact neighborhood of Φ t (K) there exists δ > 0 such that ∀x ∈ K and u−Φ t (x) < δ implies |g(u)−g(Φ t (x))| < η. By continuity of the flow Φ · (·) on a compact set [t−1, t]×K there exists δ 1 := δ 1 (δ) ∈ (0, δ) such that Φ n (x) − Φ t (x) < δ 2 for any x ∈ K and < δ 1 . Hence |g(Φ n (x)) − g(Φ t (x))| < η for any < δ 1 , x ∈ K. We claim that To prove the claim, we use the fact that for any i < δ 1 and for i < δ 1 and x ∈ K. Clearly for i < δ 1 ,

Proposition 3.1 yields
Since η > 0 is arbitrary and for any g ∈ C b (R m ). This shows that µ is an invariant measure of the flow Φ.
According to Proposition 3.3, some further investigations can be carried out by studying the properties of the Birkhoff's center B(Φ). Following [3,Proposition 3.2], if the flow Φ is dissipative, then each component of supp(µ) is internally chain transitive [5,7].
The main result in this paper is summarized as follows. We now consider the ordinary differential equation that is the average equation for recursion (6) dx Figure 2. (i) If α = 0, then µ is a Dirac measure supported at 0, i.e., µ = δ 0 .
(ii) If α > 0, then (10) has a unique equilibrium p, which is positive and globally asymptotically stable, and µ is a Dirac measure supported at p, i.e., µ = δ p .
Proof. (i) follows from the Markov chain theory. (ii) follows from Theorem 3.4.
The following assertion gives precise description about the connection between the deterministic and stochastic models.
Proof. The idea of the proof is borrowed from [3]. In order to prove the theorem, it suffices to show that there exists a full measure set Ω 0 , for every ω ∈ Ω 0 , where U c denotes the complementary set of U in [0, 1]. Given N ∈ N * , for ω ∈ Ω, there exists a subsequence n i → +∞ (depending on ω, N ) such that lim sup By the tightness of {τ N n } n∈N , we can suppose that {τ N ni } i∈N weakly converges toward some probability measure µ N . On the other hand, the process {X N k } k∈N is a finite Markov chain and the assumption α > 0 implies that this chain is irreducible and all states are positively recurrent. Therefore the sample-path ergodic theorem implies that µ N is almost surely (Ω N ) an invariant probability measure of {X N k } k∈N (see, e.g., [

This implies that lim sup
Then Ω 0 is a full measure set and for every ω ∈ Ω 0 , we get lim sup Assume that the claim is not true. Then there exists a sequence N i → ∞ such that lim Ni→∞ [0, 1] g(x)µ Ni (dx) > 0.
By the tightness of {µ Ni } Ni∈N , we can suppose µ Ni w − −→ µ = δ p , and Theorem 3.5 (ii) implies that µ([0, 1] \ W ) = 0. Thus we have lim contradicting to (11). Therefore we obtain that lim N →∞ lim sup n→∞ τ N n (U c ) = 0 for every ω ∈ Ω 0 . Theorem 3.6 shows that as N → ∞, the process {X N k } k∈N almost surely spends most of the time in the neighborhood of the unique equilibrium p.
4. Application to SIR epidemic models with stochastic incidence. In this section we will build the stochastic SIR model. Then we shall apply our previous theoretic results to analyze the long-run behavior of this stochastic model. In SIR epidemic model, individuals in the population are classified by three status: susceptible, infected or removed which are denoted by S, I and R, respectively. The classic SIR model with a standard incidence is given by the following form: where b > 0 is the birth rate, γ > 0 is the recovery rate, β > 0 is the contact rate and N = S(t) + I(t) + R(t) is the total population size. As explained in SIS model, we have still assumed that birth rate equals the death rate, so that the total population size N is constant. For details we refer to Hethcote [9] and the references therein. Let s(t) = S(t) N , i(t) = I(t) N and r(t) = R(t) N be the susceptible, infectious and removed fractions, respectively. Dividing the equations (12) by the constant total population size N yields Note that r(t) = 1 − s(t) − i(t), we can consider two independent variables s(t) and i(t). Therefore, we merely consider the following form: where (s, i) ∈ D 1 = {(s, i) : s ≥ 0, i ≥ 0, s + i ≤ 1}.
(i) If R 0 ≤ 1, then there is a unique equilibrium p 0 = (1, 0) in D 1 and it is globally asymptotically stable. ( is a unique positive equilibrium in D 1 \{0} and it is globally asymptotically stable for the positive initial conditions. We now use the language of probability theory to build stochastic SIR models which are similar to the previous stochastic SIS models. Let the total population size N be fixed. At any discrete time k ∈ N, let S k , I k and R k denote random variables for the number of susceptible, infected and removed individuals at time k, respectively. Note that R k = N − S k − I k , there are two independent variables S k and I k . The process {(S k , I k )} k∈N has the Markov property. Let X k = S k N and Y k = I k N denote the susceptible and infectious fractions at time k, respectively. The process {(X k , Y k )} k∈N also enjoys Markov property from the assumption of For x, y ∈ [0, 1] and suppose X k = x, Y k = y. We assume that, at any time k ∈ N, there is at most one individual changing her status (susceptible, infected or removed) in the population during time interval k → k + 1. This implies that the possible values As a brief introduction, we merely give the thought of how to build stochastic models.
The conditional probability of the event that occurs if and only if the probability that she (called Q k ) is chosen from susceptible class, which is x, times the conditional probability that Q k becomes infected at next time k + 1, which is βy.
The conditional probability of the event that X k+1 = x, Y k+1 = y − 1 N occurs if and only if the probability that Q k is chosen from infective class, which is y, times the conditional probability that Q k is recovery at time k + 1, which is γ. That is, The conditional probability of the event that X k+1 = x + 1 N , Y k+1 = y occurs if and only if the probability that Q k is chosen from removed class, which is 1 − x − y, times the conditional probability that Q k is death at next time k + 1 (since the total population size N is constant), which is b. We obtain P( Similarly, X k+1 = x + 1 N , Y k+1 = y − 1 N if and only if Q k is chosen from infective class, and at time k + 1, she is death, i.e., P( Finally, the conditional probability of no change (the only other possibility), For simplicity, we make additional assumption that b, β and γ are all positive constants such that 1 − βxy − γy − b(1 − x) ≥ 0 for all (x, y) ∈ D 1 .
For each N ∈ N * , the sequence of two-dimensional random variable (X N , Y N ) = {(X N k , Y N k ) : k ∈ N} is a discrete-time Markov chain taking value in a finite lattice It is easy to show that the state {(1, 0)} is absorbing and that all other states are transient. Thus it has a unique invariant probability measure δ {(1,0)} (see the discussion of the SIS stochastic model). And it implies that the disease eventually dies out.
Compare this asymptotic result to Theorem 4.1, there is a big difference between the deterministic and stochastic models. We adjust the model by introducing a stochastic incidence. The only useful difference is that where α is for simplicity a small positive probability, because the disease can occur sporadically in the susceptible class by the environmental variability. Then it is easy to see that Markov chain {(X k , Y k )} k∈N is irreducible and every state is positive recurrent. Therefore, {(X k , Y k )} k∈N has a unique invariant probability measure µ N , where each component of µ N is positive. In order to show the average equation, we compute the conditional probability that P(X k+1 − X k = 1 Let F (x, y) = (P (x, y), Q(x, y)) and Z k = (X k , Y k ), it is easy to verify that Let σ-algebra F k denote by σ{Z 0 , · · · , Z k }. It is clear that {U k } k∈N * is a martingale difference sequence, i.e., U k is measurable with respect to F k and satisfies E[U k+1 |F k ] = 0. Together with above assumptions, we also have U k ≤ max (x,y)∈[0,1] 2 F (x, y) + 2.
(15) Let = 1 N , then the process {Z k } k∈N , parameterized by , also satisfies the recursion form (6). We now consider the average equation: Here β, b, γ and α are positive numbers and (x, y) ∈ D 1 .
The asymptotic behavior of solution paths is described in the following theorem.
Theorem 4.2. Let F = (P (x, y), Q(x, y)) be the vector field defined by (16), then there exists a unique positive equilibrium p and it is globally asymptotically stable.
Proof. The proof of this theorem follows the line of argument in Hethcote [9] using standard phase plane methods. The set D 1 is positively invariant since no direction vectors at the boundary of D 1 are outward (see, e.g., [13] and the references therein). It is easy to see that the system (16) has a unique positive equilibrium p = (x * , y * ) i.e., the characteristic roots of the linearization around p have negative real parts, this implies that it is locally asymptotically stable. Choose B(x, y) = 1 y . Then from routine computation, ∂(BP ) ∂x + ∂(BQ) ∂y = 1 y (−b − α) − β − αx y 2 < 0. Thus, by Dulac's criterion, the system (16) has no periodic orbit in D 1 . Consequently, the unique equilibrium p is globally asymptotically stable. Using Theorem 4.2 it is easy to derive from Theorem 3.4 the following assertion. (ii) If α > 0, then (16) has a unique equilibrium p, which is positive and globally asymptotically stable, and µ is a Dirac measure supported at p, i.e., µ = δ p .
As in the stochastic SIS case we can also have the same result as Corollary 3.7. In SIS and SIR models with stochastic incidence, the results of Theorem 3.5 (ii) and Theorem 4.3 (ii) imply that, the deterministic models have a unique globally stable equilibrium p. And for the discrete time stochastic models, it follows from Corollary 3.7 that sample paths tend to cluster near p as N → ∞. This implies that the results of deterministic and stochastic models are reconciled.