Analysis of a Lévy-diffusion Leslie-Gower predator-prey model with nonmonotonic functional response

In this paper, a Levy-diffusion Leslie-Gower predator-prey model with a nonmonotonic functional response is studied. We show the existence, uniqueness and attractiveness of the globally positive solution to this model. Moreover, to its corresponding steady-state model, we obtain the stability of the semi-trivial solutions, the existence and nonexistence of coexistence states by the method of topological degree, the uniqueness and stability of coexistence state, and the multiplicity and stability of coexistence states by Grandall-Rabinowitz bifurcation theorem. In addition, to get these results, we study the property of the Levy diffusion operator, and give out the comparison principle of the generalized parabolic Levy-diffusion differential equation, as well as the existence and stability of the solution for the steady-state Logistic equation with Levy diffusion. Furthermore, we obtain the comparison principle of the steady-state Levy-diffusion equation. As far as we know, these results are new in the ecological model.


1.
Introduction. From the pioneering works of Lotka and Volterra [23], a lot of authors studied dynamical features of ecosystem, such as the interaction of predator and prey. Therein, the well-known predator-prey model is considered as the following form: which is called as a modified Leslie-Gower predator-prey model. where x and y represent population densities of prey and predator at time τ , respectively. All parameters s 1 , s 2 , k 1 , k 2 and h are positive. s 1 and s 2 denote the intrinsic growth rates of the prey and predator, respectively. k is carrying capacity of the prey. p(x), which is called a functional response, is a consumption rate of prey by a predator. The concept of the functional response was first introduced by Solomon (1949). The carrying capacity of the predator's environment is assumed to be proportional to the prey abundance, i.e., (x+k 2 )/h, which is called as a modified Leslie-Grower term [2]. The functional response p(x) has been developed during various different processes of energy transfer (see [35]). In detail, there are three important forms of functional responses, that is, (I) p(x) = mx, (II) p(x) = mx/(a+x) and (III) p(x) = mx 2 /(a+ x 2 )(sigmoidal), where m is the maximum predation rate and a is a half-maximum rate. These three responses are monotone-increasing functions with respect to the prey x. In addition, the form of p(x) = mx/(a + x 2 ) in [12,25] is called Holling type IV functional response, which is a nonmonotonic function with respect to x. The generalized expression of Holling IV type is p(x) = mx/(ax 2 + bx + 1), (2) which is introduced in [26,34]. In this functional response, the parameter a is a positive constant. If b > −2 √ a, then ax 2 + bx + 1 > 0 for x ≥ 0. This functional response seems reasonable if the prey and webbing densities are directly related [1]. When b is nonnegative, the functional response (2) resembles the Holling type II for small enough x while the effect of inhibition appears for large x, see Figure 1(A). If −2 √ a < b < 0, the function (2) remains nonnegative and the inhibition effect still works for large x. This function is like Holling type III for small x, see Figure 1(B) [34]. Introducing this functional response into the system (1) yields    dx dτ = x s 1 − x k1 − mxy ax 2 +bx+1 , dy dτ = y s 2 − hy x+k2 .
For simplicity, we apply the following transformation to the variables and the parameters in the system (3) x = k 1 u, y = v/k 1 , ak 2 1 =ā, bk 1 =b, τ = k 1 t, s 2 k 1 = η, r = h/k 2 1 , k 2 /k 1 = k, and we still use a, b to stand forā,b. Thus (3) becomes the following form: In real world, the spatial heterogeneity of the predator's and prey's distributions is obvious more or less. Thus, the predator or prey can spread from higher density regions to lower density ones. This phenomenon is called as Gaussian diffusion (or normal diffusion). This process can be often described by the Laplacian operator. Then, (4) can be rewritten into a reaction-diffusion system as the following form: ∂u ∂t = ∆u + u (c − u) − muv au 2 +bu+1 , x ∈ Ω, t > 0, ∂v ∂t = ∆v + v η − rv u+k , x ∈ Ω, t > 0, (5) where the diffusion coefficients of the prey and predator are taken as 1. The region Ω ⊂ R n is open, bounded and equipped with the Lipschitz boundary, where R n is an arbitrary positive-integer n dimension space. ∆ is the Laplacian operator and the infinitesimal generator of Brown motion (also called as Gaussian process) with the corresponding characteristic of (0, I, 0). It is well known that Brown motion is a special rotationally invariant stable Lévy process. At present, many authors usually use the Laplacian operator to describe diffusion process of population and study mathematical properties of this kind of reaction-diffusion population models, such as stationary patterns [15,22,13,14,24,29,30,31], globally asymptotic stability [16,11,10,27,7] and the existence of positive solutions for steady state equation [32,33] and so on. However, diffusion process of population is always complex and diverse, for example, population often face some obstacles, stay and jump etc. As a result, population can not obey the rule of Gaussian diffusion. Moreover, the recent researches [21,9] have showed that predator (such as sharks, bony fishes, sea turtles and penguins) and prey exhibit Lévy-walk-like behaviours. Based on this case, we should consider the predator-prey system equipped with Lévy diffusion (also called as anomalous diffusion), instead of normal diffusion. A Lévy diffusion process is a stochastic process with independent and stationary increments, which extends the concept of Brownian motion. Essentially, Lévy processes are obtained when one relaxes the assumption of continuity of paths (which gives the Brownian motion) by the weaker assumption of stochastic continuity. By the Lévy-Khintchine formula, the infinitesimal generator of any Lévy processes is an operator in the following form where ν is the Lévy measure, and satisfies R n min{1, |y| 2 }dν(y) < ∞. When the process has no diffusion and drift parts, (6) takes the following form Further, if one assumes that the process is symmetric, and that the Lévy measure is absolutely continuous, then (7) is turned into Here K : R n \ {0} → (0, +∞) is a function, and We assume that the operator L K satisfies the conditions: there exists a constant θ > 0 such that The parameter s ∈ (0, 1) is fixed and n > 2s. The conditions (9) are throughout this paper. A typical example for K is given by K(x) = |x| −(n+2s) , s ∈ (0, 1). For this case, the operator L K is the fractional Laplace operator −(−∆) s , see [6], which is the infinitesimal generator of the rotationally invariant stable Lévy process with stable index s ∈ (0, 1). The fractional Laplace operator is an important nonlocal diffusion one and has been successfully used in quantum physics [17,20] and epidemic model [8]. For the case of K(x) = |x| −(n+2) , the operator L K corresponds to the normal Laplace operator (or normal diffusion operator) ∆. It is worth mentioning that the fractional Laplace operator −(−∆) s converges to the normal Laplace operator ∆ as s → 1 − .
In this paper, we introduce the Lévy diffusion operator L K in the form of (8) with the conditions (9) into (4) as the following form Here, we consider the initial-boundary condition The Dirichlet datum is given in R n \ Ω and not simply on ∂Ω, consistently with the non-local character of the operator L K . The steady-state equations of (10) is As far as we know, the systems (10) and (11) have not been investigated. In this paper, we make the first attempt to fill this gap and study the mathematical properties for (10) and (11). To this end, we first examine the properties of the Lévy diffusion operator L K , and then obtain the existence and uniqueness of the positive solution to (10). By the comparison principle of the parabolic Lévydiffusion equation, asymptotic behavior of this solution is obtained. By the method of the topological degree, we show the existence, uniqueness and nonexistence of coexistence states to (11). Moreover, by applying Grandall-Rabinowitz bifurcation theorem and the property of the Lévy diffusion operator L K , we get the multiplicity and stability of coexistence states to (11).
The remainder of the paper is organized as below. In Sec.2, some necessary results and notations are introduced. In Sec.3, the basic properties of the Lévy diffusion operator L K are given out. In Sec.4, the existence of the globally positive solution of (10) is showed. In Sec.5, we examine the stability of semi-trivial solutions of the steady state system to (11). In Sec.6, we give out the conditions of the existence and nonexistence of coexistence states to (11). In Sec.7, we prove the uniqueness and stability of coexistence state to (11). In Sec.8, we apply Grandall-Rabinowitz bifurcation theorem and the spectrum of the Lévy diffusion operator L K to get the multiplicity and stability of the coexistence states to (11). Finally, we conclude this paper with a discussion in Sec.9.
2. Preliminaries. In this section we state the general assumptions on the quantity we are dealing with. We keep these assumptions throughout the paper. Denote X by the linear space of Lebesgue measurable functions from R n to R such that the restriction to Ω of any function g ∈ X belongs to L 2 (Ω) and the map where CΩ := R n \ Ω. Moreover, Note that The space X is endowed with the norm defined as From [18], it is well known that · X is a norm on X. We denote by H s (Ω) the usual fractional Sobolev space endowed with the norm In [18], when K(x) = |x| −(n+2s) , the norms in (12) and (13) are not the same, because Ω×Ω is strictly contained in Q. Thus, the classical fractional Sobolev space approach is not sufficient for studying the problem. But, there is the connections between the spaces X and X 0 with the fractional Sobolev spaces, see [18].
Theorem 2.4. [28] Assume that A is a sectional operator with the section S α,β , and Γ is a path in ρ(−A) satisfying the condition that there exists a constant θ ∈ (π/2, π) such that arg λ → ±θ as |λ| → ∞ and λ ∈ Γ, then e −tA is an analytical semigroup, whose infinitesimal generator is −A, where Lemma 2.5. [28] An operator A is self-adjoint and dense on Hilbert space H, and bounded below, that is, there exists a real constant µ such that then A is a sectorial operator.

Basic properties of Lévy diffusion equation.
To study the existence, uniqueness, stability and bifurcation of (10) and (11), we first discuss the comparison principle of Lévy diffusion equation and the existence and stability of the steady-state Logistic equation with Lévy diffusion. For convenience, we introduce some notations. For x, y ∈ R, we denote x ∧ y := min{x, y}, x ∨ y := max{x, y} x + := x ∨ 0, x − := (−x) ∨ 0 and extend these notations to real-valued functions. If (V, ≥) is partially ordered vector space, we denote its positive cone by V + := {v ∈ V : v ≥ 0}. Next, we consider the following linear Poisson equation with the Lévy diffusion operator as the following form where f ∈ L 2 (Ω). The operator −L K is an invertible operator. In view of Lemma 2.2, the operator (−L K ) −1 : X 0 (Ω) → X 0 (Ω) is compact. Then, from Proposition 2 in Appendix, it is known that the eigenfunction sequence {e k } k∈N corresponding to the eigenvalue λ k of −L K is a complete orthonormal basis of X 0 .
Lemma 3.1. The operator −L K is a sectorial operator.
Proof. According to the definition of the operator L K , we know that the definition domain of −L K is X 0 . X 0 is dense in L 2 (Ω). From the above discussion, for any h ∈ X 0 there exist a constant sequence a k , k ∈ N, such that h = ∞ k=1 a k e k , a k = h, e k . From Proposition 2 and by taking µ = 0.5λ 1 , we have Thus, by Lemma 2.5, it is well known that the operator −L K is a sectorial operator.
In view of Lemma 3.1, (d) in Proposition 2 and Theorem 2.4, one can get the following result. Remark 1. It is well known that the normal Laplace operator −∆ as a special case of the operator −L K is a sectorial operator, and that it also is the infinitesimal generator of the analytically positive semigroup, see [28].
Next, we introduce the comparison principle of the following equation, Lemma 3.3. Let u be a solution to (16) with f, g ≥ 0 a.e. in Ω. Then u ≥ 0 a.e.
Assume that u − is not identically zero and write u = u + − u − in Ω. Let φ = u − , and then R 2n Noticing that (u + (x) − u + (y))(u − (x) − u − (y)) = 0, one has On the other hand, we have known that Thus, we obtain a contradiction and finish this proof. Of course, we can get the following result immediately.
Corollary 1. Let L K be any operator and u 1 , u 2 ∈ X 0 satisfy According to the comparison principle, we can introduce the definition of upper and lower solutions of (15).
(ii) u ∈ X 0 is called as a lower solution if Next, we introduce the comparison principle for the parabolic Lévy-diffusion equation as the following form Lemma 3.5. Let w be a solution to (18), assume f ≥ 0, w 0 ≥ 0 a.e in Ω and g ≥ 0 Proof. Let w(x, t) is a solution of (18). Assume that w(x, t) is not identically zero. Thus, there exists a maximal subset Ω × J with Ω ⊂ Ω and J ⊂ [0, T ] such that w(x, t) < 0 for (x, t) ∈ Ω × J. By Theorem 3.2, the solution w(x, t) is continuous and derivable with respect to the time variable t. Since the initial value w(x, 0) ≥ 0, x ∈ Ω, there must exist a point (x 0 , t 0 ) on the boundary of Ω × J, such that w(x 0 , t 0 ) = 0, ∂w(x,t) ∂t (x0,t0) < 0 and w(x, t 0 ) ≥ 0, x ∈ Ω. By the definition of nonlocal operator, we check the sign of the first equation in (18) on the point (x 0 , t 0 ), But, the sign for the left hand side of the first equation in (18) on the point (x 0 , t 0 ) is the less-than sign (<). This contradiction is obtained. Therefore, we confirm w(x, t) ≥ 0 for (x, t) ∈ Ω × (0, T ], and complete the proof of this lemma. (18) is equipped with the normal Laplace operator, then Eq. (18) degenerates into the standard second-order partial differential equation of parabolic type. For this case, the conclusion of Lemma.3.5 is directly gotten by the maximum principle of second-order parabolic equation.
Hence, we can obtain the following Corollary. then Let λ 1 (−L K + q) be the principal eigenvalue of the following equation We denote λ 1 (−L K ) by λ 1 for simplicity and the eigenfunction corresponding to λ 1 is denoted by φ 1 .
First, we consider the steady-state Lévy diffusion equation as the following form where a > 0, b(x) ≥ , a.e., x ∈ Ω. The solution to (20) is denoted by Θ (a,b(x)) . Then, we have the following result. Proof. (i) Assume that (20) has a nontrivial positive solution u(x). We define a mapping J : X 0 → R as follows By Corollary 1, we know that 0 ≤ u(x) ≤ a , a.e., x ∈ Ω, and then the following inequation holds We get a contradiction. Thus, when a < λ 1 , then (20) has no nontrivial positive solution. Next, we will check the stability of the trivial solution u ≡ 0. Linearizing the equation (20) at u = 0, we obtain an eigenvalue problem Since a < λ 1 , all eigenvalues of (21) are positive. Therefore, the trivial solution u = 0 is stable. (ii) Let e 1 be an eigenfunction corresponding to the principle eigenvalue λ 1 for the operator −L K , where e 1 ∈ X 0 and λ 1 have been defined in Proposition 2. We define two functions with respect to the parameter θ ∈ R + , and

HONGWEI YIN, XIAOYONG XIAO AND XIAOQING WEN
Clearly, Ω b(x)e 3 1 (x)dx is a positive constant. The function F 2 (θ) is a monotone decreasing function with respect to the parameter θ. Since a > λ 1 , there exist a positive constant θ 0 such that that is, u = θ 0 · e 1 is a nontrivial positive solution to (20). Next, we will prove the uniqueness of the nontrivial positive solution to (20). Assume that there exists two nontrivial positive solutions to (20), denoted by u 1 and u 2 , respectively. Then, for any Thus, we have u 1 (x) = u 2 (x), a.e., x ∈ Ω. The unique nontrivial positive solution to (20) Finally, we discuss the stability of the solutions in the case of λ 1 < a. It is obvious that the first eigenvalue of (21) is less than 0. Therefore, the solution u = 0 is unstable. Linearizing the equation (20) at u =ǔ, we obtain an eigenvalue problem By virtue of (22), the minimum eigenvalue λ * 1 of (26) is Therefore, the solution u =ǔ of (20) is stable in the case of a > λ 1 . We complete this proof.
x ∈ Ω. For this case, Lemma 3.6 is still available. This proof only needs to make small modification. In detail, let be a sufficient small positive constant, we consider the following equation where b(x) + ≥ a.e. x ∈ Ω. By the method in Lemma 3.6 and by letting → 0, we can obtain lim →0 u = u in the sense of the X 0 norm, where u satisfies (20).
The results about Eq. (20) with the special case of the normal Laplace operator are well known. But, for the general Lévy-diffusion form, few result is known. Eq. (20) is crucial to prove the stability and bifurcation of the steady-state solution for (11). Thus, here we give out the detailed proof of Lemma.3.6.

4.
Existence and uniqueness of globally positive solution to (10). In this section, by using Theorem 3.2 we will prove the existence and uniqueness of the solution to the system (10). (10) can be rewritten as a abstract differential equation: First, we give out the following Lemma.
Lemma 4.1. For every z 0 ∈ H + , the Cauchy problem (28) has a unique nonnegative maximal local solution which satisfies the following Duhamel formula for t ∈ [0, T max ) : Proof. The operator A defined in H can generate an analytic, condensed, strong continuous operator semigroup e tA . Furthermore, from Proposition 2 and for t > 0 we have where M is a positive constant and 0 < M < λ 1 . Noting that F : D(A) → H satisfies locally Lipschitz condition on a bounded set and by the contracting mapping principle and theorem 7.2.1 in [3], we know that there exists the unique solution for the evolution equation (28) defined on a maximal interval (0, T max ). Next, we shall show that this solution is nonnegative. To prove u(x, t), v(x, t) ≥ 0 for t ∈ (0, T max ) and a.e. x ∈ Ω, we consider the following auxiliary system   with boundary and initial conditions

HONGWEI YIN, XIAOYONG XIAO AND XIAOQING WEN
Multiplying (34) by u − and v − , respectively, integrating over the domain Ω and noting is a solution of (10), and according to the uniqueness of the solution in Lemma 4.1 we get that So, we get this lemma.
Remark 6. Of course, there are other ways to prove the nonnegativity of this solution. For example, we may apply the positive property of the operator semigroup e tA t≥0 because the resolvent (λ − L K ) −1 is positive for λ ∈ ρ(L K ). Next, we shall prove the existence of the globally positive solution for (10). Via Lemma 4.1 we only need to show that the solution of (10) is essentially bounded, i.e., dissipation.
By Corollary 2, we can know that u( x ∈ Ω.
By the same method, we can obtain v(x, t) ≤v a.e. (x, t) ∈ Ω × (0, T ]. As a result, we complete this proof.
By Lemmas 4.1 and 4.2, we can get the following theorem.
Theorem 4.3. The system (28) has a unique, nonnegative and bounded solution
Next, we will discuss the sufficient conditions for the globally asymptotic behavior of the semi-trivial solutions of (11).
Proof. (i) Since c > λ 1 and −L K u ≤ u(c − u) and by Corollaries 2 and 1, we get Let be a sufficiently small positive constant with c − m > λ 1 . Then, there exists a constant T ≥ 0 such that u(x, t) ≤ u * + for t > T . Then, we have Since η < λ 1 , by virtue of Lemmas 3.6, 3.5 and Corollary 1, we get v(x, t) → 0 as t → ∞ for all x ∈ Ω. Therefore, there exists a constant T such that v(x, t) < for t > T . Hence, we have Similarly, we obtain Using the continuity for → 0, (36) and (39), it follows that u(x, t) → u * as t → ∞.
(ii) It can be shown similarly as in the proof of (i).
Assume that c > λ 1 , and by Lemma 3.6, the following problem has a unique nontirivial positive solution if η > λ 1 . We denote the unique nontrivial positive solution by v * . From Corollary 1 we know v * ≤ v * .
The following theorem provides sufficient conditions to ensure permanence to (10).
and the equations is a positive global attractor of (10).
Proof. Let It is obvious that f is monotone decreasing with respect to v, and that g is monotone increasing with respect to u. According to the definition 3.4 of the upper and lower solutions, it follows that (u * , v * ) and (Θ (c−mv * /z,1) , v * ) are a pair of ordered upper and lower solutions of (11). Furthermore, f, g satisfies the Lipschitz condition in a bounded set [Θ (c−mv * /z,1) , u * ] × [v * , v * ]. There exist two pairs of functions (û,v) and (ũ,ṽ), which satisfy (41) and (42).
Next, we prove that [ũ,û] × [ṽ,v] is a positive global attractor of (10). Let be a sufficiently small positive constant. Using similar method in the proof of Theorem 5.2, there exists a T ≥ 0 such that for all t > T , which satisfies (37). Thus, lim sup t→∞ v(x, t) ≤ v * by Lemma 3.5 and Corollary 1.
for all t > T . On the other hand, the first and second equations of (10) indicate that The condition of c − m(c+k)η zr > λ 1 ensures that the solution Θ (c−mv * /z,1) exists. By Lemma 3.5, we have Then, there exists a constant T ≥ 0 such that for all t > T . Finally, take T * = max{T , T , T ε }, and then for all t > T * , Letting → 0, we can obtain this result. 6. Existence and nonexistence of coexistence states. In this section, we will discuss the existence and nonexistence of coexistence states to (11).
6.1. Existence of coexistence states. In this subsection, we determine the sufficient conditions for the existence states to (11) by calculating the index of fixed points. First, we give out the priori estimates for coexistence states to (11).
Proof. At first, from the first equation in (11), it follows that . Thus, c > λ 1 . Similarly, we can obtain η > λ 1 . Since −L K u ≤ u(c − u), u ≤ u * and by the uniqueness of u * and the method of the upper and lower solution, we have u(x) ≤ u * < c. Similarly, we can get v * ≤ v(x) ≤ Θ (η,r/(c+k)) < η(c+k)/η.
Thus, from the proof of Lemma 6.1 we have the following result.
(2) By direct calculation, we get Here, we take q large such that c x ∈ R n \ Ω.
6.2. Nonexistence of coexistence states. In this subsection, we give some conditions to ensure the nonexistence of coexistence states to (11).
Proof. Assume that problem (11) has a positive solution (u, v), and by Lemma 3.3, we obtain 0 ≤ u < c and 0 ≤ v < η(c+k) r . Further, according to the property of principle eigenvalue, we have We get a contradiction.
a , η > c and mk > r(a + b + c), we have We get a contradiction.
(iv) Since a ≥ 4, −2 √ a < b < 0, c ≥ − b 2 √ a , η > c and km > r(ac 2 + bc + 1), we have We obtain a contradiction. ( We get a contradiction. The proof of this result is completed. 7. Uniqueness of coexistence state. In this section, we will study the uniqueness of coexistence state. At first, we discuss the following equation x ∈ R n \ Ω. (61) where v * is the unique positive solution of (40). Furthermore, the unique positive solution (u * , v * ) is non-degenerate and linearly stable.
Proof. By virtue of Lemma 3.6, the existence, stability and uniqueness of positive solution (u * , v * ) are obvious.
Let E be a real Banach space. W is called a wedge if W is a closed convex set and βW ⊂ W for all β ≥ 0. For y ∈ W, we define We always assume that E = W − W . Let T : W y → W y be a compact linear operator on E. We say that T has property α on W y if there exists s ∈ (0, 1) and w ∈ W y \ S y such that (1 − sT )w ∈ S y . Let A : W → W be a compact operator with a fixed point y ∈ W and A is Fréchet differentiable at y. Let L = A (y) be the Fréchet derivative of A at y. Then, L maps W y into itself. We denote by index W (A, y) the fixed point index of A at y relative to W. Theorem 7.2. Suppose η > λ 1 is fixed. If c > λ 1 (−L K + mv * ) and m is sufficient small, then (11) has a unique positive solution and it is linearly stable; Proof. From Theorem 6.5, (11) has at least a positive solution. In the following, we show the uniqueness and stability. When m is sufficient small, (11) is a regular perturbation of (61). In view of Lemma 7.1 and a standard regular perturbation argument, we know that any positive solution of (11) is non-degenerate and linearly stable for small m. On the other hand, since c > λ 1 (−L K + mv * ), c > λ 1 , it can easily be checked that trivial solution and semi-trivial solution of (11) are bounded away from any positive solution. In help of a compactness argument and the assumption, the equation (11) has at most finitely positive solutions. Let them be {(u i , v i )} : 0 ≤ i ≤ n. From the discussion above, we know that I − F (u i , v i ) is invertible on W (ui,vi) and F (u i , v i ) has no real eigenvalue being greater than 1.
8. Multiplicity and stability of coexistence states. In this section, we discuss the bifurcation of the parameter c by using the Grandall-Rabinowitz bifurcation theorem. To simplify the notation we definẽ Our main result in this section is that there exists a positive constant c * ∈ (λ 1 ,c) such that the problem (11) has at least two positive solutions for c ∈ (c − ε,c) for some small ε > 0, and has at least one positive solution for c ∈ [c * ,c]. We consider the bifurcatoin of positive solutions from the branch of semi-trivial solution: {(c, 0, v * ) : c > λ 1 }. By linearizing (11) at (c, 0, v * ), we obtain the following eigenvalue problem: A necessary condition for bifurcation is that the principle eigenvalue µ of (66) is zero, which occurs if c =c. Let φ be the positive eigenfunction corresponding toc, i.e., (c, φ) satisfies We assume that φ is normalized so that Ω φ 2 dx = 1. Since then the operator −L K + 2r k v * −η is invertible, and the operator −L K + 2r k v * − η Thus, we have the following result regarding the bifurcation of positive solution of (11) from (c, 0, v * ) at c =c.
By the compactness of (−L K ) −1 , it is easy to see that, by extracting subsequence, re-labeled by n, we have that lim n→∞ u n u n L 2 (Ω) = φ > 0 for some φ ∈ X 0 (Ω) with φ L 2 (Ω) = 1. Thus, passing to the limit as n → ∞ in the previous identities, we find that that is, Thus,c 1 =c. This proof is finished. Next, the stability of the positive solutions bifurcating from the semi-trivial solutions will be studied.
Proof. Denote c = c(s) and (u, v) = (u(s), v(s). Then the corresponding linearized problem at (u, v) can be written as Letting s → 0 + , we get It is obvious that 0 is the first eigenvalue of the operator −L K −c + mv * ifc = λ 1 (−L K + mv * ). On the other hand, since λ 1 (−L K − η + 2rv * /k) > λ 1 (−L K − η + rv * /k) = 0, 0 is the first eigenvalue of L 0 with the corresponding eigenfunction (φ, ϕ). Moreover, all other eigenvalues of L 0 are positive and part from 0. By the perturbation theory of linear operator, we know that for the small s > 0, L(s) has a unique eigenvalue µ(s) satisfying µ(s) → 0 as s → 0 + and all other eigenvalues of L(s) have positive real parts and apart from 0. In the following, we denote L(s) = L and µ(s) = µ. Now we determine the sign of Re(µ) for small enough s > 0. Let (ξ 1 , ξ 2 ) be the corresponding eigenfunction to µ such that (ξ 1 , Multiplying the first equation of (75) by u and integrating over Ω, we get R 2n (u(x) − u(y))(ξ 1 (x) − ξ 1 (y))K(x − y)dxdy By multiplying the first equation of (11) with (u, v) = (u(s), v(s)) and integrating over Ω, we have Combining (76) and (77) yields Recall that (u, v) = (φs + O(s 2 ), v * + ϕs + O(s 2 )) and (ξ 1 , ξ 2 ) → (φ, ϕ) as s → 0 + . Taking the real part in (78), then dividing the results by s 2 and letting s → 0 + , we have where Re(µ) = 0 for s > 0 small. Since all the other eigenvalues of L has positive real parts and apart from 0, then the stability assertions follow from (79).
So, we get a contradiction. When c <c and near toc, there exists at least two positive solutions of (11).
From a global bifurcation result of Rabinowitz [5], the curve Γ of the bifurcating positive solutions is contained in a connected component S 0 of the set of positive solution of (11). Moreover, either the closure of S 0 contains another trivial solution on {(c, 0, v * ) : c > 0} or S 0 is unbounded. By Theorem 8.1 c =c is the unique bifurcation value to positive solutions of (11) from the line of trivial solutions {(c, 0, v * ) : c > 0}, so the first alternative is not possible and S 0 must be unbounded. Furthermore, 0 < u < c for λ 1 ≤ c ≤c. Finally, there is no positive solution when c ≤ λ 1 by Theorem 6.5. Thus the projection of S 0 contains an interval [c * , ∞) for some c * satisfying λ 1 < c * <c. In particular, (11) has at least one nontrivial positive solution for c ∈ [c * ,c].

9.
Conclusion. The Lévy diffusion operator is a nonlocal diffusion operator, which can stand for particles' or species' spread in the form of jump from one position to another position. However, the Laplace operator is local. The Lévy diffusion operator is a generalized form of the Laplace operator and has been successfully introduced into quantum physics and biology and so on. Furthermore, there are some phenomenons that some species exhibit Lévy-walk-like behaviours, see [21,9]. Therefore, it is very significant to introduce the Lévy diffusion operator into the ecosystem. We are first to introduce this operator into the predator-prey model, which is a partial integro-differential equations. In this paper, we first argue the basic properties of the Lévy diffusion equation. In detail, we have shown that the Lévy diffusion operator is a sectional operator, which can generate an analytically positive semigroup e L K t t≥0 ; we have given out the compare principles of the generalized Lévy-diffusion equation and the parabolic Lévy-diffusion differential equation. In help of these results above, we have succeeded to obtain the dynamical properties of the Lévy-diffusion Leslie-Gower predator-prey model with nonmonotonic functional response, such as the existence, uniqueness and attractiveness of the globally positive solution, the stability of the semi-trivial solutions, the existence and nonexistence of coexistence states, the multiplicity and stability of coexistence states. Our results enrich and extend the present research results of the predator-prey system. The Lévy diffusion of ecosystem is a new open problem. In our future work, we will show other dynamical properties of the ecosystem with Lévy diffusion, such as pattern formation.
Appendix. In this appendix, we discuss the spectrum of the Lévy-diffusion operator L K with the condition (9) and consider the following eigenvalue problem The weak formulation of (A.80) consists in the following eigenvalue problem R 2n (u(x) − u(y))(φ(x) − φ(y))K(x − y)dxdy = λ Ω u(x)φ(x)dx, ∀φ ∈ X 0 , u ∈ X 0 . (A.81) We recall that λ ∈ R is an eigenvalue of −L K provided there exists a non-trivial solution u ∈ X 0 to (A.80).

Proposition 2. [19]
The function K satisfies the assumptions (9), and then (a) the problem (A.80) admits an eigenvalue λ 1 which is positive and that can be characterized as follows (c) λ 1 is simple, that is if u ∈ X 0 is a solution of the following equation (e) for any k ∈ N there exists a function e k+1 ∈ P k+1 , which is an eigenfunction corresponding to λ k+1 , attaining the minimum in (A.87), that is e k+1 L 2 (Ω) = 1 and |e k+1 (x) − e k+1 (y)| 2 K(x − y)dxdy; (A.89) (f ) the sequence {e k } k∈N of eigenfunctions corresponding to λ k is an orthonormal basis of L 2 (Ω) and an orthogonal basis of X 0 ; (g) each eigenvalue λ k has finite multiplicity; more precisely, if λ k is such that λ k−1 < λ k = · · · = λ k+h < λ k+h+1 (A.90) for some h ∈ N 0 , then the set of all the eigenfunctions corresponding to λ k agrees with span{e k , . . . , e k+h }.