Liouville results for fully nonlinear integral elliptic equations in exterior domains

In this paper, we obtain Liouville type theorems both in the whole space and exterior domain in viscosity sense for fully nonlinear elliptic inequality involving nonlocal Pucci's operator. The nonlocal property of the operator, we only have a much weaker comparison principle, compared with the inequality with classical Pucci's operators, which give rise to the difficulties for the Hadamard type property in exterior domain.


1.
Introduction. The aim of this paper is to establish Liouville type theorems for fully nonlinear integral inequality where Ω is the whole space R N or an exterior domain of R N with N ≥ 2. The nonlinear operator M − is defined, for a regular function u, as with δ(u, x, y) = u(x + y) + u(x − y) − 2u(x), S − (t) = t + + Λt − , t + = max{0, t}, t − = min{0, t}, Λ ≥ 1, 0 < α < 1. It is known in [7] that M − , named as nonlocal Pucci's operator, is the minimum operator taken over the set L 0 , that all operator L is given as where the kernel 1 ≤ K ≤ Λ in R N . When Λ = 1, L 0 reduces into a single point set of the fractional Laplacian ∆ α . The semilinear problems involving the fractional Laplacian have been studied extensively, in the variational solutions, see [22,23,24], the nonexistence by Pohozǎev [26] and singular solutions in [10,11]. The nonexistence of positive solutions are derived by the method of moving planes [18,12,13,14]. More discussion on fractional Laplacian could refer to [6,10,11,17].
It is known that the Liouville results for positive solutions of some nonlinear elliptic equations in R N play an important role for the blowing-up technique in the study of the corresponding problems in bounded domain. Furthermore, there has been much interest in doing Liouville type theorems for kinds of different fully nonlinear elliptic equations, see the references [3,5,9,19]. In particular, [3,9] provide a complete understanding of Liouville type theorems in exterior domain, with assumption that the behavior of the nonlinear term f (x, s) is not only controlled near the point s = 0, but also near s = +∞. In the present paper, we are interested in establishing the non-existence solution for problem (1) in exterior domain. Throughout this paper, by an exterior domain we mean a connected set Ω = R N \ G, where G is a non-empty compact subset of R N . We let R 0 > 0 such that G ⊂ B R0 , where and in what follows B r = B r (0) denotes the open ball of radius r > 0 centered at the origin. The nonlocal operator M − is defined in R N , we assume that u G (x) : G → [0, +∞) is integrable and rewrite the problem (1) as It is known that u G could be omitted if the Hausdorff measure of the set G is 0. The fundamental solutions of nonlocal Pucci's operator play a crucial role in the study of Liouville type theorems for (1). Felmer and Quaas in [19] proved that has two fundamental solutions where r = |x|, σ ± = 2α − N ± , here N ± = N ± (α, Λ, N ) are dimension-like numbers such that 0 < N + ≤ N ≤ N − < N + 2α.
The existence of fundamental solutions for (4) involving more general integral operator has been studied in [20,21]. A Serrin type index N + /(N + − 2α) when N + > 2α is involved to prove the Liouville theorem of (1) when Ω = R N and f (x, u) = u p with p ≤ N + /(N + − 2α). Our aim in this article is to show Liouville Theorem for (1) with general nonlinearity f . We assume that f : Ω × [0, ∞) → [0, ∞) is a continuous function which satisfies: (f 1 ) For two positive constants c 1 and A, we have f (tx, s) ≤ c 1 t A f (x, s), for all t ≥ 1 and (x, s) ∈ B c R0 × (0, ∞). (f 2 ) lim |x|→+∞ |x| 2α f (x, s) = +∞ holds uniformly for s in any compact subset of (0, ∞). Note that (f 1 ) and (f 2 ) are the restrictions of f on the variable x. Next we give the assumptions that control the asymptotic of f near the point s = 0, but also near s = +∞. To this end, we introduce some notations. Given µ > 0, k > 0 and a > 1, we define f (x, s) s .
We further assume that f satisfies: (f 3 ) There exist constants µ > 0 and a > 1 such that, for the function h defined as one of the following conditions holds i) for all k > 0, we have h(k) = +∞ or ii) for all k > 0, we have h(k) > 0, moreover, lim k→+∞ h(k) = +∞.
Remark 1. Assumptions (f 3 ) and (f 4 ) are used to control the asymptotic behavior of f at s = 0 and at s = ∞ respectively. Assumption (f 3 ) part i) says that the non-linearity f is subcritical at s = 0, while part ii) does that the nonlinearity f is critical at s = 0. Similarly, (f 4 ) part i) means that the non-linearity f is subcritical at s = ∞ and part ii) does the nonlinearity f is critical at s = ∞.
When σ + < 0, there is no statement of f in the hypothesis (f 4 ), this is to say that we do not have to give any restriction on f at s = +∞.
It is illuminating the example of a function f with behavior as a power near s = 0 and s = ∞. Precisely, let f be defined by Assume that 0 < σ + < 2α (the exponent σ − is always negative). We define the critical numbers Then we consider µ = 1 and a = 2 in (5) and (6) and by direct computation, we find that If we assume that p < p * ,then we have that 2α + (p − 1)σ − > 0 and consequently h(k) = ∞ for all k > 0, that is, assumption (f 3 )i) holds, the subcritical case at s = 0. If p = p * , then we have that Ψ k (r) = k p−1 and then assumption (f 3 )ii) holds, the critical case at s = 0.
On the other hand, if q * < q, then we have that 2α + (q − 1)σ + > 0 and theñ h(k) = ∞ for all k > 0, thus condition (f 4 )i) holds, the subcritical case at s = ∞. If q = q * , then we haveΨ k (r) = k q−1 and then condition (f 4 )ii) holds, the critical case at s = ∞. Now we state our Liouville type theorems for inequality (1) in the whole space: Then inequality (1) in R N does not admit a non-trivial and non-negative viscosity solution.
Liouville type theorems for inequality (3) in exterior domain is stated as: (3) in exterior domain does not admit a non-trivial and non-negative viscosity solution.
Hadamard type properties are in connection with the fundamental solutions, as well as comparison principle. To apply comparison principle, we have to compare the data in the whole outside of targeted set, due to the nonlocal property of M − . Combining the fundamental solution ϕ σ − verifying that lim r→+∞ ϕ σ − (r) = 0, it gives rise to difficulties to obtain the Hadamard property in exterior domain. To overcome them, we use some truncated techniques in the proof of Theorem 1.1 and 1.2.
The rest of this paper is organized as follows. In Section §2, we review the definition of viscosity solution and global Maximum Principle, then extend Comparison Principle for unbounded functions. In Section §3, we prove the non-existence Theorem 1.1 and 1.2 in subcritical case, and the critical case is placed in Section §4.

2.
Preliminaries. In this section, we recall the notion of viscosity solution and we extend the comparison principle for M − given in [7], to super and sub-solutions, which are allowed to be unbounded. We start recalling the definition of viscosity solution.
Definition 2.1. A measurable function u : R N → R, continuous in Ω ⊂ R N , is a viscosity super-solution (sub-solution) of (1), if for any point x 0 ∈ Ω and some open bounded neighborhood V of x 0 , withV ⊂ Ω, for any ϕ ∈ C 2 (V ) such that for all x ∈ V , and defining the functioñ we have thatũ satisfies We note that Caffarelli and Silvestre in [7] introduced the viscosity super-solution (sub-solution ) with lower semi (upper) continuity. In this paper, we will use both super-and sub-solutions sometimes, so we take the continuity in viscosity solution for convenience. Now we state maximum principle which is important in our analysis.

LIOUVILLE RESULTS FOR FULLY NONLINEAR INTEGRAL ELLIPTIC EQUATIONS 89
Theorem 2.2. Assume that a function u : R N → R is a viscosity super-solution of the equation where O is a nonempty domain in R N , and that u achieves its minimum value at x 0 in the sense that u ≥ u(x 0 ) a.e. in R N . Then Proof. Since u is continuous in O and u(x 0 ) is the global minimum, so if (8) fail, it implies that the set {x ∈ R N : u(x) > u(x 0 )} has a positive measure. Then we consider the functionũ for r > 0 small enough so that {x ∈ B(x 0 , r) c :ũ(x) > u(x 0 )} has a positive measure and B(x 0 , r) ⊂ Ω. Usingũ as a test function, and noticing that δ(ũ, x 0 , y) ≥ 0 in R N with strict inequality in a set of positive measure, we see that which is impossible, since u is a super-solution.
Remark 2. An analogous result holds for sub-solutions.
Next we extend the comparison principle presented in Theorem 5.2 in [7] to possibly unbounded functions. Our result states as follows.
such that u and v are continuous inŌ. Assume further that M ≤ u ≤ v a.e. in It will be convenient for our analysis to denote by L 1 ω (R N ) the weighted L 1 space of measurable functions u defined in R N satisfying and u 1 = u − u 2 . We observe that u = u 1 , |u| ≤ m/4 inŌ, u 1 (z) = m if u(z) > 2m and u ≥ u 1 ≥ −m/4 in R N . We prove next that u 1 is a viscosity sub-solution of In order to prove this, let φ be a C 2 function on V ⊂ O, which contacts u 1 at the point x ∈ O from above and so that φ ≤ m 4 in V . Then φ also contacts u from above and we considerũ andũ 1 as in Definition 2.1. Then we haveũ =ũ 1 + u 2 , supp(u 2 ) ⊂ R N \ O and consequently δ(ũ, x, y) = δ(ũ 1 , x, y) + δ(u 2 , x, y). Moreover, we claim that δ(ũ, x, y) + = δ(ũ 1 , x, y) + + δ(u 2 , x, y), from where we obtain that Thus, u 1 is a viscosity sub-solution of (11). Now we prove (12). In caseũ(x + y) ≤ 2m andũ(x − y) ≤ 2m, we have δ(u 2 , x, y) = 0, so (12) holds. In caseũ(x + y) > 2m, we have that x + y ∈ O c and u 1 (x + y) = u 1 (x + y) = m. By definition of m, it follows that which implies (12). The caseũ(x − y) > 2m is similar.
If we define v 1 and v 2 in an analogous way, we can also prove that v 1 is a viscosity super-solution of Since v 2 ≥ u 2 in R N and v 2 = u 2 = 0 in O, we also see that Since u 2 and v 2 vanish inŌ, then ∆ α u 2 and ∆ α v 2 are continuous in O, which implies that g − ∆ α u 2 and g − ∆ α v 2 are continuous in O. Finally we see that, by definition of u 1 and v 1 , they are bounded and satisfy v 1 ≥ u 1 in O c , so that we can apply Theorem 5.2 in [7] 3. Proof of Theorem 1.1 and 1.2 in subcritical case. In this section, we prove Theorem 1.1 and 1.2 in subcritical case. The idea of the proof is to assume that (3) has a non-trivial solution u ≥ 0. The first step is to obtain the behavior of u at infinity, but using a proper test function. The second step is to obtain Hadamard type estimates for u, which are in contradiction with the former ones.
We start with some preliminaries on the fundamental solutions and related estimates, that follows from the arguments in [19]. Let and ψ σ (r) = −ϕ σ (r). Then, for x ∈ R N \ {0},
Now we present a preparatory lemma for proving various Hadamard estimates.
where a > 1, Let δ = β σ − > 1 and we consider r δ > R, then For y = r 1−δ x ∈ B ar \ B r and kr β ≤ s ≤ µ, it infers by hypothesis (f 1 ) that from where the result follows.
For r > R 0 , let where R 0 satisfies G ⊂ B R0 . The following lemma is one type of Hadamard property related to the decreasing fundamental solution.
Lemma 3.3. Suppose that u is a nonnegative solution of (3), f is nonnegative and for some c 4 > 0 independent of r. Moreover, we assume that f satisfies (f 1 −f 3 ), then there exists

HONGXIA ZHANG AND YING WANG
Proof. We prove (14). To this end, let us consider the functions and where r ≥ 8R 0 and n ≥ 2 will be chosen later. We observe that for any n ≥ 2, Moreover, we claim that there exists n 0 ≥ 2 independent of r such that Then, assuming that the claim holds at this moment and observing we use Theorem 2.3 to obtain that By taking minimum in R 0 ≤ |x| ≤ 2r, we have that which implies (14) by taking > 0 and σ − < 0. Now we prove (17). We only have to prove that for n large enough, where the last equality used the fact that δ(ϕ σ − , x, y) = δ(w n , x, y) for |x − y| ≥ r/4 and |x + y| ≥ r/4. On the one hand, for x ∈ B 4r \ B r and y ∈ B r/4 (x) \ B r/8 (x), we have and since n ≥ 2, σ − < 0, we have that On the other hand, S − (δ(ϕ σ − , x, y)) ≤ δ(ϕ σ − , x, y). Then for r < |x| < 4r, where c 5 > 0 independent of |x|, n and r.
where c 6 > 0 independent of |x|, n and r. Then there exists n 1 ≥ 2 such that Similarly, there exists n 2 ≥ 2 such that Taking n 0 = max{n 1 , n 2 }, we have that which implies (17).
In what follows, we prove (15). We first prove that
To this end, let us define the functions and where R > r 1 > 8 max{1, R 0 } will be chosen later. We claim that for r 1 large enough, We assume that (20) holds for this moment, and we see that f is nonnegative, then we obtain from (20) that which, by taking minimum in R 0 ≤ |x| ≤ r and then taking R → +∞, implies that (18) holds. Now we prove (20). In fact, for any a, b ∈ R, where S + (t) := Λt + + t − . Then for |x| ≥ 8 max{1, R 0 } and β ∈ (−N, σ − ), we have where some c 7 > 0 independent of |x|. Combining the facts that N + β > 0, which implies (20). We finally prove (15) by using (18). As estimating (20) and by using c − (σ − ) = 0, for |x| ≥ r 1 , we have that where c 8 > 0 and w σ − is defined as (19) and we see that Applying (18) into Lemma 3.2, there exists c 9 > 0 such that then by the fact of N + σ − > 0, we have that for |x| ≥ r 1 , Choosing r 1 large enough such that c 9 r which, by taking minimum in R 0 ≤ |x| ≤ r (r < R) and then taking R → +∞, implies that (15) holds. The proof ends.
We next do Hadamard type estimate related to the increasing fundamental solution, which is used in the proof of the case exterior domain for controlling the unbounded behavior at infinity of solution u of (3). (ii) Assume that σ + < 0 and u is a nonnegative solution of (3), then there exists c 11 > 0 such that Proof. We first prove the part (i). Let us define and where parameters k > 4, r > 2R 0 and σ 0 = σ + +2α 2 > σ + ≥ 0. We first claim that there is k 0 > 4 such that for any r > 0, which, assuming the claim holds for this moment, implies that Then, combining u(x) ≥ M (2r) ≥ φ k0 (|x|) for |x| ≥ 2r and u(x) ≥ 0 ≥ φ k0 (|x|) for |x| ≤ R 0 , we apply Theorem 2.3 to obtain that HONGXIA ZHANG AND YING WANG which, taking minimum in |x| ≥ r and r ≥ 2R 0 , implies that Thus, by taking c 10 In order to finish the proof of the part (i), we prove (23). By the fact that we just need to prove that there exists k 0 > 4 such that for any r > 2R 0 In fact, since M − ϕ σ0 (|x|) = c + (σ 0 )|x| σ0−2α for |x| > 0 and c + (σ 0 ) > 0, then by (21), we have where Ω 1 := {y ∈ R N : |x + y| ≥ kr, |x − y| < kr}, Ω 2 := {y ∈ R N : |x + y| < kr, |x − y| ≥ kr} and Ω 3 := {y ∈ R N : |x + y| ≥ kr, |x − y| ≥ kr}. We only compute the estimate for I 1 (x, k), for the term I 2 (x, k) and I 3 (x, k), it can be obtained in an analogous way. By σ + > 0 and σ 0 − 2α < 0, we have that for any 0 < |x| < 2r, where c 12 > 0 independent of x, r and k.
Proof. We only prove the case of σ + > 0, for σ + = 0 the proof is similar. Let us redefine and where parameters k ≥ 8 and r > 2R 0 .
We first claim that there exists k 0 ≥ 8 independent of r such that
Proof of Theorem 1.1 and 1.2 in subcritical case. We prove Theorem 1.1 under assumption (f 3 )i) and Theorem 1.2 under assumptions (f 3 )i) and (f 4 )i). We obtain the results by contradiction. Suppose that u ≥ 0 is a nontrivial solution of (3). We know that for any r > 2R 0 , m 0 (r) is positive and decreasing. In fact, if m 0 (r) = 0, then there exists x ∈ B r \ B R0 such that x attains the global minimum, i.e. u(x ) = 0, which, by Theorem 2.2, implies that u ≡ 0. So we can suppose that u(x) > 0, x ∈ Ω.
From (13) It should be paid attention that there are only Case 1 and Case 3 if Ω = R N or σ + < 0. In fact, if Ω = R N and M (r) → +∞ as r → +∞, we will get a global minimum at some point in R N , then by Theorem 2.2, u is a constant, which is impossible. For σ + < 0, M is bounded by Lemma 3.4 part (ii). Case 1. m 0 (r) → 0 as r → +∞. We can suppose that increasing sequence (R n ) n satisfies R 1 > r 1 (r 1 > 8R 0 is from Lemma 3.3), lim n→+∞ R n = +∞ and m 0 is strictly decreasing at r = R n . Then there existsx n such that |x n | = R n and u(x n ) = m 0 (R n ).
By scaling property of M − , for some c 19 > 0, we have that In addition that ξ n (x) = 0 ≤ u(x) if |x| > 2R n or |x| ≤ R n /4, as well as ξ n (x) ≤ m 0 (R n ) < m 0 (|x|) ≤ u(x) if R n /4 < |x| < R n . In particular, ξ n (x n ) = m 0 (R n ) = u(x n ). Thus there exists a global minimum of u − ξ n achieved at a point x n with R n ≤ |x n | < 2R n . Then Let ϕ(x) = ξ n (x) − ξ n (x n ) + u(x n ), then ϕ(x n ) = u(x n ) and u(x) ≥ ϕ(x) for all x ∈ R N . Letũ define as (9), we have We next claim that In fact,ũ(x) − ϕ(x) ≥ 0 for all x ∈ R N and x n is a global minimum ofũ − ϕ. Thus, M − (ũ − ϕ)(x n ) ≥ 0 and the claim follows by the fact that From (33)-(36), we obtain that Then, by Lemma 3.3 and R n ≤ |x n | < 2R n , we obtain that for any n ≥ 1 which, combining (34) and (15), we have that where k = m 0 (r 1 ) and h(k) is bounded. This is impossible with the hypothesis (f 3 )i). Case 2. M (r) → +∞ as r → +∞. We could choose (x n ) n such that lim n→+∞ |x n | = +∞, u(x n ) = M (r n ), r n = |x n | and M (r) > M (r n ), r > r n .
So let r 1 = 8R 0 and by Claim 1 we obtain R 1 such that R 1 ≥ r 1 . Inductively, we choose r n = max{2r n−1 , 2R n−1 }, and use Claim 1 to obtain R n and x n achieve the minimum of u(·) − t(R n )ξ(·, r n , R n ). Then c 17 ≤ u(x n ) ≤ 2c 18 , R n /8 < |x n | < R n and R n ≥ 2 n+2 R 0 . By the same way of (37), there exists c 22 > 0 such that it follows that which is impossible with (f 2 ). The proof ends.
4. The proof of Theorem 1.1 and 1.2 in critical case. In this section, we prove Theorem 1.1 under assumption (f 3 )ii) and Theorem 1.2 under assumption (f 3 )ii) or (f 4 )ii). Before the proof, we give some useful lemmas. Let the functions Γ − (|x|) = |x| σ − log(e + |x|) and where e is the natural number andẽ = e e .
We give the proof of Lemma 4.1 in Annex.

Lemma 4.2. Let
There exist r 2 > 8R 0 and c 25 > 0 such that Proof. By (21) and σ − > −N , we have that where c 26 > 0. By Lemma 6.1 in [19], there exists c 27 > 0 such that then we have that The proof ends.

LIOUVILLE RESULTS FOR FULLY NONLINEAR INTEGRAL ELLIPTIC EQUATIONS 103
Proof of Theorem 1.1 and 1.2 in the critical case. By contradiction, we assume that u > 0 in Ω is a solution of (3). As the analysis in the subcritical case, we only consider Case 1 for Theorem 1.1 or Case 1 and Case 2 for Theorem 1.2. Case 1. lim r→∞ m 0 (r) = 0. From (37), there exist c 28 > 0 and R n such that |x n | ∈ [R n , 2R n ), lim n→∞ R n = +∞, and u(x n ) = m 0 (|x n |).
We claim that Let us assume that (49) is true at this moment, then for every k > 0, there is for n large enough, then We conclude that h(k) ≤ c 28 for all k > 0, which contradicts (f 3 )ii). Now let us prove the claim (49). We consider the function where w is given by (47) and R >r > 2R 0 will be chosen later. By Lemma 3.3, we have that u(x) ≥ m 0 (r 1 )|x| σ − , |x| ≥ r 1 .
We see that u(x) ≥ m 0 (r) ≥ φ(|x|), R 0 ≤ |x| ≤r, ∩ Ω c r . Then we use Theorem 2.3 to obtain that By taking minimum in B r \ B R0 and R → +∞, then m 0 (r) ≥ cr σ − log(e + r), so (49) holds. Before proving Case 2, we provide an argue that plays an important role in the following analysis.
By hypothesis (f 4 )ii), let R 1 large such that for x ∈Ω R1 , we have that then together with Lemma 3.5, we have that Then for some suitable R 1 , we have that We observe that by (56), We use Theorem 2.3 to obtain that which implies that ). Then If we choose R 1 > R 0 such that which, by Lemma 3.5, implies that Then for some suitable R 1 , we have that We observe that by (60), We apply Theorem 2.3 to obtain that which implies that ]. Then If we choose R 1 > R 0 such that 2c 2 14 log R1 < 1/4, then by Lemma 3.5, we have that (38) and (53), which implies a contradiction from Argue 1. We complete the proof.

5.
Annex. In this appendix, we give the proof of Lemma 4.1.
We prove this claim by 3 steps.
We conclude the result in Lemma 4.1(ii) from Step 4 to Step 6.