SYMMETRY AND MONOTONICITY OF SOLUTIONS FOR THE FULLY NONLINEAR NONLOCAL EQUATION

. In this paper, we consider equations involving the fully nonlinear fractional order operator with homogeneous Dirichlet condition: (cid:40) where Ω is a domain(bounded or unbounded) in R n which is convex in x 1 − direction. By using some ideas of maximum principle, we prove that the solution is strictly increasing in x 1 − direction in the left half of Ω. Symmetry of solution is also proved. Meanwhile we obtain a Liouville type theorem on the half space R n + .


1.
Introduction. This paper is mainly devoted to investigate the symmetry and monotonicity properties for the solution of nonlinear equation involving fully nonlinear non-local operator F a , which is defined as F α (u)(x) = C n,α P.V.
where P.V. stands for the Cauchy principle value, C n,α > 0 and 0 < α < 2. This kind of operator was introduced by Caffarelli and Silvestre in [3]. In order to make sense for (1) we require that G being at least local Lipschtiz continuous and G(0) = 0, while u be a Schwartz function initially. One can extend this operator to more wider spaces of functions. In this paper, we consider u ∈ C 1,1 loc ∩ L α (R n ), where L α (R n ) = {u ∈ L 1 loc | R n |u(x)| 1 + |x| n+α dx < ∞}.
with f (·) being Lipschitz continuous. Then u must be radially symmetric and monotone decreasing about the origin. (b) Suppose that u ∈ C 1,1 loc ∩ L α is a positive of F α (u(x)) = g(u(x)), x ∈ R n .
On the other hand, Cheng-Huang-Li [9] considered the zero-Dirichlet problem with more generalized nonlinear term, In (3), Ω has been considered as a convex domain in x 1 -direction (bounded or unbounded) in R n ( we say a convex domain Ω is convex in x 1 -direction if and only if ( ∈ Ω for any t ∈ (0, 1)). Moreover, the nonlinear term f belongs to the function space , where is defined as the collections of functions f (x, u, p) : R n ×R×R n → R such that for any M > 0, Under some other adaptable condition on u and f , Cheng-Huang-Li [9] obtained the solution u for (3) is monotone and symmetric when Ω bounded or unbounded respectively. In this paper we are interested in the following problem, which can be regarded as the problem (3) while (−∆) α 2 be replaced by F α . More precisely, we consider Unlike the operator (−∆) α 2 is linear, while the operator F α is nonlinear, some technique in [9], for example the key lemma (Lemma 1 in page 4 [9]) can not been established, and can not be used in here. The idea to overcome the difficulty of nonlinearity of F α was coming from [5] and [19]. Our results are listed in the following three theorems, where the domain Ω be considered as the bounded domain, unbounded domain and the half space of R n , respectively.
Then u(x 1 , x ) is strictly increasing in the left half of Ω in x 1 −direction and Moreover, if f (x 1 , x , v, p 1 , p 2 , · · ·, p n ) = f (−x 1 , x , v, −p 1 , p 2 , · · ·, p n ), then To state the second theorem in unbounded domain Ω, we must impose a growth condition on f (x, v, p): Let Ω be an unbounded domain in R n , which is convex in (6) and u(x) has the following asymptotic: then there exists µ 0 ≤ 0 such that u( where Now we consider the follow problem, We have the following theorem. Theorem 1.4. We suppose that u ∈ C(R n + ) C 1,1 loc (R n + ) is a nonnegative solution of (9) and f (x, v, p) ∈ satisfies: and This note is organized as follows. In Section 2 and 3, we will prove Theorem 1.2 and Theorem 1.3 respectively. While in section 4, we will give the proof of Theorem 1.4.

2.
Proof of Theorem 1.2. Let T λ be a hyperplane in R n , without loss of generality, we assume that: We carry out the proof of Theorem 1.2 in two steps. Firstly, we show that for λ > −1 and sufficiently close to −1, we have Next we move the plane T λ along the x 1 −axis to the right in the left half of Ω as long as inequality (12) holds. The plane will just stop at the limiting position λ = 0.
Since Ω is bounded and convex in x 1 -direction, without loss of generality, we may assume Ω ⊂ {|x 1 | ≤ 1} and ∂Ω ∩ {x 1 = −1} = ∅. We claim that there exists δ > 0 small enough such that Suppose not, we set and Moreover with (a), we get Then using the property (5) of f (x, u, p) and the condition (14)- (16), we have that Here we set Since u(x) ∈ C(R n ) with compact support and f (x, u, p) ∈ , then c(x) is uniformly bounded. On the other hand, by the definition of F α , To estimate I 1 , we have that While for second part of I 1 , we have for y ∈ Σ λ0 , for y ∈ Σ λ0 , and then To estimate I 2 , we use the fact Then using (2) and the fact ω λ0 (x 0 ) < 0, we have Combining (18)-(20), we deduce and (x 0 ) n denotes for the last coordinate of x 0 . It follows that B δ (x * 0 ) ⊂ Σ c λ0 and for any y ∈ B δ (x * 0 ), |y − x 0 | ≤ 4δ. Then where C is a positive constant only relies on n and α. Thus, then combining (17) with (21), we have This yields a contradiction and proves claim (13).
We claim that there exists > 0 small enough such that Now we prove (24) is true. Since we have (23) and ω λ (x) is lower semi-continuous in Ω. Thus ∀ δ > 0, By the continuity of ω λ with respect to λ, there exist > 0, such that Suppose (24) is not true, then we have ∀ > 0, Since Ω is a bounded domain, then by (25), A can be obtained for some point Obviously we have ω µ ≥ 0 on ∂(Σ µ \Σ λ0−δ ∩ Ω), thus x 0 ∈ Σ µ \Σ λ0−δ ∩ Ω, then going through the similar proof in Step 1 (17)-(22), we get if we choose δ, small enough. This yields a contradiction and proves the claim (24). This contradicts the definition of λ 0 . Therefore, we must have λ 0 = 0. It follows that 3. Proof of Theorem 1.3. We carry out the proof in two steps, first, to begin with, we show that for λ sufficiently negative, we have Then we move the plane T λ along the x 1 −axis to the right in the left half of Ω as long as inequality (26) holds. The plane will eventually stop at some limiting position at λ = λ 0 < 0 or λ = 0.
Step 1. Start moving the plane T λ along the x 1 -axis from near −∞ to the right. We claim: there exists R 0 > 0 large enough such that Suppose not, so there exist λ k → −∞ such that Now on the one hand, the same as (18), we write F α (u µ k (x k )) − F α (u(x k )) = I 1 + I 2 . The fact we have I 1 ≤ 0, then going through the similar proof in (17)-(22), we get To get the last inequality, we notice the fact that 0 < u µ k (x k ) < u(x k ). For x k ∈ Σ µ k and |x k | sufficiently large, let Σ c µ k = R n \Σ µ k , choose a point in Σ c µ k : there exists a C > 0 such that On the other hand, from (17), we have Then from (7) and (28)-(29), we can get since |x k | ≥ |λ k | → ∞. This yields a contradiction and ends the proof of Step 1.
4. Proof of Theorem 1.4. First, we claim that To proof (33), we assume that there exist x 0 ∈ R n + such that u(x 0 ) = 0, then by (11), we have that On the other hand, using the condition (2), we have where the last inequality is obtained by u(y) ≥ 0 and u(y) ≡ 0 with u ∈ C(R n + ). This yields a contradiction, and so (33) holds. Now we assume that We carry on the method of moving planes on the solution u along x n -direction. Let T λ be a hyperplane in R n with be the reflection of x about the plane T λ . Set Step 1. We prove the claim that there exists δ > 0 small enough such that Suppose the claim is not true, then we have for any δ > 0 (we can restrict δ ∈ (0, 1) for simplicity), First we know that ω λ (x) = u λ (x) − u(x) ≥ −u(x) uniformly with respect to λ, so by the condition (10), we have uniformly with respect to λ. Then there must exist R 0 > 0 large enough such that ω λ (x) ≥ A 2 , x ∈ B c R0 (0) uniformly with respect to λ. So we have A in (36) can be obtained for some point Noticing that u ≡ 0 in R n − and u ≥ 0 in R n and ω λ0 ( On the other hand it is easy to see that λ 0 = 0 and λ 0 ∈ (0, δ]. Since (λ 0 , x 0 ) is a minimizing point, we can get: 0 ) = 0. Then going though the similar proof in Theorem 2. 1 Step 1 (17)-(21), we have that This yields a contradiction and proves claim (35).
Then going through the similar proof in Theorem 2.1 (17)-(22), one can see that if we choose δ, small enough. This yields a contradiction and proves claim (41).
This contradict with the assume (34). Therefore we proved the claim (38): λ 0 = ∞ and consequently the solution u(x) is monotone increasing with respect to x n . The condition (10): lim |x|→∞ u(x) = 0 gives us the claim (38) is not true and then u(x) ≡ 0, x ∈ R n + .