An instability theorem for nonlinear fractional differential systems

In this paper, we give a criterion on instability of an equilibrium of nonlinear Caputo fractional differential systems. More precisely, we prove that if the spectrum of the linearization has at least one eigenvalue in the sector $$\left\{\lambda\in\C\setminus\{0\}:|\arg{(\lambda)}|<\frac{\alpha \pi}{2}\right\},$$ where $\alpha\in (0,1)$ is the order of the fractional differential systems, then the equilibrium of the nonlinear systems is unstable.


Introduction
In recent years, fractional differential equations have attracted increasing interest due to their many applications in various fields of science and engineering, see e.g., [OS73,SKM93]. One of the fundamental problems of the qualitative theory of fractional differential equations is stability theory. So far, there have been a number of publications on stability theory for different types of fractional systems, e.g., linear fractional differential equations [M96,BRT07,CKN13,KSZ15], linear fractional difference equations [AA13,CGN15] and nonlinear fractional differential equations [AEE07,De10,AOH15,CDST16a].
In this paper, we are interested in stability of the trivial solution of a nonlinear Caputo fractional differential system of order α, 0 < α < 1, where t ≥ 0, x ∈ R d , A ∈ R d×d and f : R d → R d is continuous on R d and Lipschitz continuous in a neighborhood of the origin, f (0) = 0 and lim r→0 ℓ f (r) = 0, where with B R d (0, r) := x ∈ R d : x ≤ r . Similarly as for ordinary differential equations, we would expect for fractional differential equations that if the linear system is asymptotically stable (or unstable) then the trivial solution of the perturbed system (1) is also asymptotically stable (or unstable, respectively), since these conclusions hold in the theory of ordinary differential equations, see e.g., [CL55,Chapter 13]. In [CDST16a], we give an affirmative answer in the case when the linear system (3) is asymptotically stable by proving that the trivial solution of system (1) is then also asymptotically stable.
However, in the case that system (3) is unstable, e.g., if A has at least one eigenvalue λ with its argument 1 satisfying that | arg(λ)| < απ 2 , the question whether the trivial solution of (1) is unstable still remains open. Notice that in the scalar case, this question is part of a conjecture of J. Audounet, D. Matignon and G. Montseny [AMM01, Theorem 3, p. 81] which is stated as follows.
Conjecture. The local stability of the equilibrium x * = 0 of the nonlinear fractional differential system C D α 0+ x(t) = f (x(t)) is governed by the global stability of the linearized system near the equilibrium C D α 0+ x(t) = λx(t), where λ = f ′ (0) ∈ C, namely: In this paper, we establish an instability theorem for the nonlinear Caputo fractional differential system (1) when the linearization (3) is unstable. As a consequence, we also prove statement (ii) of the conjecture above. The main ingredient in the proof is to construct a suitable Lyapunov-Perron operator and to show that a bounded solution of (1) must be a fixed point of this operator. By constructing a suitable initial value for which the associated Lyapunov-Perron operator has no nontrivial fixed point, we show the existence of an unbounded solution which leads to the instability of the system.
We note that [LM13] claimed that they proved the conjecture above, but their paper contains serious flaws which make their construction as well as their proof incorrect; a discussion about their paper will be given in Remark 10 below.
The paper is organized as follows. Section 2 is a preparatory section where we present some basic notions from fractional calculus and give some basic properties of Mittag-Leffler functions. Section 3 is devoted to the main result of the paper in which we prove a theorem on instability of the trivial solution of the nonlinear Caputo fractional differential system (1).
To conclude the introductory section, we fix some notation which will be used later. Let R ≥0 denote the set of all nonnegative real numbers. For α ∈ (0, 1), we define Let (X, · ) be a Banach space. Denote by C(R ≥0 ; X) the linear space of all continuous functions ξ : R ≥0 → X, and by C ∞ (R ≥0 ; X) the linear space of all continuous functions ξ : R ≥0 → X such that Clearly, (C ∞ (R ≥0 ; X), · ∞ ) is a Banach space.

Fractional differential equations
We briefly recall an abstract framework of fractional calculus. Let The Riemann-Liouville integral operator of order α is defined by where D = d dt is the usual derivative and m := ⌈α⌉ is the smallest integer larger or equal to α.
The Caputo fractional derivative of a d-dimensional vector function x(t) = (x 1 (t), . . . , x d (t)) T is defined component-wise as We now recall the notions of stability of the trivial solution of the fractional differential equation (1), see also [Di10,Definition 7.2,p. 157]. Note that since f is locally Lipschitz continuous, for any initial value x 0 ∈ R d in a neighborhood of 0, the equation (1) has a unique solution, which we denote by ϕ(·, x 0 ), with its maximal interval of existence I = [0, t max (x 0 )), 0 < t max (x 0 ) ≤ ∞.
Definition 1. The trivial solution of (1) is called stable if for any ε > 0 there exists δ = δ(ε) > 0 such that for every The trivial solution is called unstable if it is not stable.

Mittag-Leffler function
The Mittag-Leffler function is a generalization of the exponential function. Like the exponential function plays a very important role in the theory of ordinary differential equation, the Mittag-Leffler function is at the heart of the theory of fractional differential equations (see, e.g., [Po99]). The Mittag-Leffler function is defined by the formula We present in this subsection some basic properties of Mittag-Leffler functions. These results are slight refinements of known results in the theory of Mittag-Leffler functions which are adapted to our case. To derive these estimates one uses the integral representation of Mittag-Leffler functions, see e.g., [Po99]; we give only a sketch of the proof.
Lemma 2. Let λ ∈ Λ u α , where Λ u α is defined in (4). There exist a real number t 0 > 0 and a positive constant m(α, λ) such that the following estimates hold: Lemma 3. Let λ ∈ Λ u α , where Λ u α is defined in (4). There exists a positive constant K(α, λ) such that the following estimates hold: for all t ≥ 0 and any function g ∈ C ∞ (R ≥0 ; C).
Proof. The proof of this lemma follows easily by using Lemma 2 and repeating arguments used in the proofs of Lemma 5 and Lemma 6 in [CDST14].
Lemma 4. For any function g ∈ C ∞ (R ≥0 ; C) and λ ∈ Λ u α , we have the following limiting relations: Proof. Use Lemma 2, Lemma 3, and arguments similar to that of the proof of Lemma 8 in [CDST14].

Instability of Fractional Differential Equations
We now state the main result of this paper about a criterion of instability of fractional differential equation.
Then, the trivial solution of (6) is unstable.
For a proof of this theorem we follow the approach of [CDST16a,CDST16b]. Namely, first we transform the linear part to a simple form; then construct an appropriate Lyapunov-Perron operator which is a contraction and its fixed point is a solution of (6), and exploit the property of the Lyapunov-Perron operator to derive the conclusion of the theorem. To do this we need some preparatory steps, the details of which we now present.

Transformation of the linear part
Let σ(A) := {λ 1 , . . . ,λ m } denote the spectrum of A, i.e.,λ 1 , . . . ,λ m ∈ C is the collection of all the distinct eigenvalues of A. Let T ∈ C d×d be a nonsingular matrix transforming A into its Jordan normal form, i.e., where for i = 1, . . . , n the block A i is of the following form , and the nilpotent matrix N d i ×d i is given by Note that with this transformation we leave the field of real numbers and consider differential equations in the complex numbers. Only if all eigenvalues of A are real, we remain in R. For a general real-valued matrix A we may simply embed R into C, consider A as a complex-valued matrix and thus get the above Jordan form for A. Alternatively, we may use a real-valued Jordan form (see [LT85, Chapter 6, p. 243]; for a discussion on similar issues for FDE see also [Di10,). For simplicity we use the embedding method and omit the discussion on how to return back to the field of real numbers. Note also that those techniques are well known in the theory of ordinary differential equations.
Let γ be an arbitrary but fixed positive number. Using the transformation P i := diag(1, γ, . . . , γ d i −1 ), we obtain that , γ}. Put P := diag(P 1 , . . . , P n ), then under the transformation y := (T P ) −1 x system (6) becomes (denoting the solution again by x instead of y) where J := diag(J 1 , . . . , J n ), J i := λ i id d i ×d i for i = 1, . . . , n and the function h is given by Note that Since the spectrum of A satisfies the instability condition we can find at least one eigenvalueλ i ∈ Λ u α . Without loss of generality, we can assume Consequently, for simplicity of notation we can write (9) in the form where h is defined by (10) and µ i ∈ σ(A) = {λ 1 , . . . ,λ m }, i = 1, . . . , d, Remark 6. Since the transforming matrix T P is constant, the type of stability of the trivial solution of equations (6) and (12) are the same, i.e., they are either both stable or both unstable.

Construction of an appropriate Lyapunov-Perron operator
We define a specific Lyapunov-Perron operator associated with the equation (12) as follows. For any ξ ∈ C ∞ (R ≥0 ; C d ), the Lyapunov-Perron operator where for i = 1, . . . , k we set and for i = k + 1, . . . , d we set here h(x) = (h 1 (x), . . . , h d (x)) T is the coordinate representation of the vector h(x). Note that under an additional assumption that µ i = 0 and |arg(µ i )| = απ 2 for i = 1, . . . , d, it is proved in [CDST16b] that there exists a neighborhood of the trivial function in C ∞ (R ≥0 ; C d ) equipped with the sup norm such that the operator T is a contraction on this neighborhood. In this paper, it is only assumed that |arg(µ i )| < απ 2 for i = 1, . . . , k and for i = k + 1, . . . , d either µ i = 0 or |arg(µ i )| ≥ απ 2 . Therefore, the operator T is in general not a contraction with respect to the sup norm. To see this, consider the case that d = 2, k = 1, µ 2 = 0, h(x) := (0, x 2 2 ) T , then the operator T defined by (14) has its second coordinate given by hence, obviously, T as well as any of its restrictions to small balls around the origin in C ∞ (R ≥0 ; C 2 ) is not a contraction.
In what follows, by introducing a suitable weighted norm · w on C ∞ (R ≥0 ; C d ), we show that the Lyapunov-Perron operator T is contractive on a neighborhood of the trivial function.
For i = 1, . . . , k, we write µ i in the form: where ι denotes the imaginary unit, r i > 0 is the modulus and − απ 2 < ϕ i < απ 2 is the argument of the complex number µ i ∈ Λ u α . Let and define a weighted norm-like function on C(R ≥0 ; C d ) by .
It is easily seen that · w has the three properties of a norm but it might take the value +∞ on the space C(R ≥0 ; C d ). It is a norm on the space From (10) and (11) it follows that h is Lipschitz continuous on a neighborhood of the origin. Hence, there exists ε > 0 such that ℓ h (r) < ∞ for all 0 < r < ε, where ℓ h (r) is defined according to (2). Using the notation the following proposition gives an estimate for the operator T .
From now on, we choose and fix the parameter γ > 0 in the definition of the transformation P above such that γ < 1 2K(α,A) . By this choice of γ the systems (9) and (12) are completely specified. Due to (11), there exists a positive constant ε such that 0 < ε < ε, and By using (21) and Proposition 7, we obtain immediately the following property of T .
Remark 9. (i) We get the second part of the conjecture [AMM01, Theorem 3, p. 81] formulated in the beginning of this paper as a special case of Theorem 5 when d = 1.
(ii) Note that if the matrix A is hyperbolic then the stability of the perturbed system (6) can be determined by using the result of [CDST16b]. If A is not hyperbolic, i.e. A has an eigenvalue λ such that λ = 0 or | arg(λ)| = πα 2 , then our Theorem 5 show the instability of (6) provided that σ(A) ∩ Λ u α = ∅. Otherwise, in case A is non-hyperbolic and σ(A) ∩ Λ u α = ∅ the stability type of the trivial solution of (6) is undecided. For an example, we consider the following one-dimensional equation The trivial solution of (22) is stable. However, if we add the perturbation x 2 (t) to this equation, then the trivial solution of the perturbed equation is unstable, see [LM13, Lemma 2, p. 629]. On the other hand, for the perturbation −x 3 (t), the trivial solution of the perturbed equation is asymptotically stable, see [SL14,p. 550].
Remark 10 (Discussion about the paper [LM13] by Li and Ma). As mentioned in the Introduction, Li and Ma claimed in [LM13] that they proved the Audounet-Matignon-Montseny conjecture as a consequence of their Theorem 3 in [LM13]. However, their paper contains flaws which make their proof incorrect. More precisely, Theorem 2 in [LM13] concerning a construction of a fractional flow in the Caputo sense is false. For a simple counterexample consider the one-dimensional fractional system C D α 0+ x(t) = x(t), α ∈ (0, 1), x(0) = x 0 , which can be solved explicitly and the solution is x(t) = x 0 E α (t α ) (see [Di10,Theorem 7.2, p. 135]). For this system, using the notation of [LM13], for s = 1 and t > 0 we have On the other hand, ϕ t+1 (x 0 ) = x 0 E α (t + 1) α .
However this assertion is false, because by using the asymptotic behavior of the function E α (·) (see [Po99, Theorem 1.3] and Lemma 2 above), for t > 0 big enough one easily gets Thus, Theorem 2 of [LM13] is false. The construction of a fractional flow in the Caputo sense in [LM13] is therefore invalid. Moreover, the proof of Theorem 3 of [LM13] which relies on Theorem 2 of [LM13] is incorrect.