The analytical aspect to the linear stability of elliptic equilibrium points of the Robe's restricted three-body problem

We study the Robe's restricted three-body problem. Such a motion was firstly studied by A. G. Robe in \cite{Robe}, which is used to model small oscillations of the earth's inner core taking into account the moon attraction. For the linear stability of elliptic equilibrium points of the Robe's restricted three-body problem, earlier results of such linear stability problem depend on a lot of numerical computations, while we give an analytic approach to it. The linearized Hamiltonian system near the elliptic relative equilibrium point in our problem coincides with the linearized system near the Euler elliptic relative equilibria in the classical three-body problem except for the rang of the mass parameter. We first establish some relations from the linear stability problem to symplectic paths and its corresponding linear operators. Then using the Maslov-type $\omega$-index theory of symplectic paths and the theory of linear operators, we compute $\omega$-indices and obtain certain properties of the linear stability of elliptic equilibrium points of the Robe's restricted three-body problem.


Introduction and main results
A new kind of restricted three-body problem that incorporates the effect of buoyancy forces was introduced by Robe in 1977. In [11], he regarded one of the primaries as a rigid spherical shell m 1 filled with a homogenous incompressible fluid of density ρ 1 . The second primary is a mass point m 2 outside the shell and the third body m 3 is a small solid sphere of density ρ 3 , inside the shell, with the assumption that the mass and radius of m 3 are infinitesimal. He has shown the existence of an equilibrium point with m 3 at the center of the shell, where m 2 describes a Keplerian orbit around it, see Figure 1.
Further, he discussed two cases of the linear stability of the equilibrium points of such restricted threebody problem. In the first case, the orbit of m 2 around m 1 is circular and in the second case, the orbit is elliptic, but the shell is empty (that is no fluid inside it) or the densities of m 1 and m 3 are equal. In the second case, we use "elliptic equilibrium point" to call the equilibrium point. In each case, the domain of stability has been investigated for the whole range of parameters occurring in the problem. of fluid density ρ 1 .
Let the orbital plane of m 2 around m * 1 (that is the shell with its fluid) be taken as the x − y plane and let the origin of the coordinate system be at the center of the mass, O, of the two primaries. The equation of motion of m 3 isR where R 3 = OM 3 and R i j = M i M j . After a detailed calculations, Robe obtained the equations of the motion: where θ is the true anomaly in the two-body problem m * 1 and m 2 , and V is given by x 1 and x 2 being the x coordinates of M 1 and M 2 : In [11], H. Robe firstly studied the equilibrium points of the problem. He obtained two kind of equilibrium points, one is the circular case, and the other is the elliptic case under K = 0. He also studied the linear stability of the above two kinds of equilibrium points. But for the elliptic equilibrium points, only numerical results was obtained. Later on, in [1], P. P. Hallen and N. Rana studied the existence of all the equilibrium points in the Robe's restricted three-body problem. They found that, in the case of equilibrium points with circular, there are serval different situations depending on K, and the linear stability of such equilibrium points was carefully studied. More details can be seen in [1].

22)
and Then Γ l , Γ m and Γ r from three curves which possess the following properties.

ω-indices on the boundary segments {0} × [0, 1) and {1} × [0, 1)
When µ = 0, from (2.34), we have this is just the same case which has been discussed in Section 4.1 of [15]. Using Lemma 4.1 of [15], A(0, e) is non-negative definite for the ω = 1 boundary condition, and A(0, e) is positive definite for the ω ∈ U\1 boundary condition. Hence we have This is just the case which has been discussed in Section 3.1 of [2]. We just cite the results here:
Note that A(µ, e) is the same operator of A(β, e) when β = µ − 1 in [15] with different parameter ranges, by Lemma 4.2 in [15] and modifying its proof to the different range of parameters, we get the following important lemma: Consequently we arrive at Corollary 4.2 For every fixed e ∈ [0, 1) and ω ∈ U, the index function φ ω (A(µ, e)), and consequently i ω (γ µ,e ), is non-decreasing as µ increases from 0 to 1. When ω = 1, these index functions are constantly equal to 0, and when ω ∈ U \ {1}, they are increasing and tends from 0 to 2.
Then under a similar steps to those of Lemma 6.2 and Theorem 6.3 in in [2], we can prove the theorem. where µ i (e, −1) are the two −1-dgenerate curves as in Theorem 4.3. By (3.28), −1 is a double eigenvalue of the matrix γ µ * ,e (2π), then the two curves bifurcation out from (µ * , 0) when e > 0 is small enough.
If Case (A) happens, for this µ 0 , by a similar argument of Theorem 1.7 in [2], there exists e 0 > 0 sufficiently close to 1 such that γ µ,e (2π) is hyperbolic for all (µ, e) in the region (0, µ l (ê i )] × [e 0 , 1). Then by the monotonicity of Case (A) we obtain (4.33) Therefore (µ l (ê i+m ),ê i+m ) will get into this region for sufficiently large m ≥ 1, which contract to the definition of µ l (ê i+m ). If Case (B) happens, the proof is similar. Thus (v) holds.
Following Definition 1.8.9 on p.41 of [9], we call the above matrices D(λ), R(θ), N 1 (λ, a) and N 2 (ω, b) basic normal forms of symplectic matrices. As proved in [7] and [8] (cf. Theorem 1.9.3 on p.46 of [9]), every M ∈ Sp(2n) has its basic normal form decomposition in Ω 0 (M) as a ⋄-sum of these basic normal forms. This is very important when we derive basic normal forms for γ β,e (2π) to compute the ω-index i ω (γ β,e ) of the path γ β,e later in this paper.
We refer to [9] for more details on this index theory of symplectic matrix paths and periodic solutions of Hamiltonian system.
In general, for a self-adjoint operator A on the Hilbert space H , we set ν(A) = dim ker(A) and denote by φ(A) its Morse index which is the maximum dimension of the negative definite subspace of the symmetric form A·, · . Note that the Morse index of A is equal to the total multiplicity of the negative eigenvalues of A.