RICCATI EQUATIONS FOR LINEAR HAMILTONIAN SYSTEMS WITHOUT CONTROLLABILITY CONDITION

. In this paper we develop new theory of Riccati matrix diﬀerential equations for linear Hamiltonian systems, which do not require any control- lability assumption. When the system is nonoscillatory, it is known from our previous work that conjoined bases of the system with eventually the same im- age form a special structure called a genus. We show that for every such a genus there is an associated Riccati equation. We study the properties of symmetric solutions of these Riccati equations and their connection with conjoined bases of the system. For a given genus, we pay a special attention to distinguished solutions at inﬁnity of the associated Riccati equation and their relationship with the principal solutions at inﬁnity of the system in the considered genus. We show the uniqueness of the distinguished solution at inﬁnity of the Riccati equation corresponding to the minimal genus. This study essentially extends and completes the work of W. T. Reid (1964, 1972), W. A. Coppel (1971), P. Hartman (1964), W. Kratz (1995), and other authors who considered the Riccati equation and its distinguished solution at inﬁnity for invertible conjoined bases, i.e., for the maximal genus in our setting.

In [20], Reid showed that when system (H) is completely controllable and nonoscillatory, the Riccati equation (R) has the so-called distinguished solutionQ(t) at infinity, which is the smallest symmetric solution of (R) existing on an interval [α, ∞) for some α ≥ a. In the subsequent paper [21], Reid derived the minimality of the distinguished solution of (R) at infinity also for a noncontrollable system (H) by considering invertible principal solutions (X,Û ) of (H) at infinity. Recently, the author andŠimon Hilscher developed the theory of principal solutions at infinity for a general nonoscillatory and possibly abnormal system (H). We showed in [28,29] the existence of principal solutions (X,Û ) at infinity with all ranks ofX(t) in a specific range depending on the maximal order of abnormality d ∞ of (H), their classification and limit properties with antiprincipal solutions at infinity [30], and the geometric structure of the set of all conjoined bases [31]. In particular, conjoined bases of (H) with eventually the same image of the first component form a genus G, which can be represented by an orthogonal projector R G (t) satisfying the Riccati type matrix differential equation This leads under (1.1) to a complete description of the set Γ of all genera of conjoined bases of a nonoscillatory system (H), being a complete lattice [31,Theorem 4.8].
This result was recently extended to a possibly oscillatory system (H) in [27, Theorem 4.14].
In this paper we continue in the above study of linear Hamiltonian system (H) by developing the corresponding theory of Riccati matrix differential equations. The presented approach and results are novel in three directions: (i) we do not require any controllability assumption on system (H), (ii) for every genus G we associate a Riccati equation where the coefficients A(t), B(t), and C(t) are given by (iii) we show that every such a Riccati equation (R) possesses a distinguished solution at infinity (defined in a suitable way), which corresponds to a principal solution of (H) at infinity from the genus G.
More precisely, given a genus G of conjoined bases of (H), we show (Theorems 4.18 and 4.21) a fundamental connection between the symmetric solutions Q(t) of (R) on [α, ∞) with some α ≥ a satisfying Im Q(t) ⊆ Im R G (t), t ∈ [α, ∞), (1.4) and the conjoined bases (X, U ) of (H) with constant kernel on [α, ∞), which belong to G. We define (Definition 7.1) a distinguished solutionQ(t) at infinity for each Riccati equation (R), which corresponds to a principal solution (X,Û ) of (H) at infinity in the genus G. We also prove (Theorem 7.16) that for every symmetric solution Q(t) of (R) on [α, ∞) with (1.4) there exists a distinguished solutionQ(t) of (R) satisfying the inequality Q(t) ≥Q(t) on [α, ∞). (1.5) The above results are particularly important for the minimal genus G = G min , which is formed by the conjoined bases (X, U ) of (H) with minimal possible rank of the matrix X(t), i.e., with rank X(t) = n − d ∞ on [α, ∞). In this case the associated distinguished solutionQ min (t) at infinity is unique and minimal among all symmetric solutions Q(t) of (R) satisfying (1.4). This latter situation generalizes the classical controllable results of Reid and Coppel [7,20,22], since in this case d ∞ = 0 and the orthogonal projector R G (t) ≡ I on [a, ∞), so that the Riccati equation (R) reduces to (R). We note that the original results by Reid [21,23] for noncontrollable system (H) and Riccati equation (R) correspond in our new theory to the maximal genus G = G max of conjoined bases (X, U ) with eventually invertible matrix X(t), i.e., to R G (t) ≡ I on [a, ∞). Therefore, the present study can be regarded as a generalization and completion of the theory of the Riccati equations (R) for completely controllable systems (H) using the minimal genus G = G min , as well as the noncontrollable systems (H) using the maximal genus G = G max .
Among other new results in this paper (Theorem 6.3 and Corollary 6.4) we mention a connection of the symmetric solutions Q(t) of (R) with the implicit Riccati equation (1.6) Such implicit Riccati equations occur in the study of nonnegative quadratic functional associated with system (H), see [13,Section 6].
The study of the Riccati equations in the context of the present paper is also motivated by several situations in the literature, which are equivalent to using the Riccati matrix differential equation for an uncontrollable linear Hamiltonian system. For example, in [35, pg. 886], [1, pp. 621-622], [11,Sections 4 and 6], and [12, pp. 17-18] the authors use a cascade system of three differential equations for the investigation of calculus of variations or optimal control problems with variable endpoints -the Riccati equation (R), a linear differential equation, and an integrator. These three differential equations are together equivalent to a Riccati equation in dimension 2n, which corresponds to an uncontrollable system (H) in dimension 4n. This connection is discusses in details in [11,Remark 6.3].
The results of this paper open new directions in the theory of Riccati matrix differential equations associated with general uncontrollable linear Hamiltonian systems. They demonstrate that, as in the completely controllable case, distinguished solutions at infinity play a prominent role in the structure of the space of symmetric solutions of (R). Moreover, the intimate connection with the principal solutions of (H) at infinity points to effective applications of the distinguished solutions of (R) at infinity in other fields of mathematics and engineering.
The paper is organized as follows. In Section 2 we display the notation and preliminary results about system (H) and its solutions. In Section 3 we present properties of principal solutions of (H) at infinity and recall the concept of a genus of conjoined bases of (H). In Section 4 we develop the theory of Riccati differential equations for a given genus G. In Section 5 we study inequalities for Riccati type quotients associated with the Riccati equation (R). In Section 6 we analyze the relationship between the two Riccati equations (R) and (1.6). In Section 7 we define the notion of a distinguished solution of (R) at infinity and study its minimality properties. Finally, in Section 8 we provide examples illustrating our new theory.
2. Preliminaries about linear Hamiltonian systems. In this section we review some recent results about linear Hamiltonian systems (H) from [18,33,28,29,30,31]. For a general theory of these systems we refer to [7,17,22]. By a matrix solution of (H) we mean a pair of functions (X, U ) such that X, U : [a, ∞) → R n×n are piecewise continuously differentiable (C 1 p ) and satisfy system (H) on [a, ∞). In order to shorten the notation and the calculations, we sometimes suppress the argument t in the solutions. For any two matrix solutions (X 1 , U 1 ) and (X 2 , U 2 ) of (H) their Wronskian X T 1 U 2 − U T 1 X 2 is constant on [a, ∞). A solution (X, U ) of (H) is called a conjoined basis if rank (X T (t), U T (t)) T = n and X T (t) U (t) is symmetric at some and hence at any t ∈ [a, ∞). The principal solution (X α ,Û α ) at the point α ∈ [a, ∞) is defined by the initial conditionsX α (α) = 0 andÛ α (α) = I. By [17,Corollary 3.3.9], a given conjoined basis (X, U ) can be completed to a fundamental system of (H) by another conjoined basis (X,Ū ). In addition, the conjoined basis (X,Ū ) can be chosen so that (X, U ) and (X,Ū ) are normalized, i.e., we have X TŪ − U TX = I. (2.1) The oscillation of conjoined bases of (H) is defined via the concept of proper focal points, see [36,Definition 1.1]. However, this concept will not be explicitly needed in this paper. By [33, Definition 2.1], a conjoined basis (X, U ) of (H) is called nonoscillatory if there exists α ∈ [a, ∞) such that Ker X(t) is constant on [α, ∞). The main result of [33] then describes the nonoscillatory behavior of system (H), see Proposition 2.1 below. Based on this result we say that system (H) is nonoscillatory if one and hence all conjoined bases of (H) are nonoscillatory. Proposition 2.1. Assume that the Legendre condition (1.1) holds. Then there exists a nonoscillatory conjoined basis of (H) if and only if every conjoined basis of (H) is nonoscillatory.
For a subspace V ⊆ R n we denote by P V the orthogonal projector onto V. That is, P V is a symmetric and idempotent n×n matrix such that Im P V = V = Ker(I −P V ) and Ker P V = V ⊥ = Im(I − P V ). Orthogonal projectors can be constructed by using the Moore-Penrose pseudoinverse. More precisely, for a given matrix M ∈ R n×n and its pseudoinverse M † the matrix M M † is the orthogonal projector onto Im M , and the matrix M † M is the orthogonal projector onto Im M † = Im M T . Moreover, rank M = rank M M † = rank M † M and Ker(M N ) = Ker(M † M N ) for any matrices M, N ∈ R n×n . For the theory of pseudoinverse matrices we refer to [4], [5,Chapter 6], and [6,Section 1.4]. In particular, we will need the following results on the differentiability of the Moore-Penrose pseudoinverse of a matrixvalued function M (t).
for every t ∈ [α, ∞). In particular, when the matrix M (t) is symmetric and Ker M (t) is constant on [α, ∞), then (2.3) yields the standard formula In the rest of this section (except of Theorem 2.9) we present known properties of conjoined bases of (H) with the corresponding references to the literature. Given a conjoined basis (X, U ) of (H), by the kernel, image, and rank of (X, U ) we mean the kernel, image, and rank of the component X. On the interval [a, ∞) we define the orthogonal projectors onto the subspaces Im X T (t) and Im X(t) by In this case (X, U ) has constant rank r on [α, ∞) with and hence it follows from Remark 2.2 that the function X † ∈ C 1 p on [α, ∞). Consequently, the Riccati quotient is piecewise continuously differentiable on [α, ∞) as well. In addition, by [25, pg. 24] the matrix Q(t) is symmetric and satisfies on [α, ∞) the properties (suppressing the argument t) X T QX = X T U, Im Q ⊆ Im R, QX = RU.

PETERŠEPITKA
This implies that the orthogonal projector onto the set Im S α (t) is eventually constant and we write (2.12) In addition, on [α, ∞) we have the inclusions The main properties of the function S α (t) are summarized in the following statement, which follows from the definition of S α (t) in (2.11), Remark 2.2, and (1.1), see also [28,Theorem 6.1].
Let (X, U ) be a conjoined basis of (H) with constant kernel on [α, ∞) and let S α (t) be the corresponding matrix defined in (2.11).
is nonincreasing on I. In particular, if S α (t) has constant kernel on I = [β, ∞), then the limit of S † α (t) as t → ∞ exists. Remark 2.5. Under (1.1), the results in Proposition 2.4 and the properties of the matrix function S α (t) discussed above imply that for every conjoined basis (X, U ) of (H) with constant kernel on an interval [α, ∞) the limit is well defined and it is referred to as the T -matrix corresponding to the conjoined basis (X, U ) on [α, ∞). Moreover, the matrix T α is symmetric, nonnegative definite, and Im T α ⊆ Im P Sα∞ by (2.12) and Im S † α (t) = Im S T α (t) = Im S α (t) on [α, ∞). Remark 2.6. The matrix S α (t) is intimately connected with a certain class of conjoined bases of (H) which are normalized with (X, U ). As we showed in [28,Theorem 4.4], for a given conjoined basis (X, U ) with constant kernel on [α, ∞) there exists a conjoined basis (X,Ū ) of (H) such that (X, U ) and (X,Ū ) are normalized, i.e., (2.1) holds, and X † (α)X(α) = 0.

We remark that system (H) is said to be
. We note that the set Λ[t, ∞) is nondecreasing in t on [a, ∞) and hence it is eventually constant. This means that the integer-valued function d[t, ∞) is nondecreasing, piecewise constant, and right-continuous on [a, ∞). In particular, there exists the limit which is called the maximal order of abnormality of (H). The monotonicity of the function d[t, ∞) justifies the existence of the point α ∞ ∈ [a, ∞) such that On the other hand, for any α ∈ [a, ∞) the subspace Λ[α, t] is nonincreasing in t on (α, ∞) and hence it is eventually constant. In particular, the integer-valued function d[α, t] is nonincreasing, piecewise constant, and left-continuous on (α, ∞), see also [28,Section 5] The proof of the first formula in ( Throughout this paper we will consider only the intervals [α, ∞) with the maximal order of abnormality d ∞ defined in (2.18). The next remark shows how this condition reflects the properties of S-matrices corresponding to conjoined bases of (H) with constant kernel.
Remark 2.8. (i) Assume (1.1) and let (X, U ) be a conjoined basis of (H) with constant kernel on [α, ∞). In [26,Theorem 4.1.12] we proved that the condition d[α, ∞) = d ∞ holds if and only if the matrix S α (t) in (2.11) associated with (X, U ) satisfies the equalities for every t ∈ [α, ∞), where P Sα∞ and T α are corresponding matrices in (2.12) and (2.13). We note that the identities in (2.26) can be equivalently replaced by see [28,Theorem 6.9]. In addition, by [28,Equation 5.13] the conjoined basis (X, U ) satisfies the conditions The following theorem is an extension of the result presented in Remark 2.8(i). Theorem 2.9. Assume (1.1) and let (X, U ) be a conjoined basis of (H) with constant kernel on [α, ∞). Moreover, let P , S α (t), and T α be its corresponding matrices in (2.5), (2.11), and (2.13), respectively. Then the condition d[α, ∞) = d ∞ is equivalent with the formulas (2.28) Proof. First we remark that by P T α = T α and P S α (t) = S α (t) on [α, ∞) we always have the inclusions Using the latter equation and the identities P Sα∞ P = P Sα∞ and by the first equality in (2.26). Moreover, with the aid of identity S α (t) = S α (t) P Sα∞ we then get which shows that v ∈ Ker P . Therefore, the inclusion Ker [P − S α (t) T α ] T ⊆ Ker P , or equivalently, the inclusion Im P ⊆ Im [P − S α (t) T α ] holds for every t ∈ [α, ∞).
Combining the latter relation with the first property in (2.29) gives the equality Im [P − S α (t) T α ] = Im P on [α, ∞). Conversely, if Im [P − S α (t) T α ] = Im P is satisfied for all t ∈ [α, ∞), then Proposition 2.10. Let (X, U ) be a conjoined basis of (H) with constant kernel on [α, ∞) and let P be its corresponding orthogonal projector in (2.5). Moreover, let (X,Ũ ) be a solution of (H), which is expressed in terms of (X, U ) via matrices M, N ∈ R n×n , that is, where (X,Ū ) is a conjoined basis of (H) satisfying (2.1) and (2.14) with regard to (X, U ). Then the inclusion ImX(α) ⊆ Im X(α) holds if and only if Im N ⊆ Im P . In this case the matrices M and N do not depend on the particular choice of (X,Ū ).
In addition, if (X,Ũ ) is a conjoined basis with constant kernel on [α, ∞) and the equality ImX(α) = Im X(α) holds, then whereP is the matrix in (2.5) associated with (X,Ũ ). The first equality in (2.15) allows to rewrite the expression for the matrixX(t) in (2.30) into the form where S α (t) is the S-matrix in (2.11) associated with (X, U ). In particular, this shows that the inclusion ImX(t) ⊆ Im X(t) holds for every t ∈ [α, ∞), see [26,Theorem 2.3.3]. We also note that the matrix N is the (constant) Wronskian of (X, U ) and (X,Ũ ).
In the next proposition we recall from [29,Theorem 7.6] and [28, Theorems 7.6] the characterization of the nonoscillation of system (H) by the existence of a principal solution of (H) at infinity with any possible rank, as well as the uniqueness of the minimal principal solution.  In [29,Equation 7.4] we defined for a nonoscillatory system (H) the pointα min ∈ [a, ∞) bŷ where (X min ,Û min ) is the minimal principal solution of (H) at infinity. We note that the equality d[α, ∞) = d ∞ holds for every α >α min , see [29,Theorem 7.9]. In turn, combining this fact with formula (2. 19) we obtain that  (ii) Every principal solution (X,Û ) of (H) at infinity is a principal solution with respect to [α, ∞) for every α ∈ (α min , ∞). In other words, the conjoined basis (X,Û ) has constant kernel on the open interval (α min , ∞) and its corresponding matrixŜ α (t) in (2.11) satisfiesŜ † α (t) → 0 as t → ∞ for every α >α min . Remark 3.3. We note that the orthogonal projector PŜ α ∞ in (2.12) associated with the principal solution (X,Û ) through the matrixŜ α (t) is the same for all initial points α ∈ (α min , ∞), see [29,Remark 7.11]. Therefore, we will use the notation Given a principal solution (X,Û ) of (H) at infinity, we define the pointα ∈ [a, ∞) associated with (X,Û ) bŷ From Proposition 3.2 it immediately follows that the pointα in (3.3) satisfies the inequalities α ∞ ≤α ≤α min with α ∞ defined in (2.19). We also note that the set (α, ∞) is the maximal open interval with the property that (X,Û ) is a principal solution of (H) with respect to [α, ∞) for every α ∈ (α, ∞). Therefore, we will often say that (X,Û ) is a principal solution of (H) at infinity with respect to the maximal interval (α, ∞). In particular, the conjoined basis (X,Û ) has constant kernel on the open interval (α, ∞) and the S-matrixŜ α (t) in (2.11) associated with (X,Û ) satisfiesŜ † α (t) → 0 as t → ∞ for every α >α. In the next theorem we derive an exact relation between the pointsα andα min . Theorem 3.4. Assume that (1.1) holds and system (H) is nonoscillatory withα min defined in (3.1). Let (X,Û ) be a principal solution of (H) at infinity and letα be its corresponding point in (3.3). Then the equalityα =α min holds.
Proof. Let (X,Û ),α, andα min be as in the proposition and suppose thatα <α min . According to (3.3) there exists a point β ∈ (α,α min ) such that (X,Û ) is a principal solution of (H) at infinity with respect to the interval [β, ∞). By Proposition 3.2(i) with α := β we know that d[β, ∞) = d ∞ . Let (X min ,Û min ) be the minimal principal solution of (H) at infinity. By [29,Theorem 7.3] it follows that the pair (X min ,Û min ) is a minimal principal solution at infinity with respect to the interval [β, ∞). For this we note that (X min ,Û min ) is contained in (X,Û ) on [β, ∞) according to the properties of the relation "being contained" in [28,Section 5]. The uniqueness of the minimal principal solution and the definition ofα min in (3.1) then yield that β ≥α min , which is a contradiction. Therefore, the equalityα =α min holds and the proof is complete.
In the following result we present a construction of a principal solution of (H) at infinity from a conjoined basis of (H) with constant kernel on [α, ∞). It is a generalization of [32,Equation (10)], where only the minimal principal solution of (H) was considered. This result will be utilized for the construction of a distinguished solution of (R) at infinity in Theorem 7.16.
for some conjoined basis (X, U ) of (H) with constant kernel on [α, ∞). Here the conjoined basis (X,Ū ) and the matrix T α are associated with (X, U ) through Remark 2.6 and (2.13).
Moreover, as we noted in Remark 2.11(ii), this condition is valid on the whole interval [α, ∞), i.e., ImX(t) = Im X(t) holds for all t ∈ [α, ∞). In particular, the last equality means that the conjoined bases (X, U ) and (X,Û ) belong to the same genus of conjoined bases of (H) as we define below, see also Remark 3.13.
In the second part of this section we recall basic concepts from the theory of genera of conjoined bases of (H) from our recent work [27,Section 4]. We wish to point out that in this context the Legendre condition (1.1) is not assumed and/or system (H) is allowed to be oscillatory. Define the orthogonal projector The orthogonal projector R Λ∞ (t) defined in (3.7) plays a crucial role in the following notion. According to [27,Definition 4.3] we say that two conjoined bases (X 1 , U 1 ) and (X 2 , U 2 ) of (H) have the same genus (or they belong to the same genus) From this definition it follows that the relation "having (or belonging to) the same genus" is an equivalence on the set of all conjoined bases of (H). Therefore, there exists a partition of this set into disjoint classes of conjoined bases of (H) with the same genus. This allows to interpret each such an equivalence class G as a genus itself. The following result is proven in [27,Theorem 4.5].
Let G be a genus of conjoined bases of (H) and let (X, U ) be a conjoined basis belonging to G. The results in Proposition 3.8 imply that for all t ∈ [α ∞ , ∞) the subspace Im X(t) + Im R Λ∞ (t) does not depend on the particular choice of (X, U ) in G. Therefore, the orthogonal projector onto Im X(t) + Im R Λ∞ (t), i.e., the matrix is uniquely determined for each genus G. The next statement is from [27, Theorem 4.7].
Proposition 3.9. Let G be a genus of conjoined bases of (H) and let R G (t) be the orthogonal projector defined in (3.11).
The next result describes important properties of nonoscillatory conjoined bases from a given genus G. These properties will be utilized in Section 4 to show their connection with symmetric solutions of the Riccati equation (R) associated with the genus G, see Theorem 4.18.
Proposition 3.11. Let G be a genus of conjoined basis of (H) with the corresponding orthogonal projector R G (t) in (3.11). Furthermore, let (X, U ) be a conjoined basis of (H) with constant kernel on [α, ∞) ⊆ [α ∞ , ∞) such that (X, U ) belongs to G and let R(t) and Q(t) be the matrices in (2.4) and (2.7). Then the equality where the matrices A(t) and B(t) are defined in (1.3).
Proof. For the proof of R G (t) = R(t) we refer to [27,Proposition 4.16]. We will prove that (3.12) and (3.13) hold. From the definition of the matrix The proof is complete. (3.12) we then obtain by the uniqueness of solutions that (3.14) Remark 3.13. In [29,Theorem 7.12] we proved that every genus G of conjoined bases of nonoscillatory system (H) contains a principal solution of (H) at infinity. Moreover, in Theorem 3.5 we described the construction of any such a principal solution in terms of conjoined bases from the genus G, see also Remark 3.6.
In the next proposition we recall from [29, Theorem 7.13] a complete classification of all principal solutions of (H) at infinity within the genus G.
Proposition 3.14. Assume that (1.1) holds and system (H) is nonoscillatory witĥ α min defined in (3.1). Let (X,Û ) be a principal solution of (H) at infinity, which belongs to a genus G. Moreover, letP and PŜ ∞ be the orthogonal projectors defined through the functionX(t) on (α min , ∞) in (2.5), (2.12), and 3.2. Then a solution (X, U ) of (H) is a principal solution belonging to G if and only if for some (and hence for every) α ∈ (α min , ∞) there exist matricesM ,N ∈ R n×n such that 4. Riccati matrix differential equation for given genus. In this section we present a new theory extending the results by Reid in [22,23] about Riccati matrix differential equation (R) to general possibly uncontrollable systems (H). Namely, for every genus G of conjoined bases of (H) we consider the Riccati matrix differential equation (R). In Lemma 4.1, Theorem 4.3, and Corollary 4.5 we first derive properties of solutions of (R) in the relation with the associated projector R G (t) in (3.11). In (4.9) and (4.18) we introduce an auxiliary linear differential system and the so-called F -matrix for a solution of this system, which serve as main tools for the formulation of the results in this section. In particular, in Theorem 4. 16 we present additional properties of solutions of (R) obtained through the above mentioned F -matrix. The main results concerning the correspondence between the solutions of the Riccati equation (R) and conjoined bases of (H) from the genus G are contained in Theorems 4.18 and 4.21.
First we derive some auxiliary properties of the projector R G (t), being a solution of the Riccati equation (1.2), and the coefficient A(t) in (1.3). In particular, we represent R G (t) as a solution of a linear differential system (4.1) We note that since R G (t) is symmetric, then it solves also the system   3).
Proof. First we note that from the definition of the matrix 3) By combining equality (4.4) with the second identity in (4.3) we obtain that for all t ∈ [α ∞ , ∞). In particular, this means that the matrices R G (t) and showing the first part of the lemma. For the proof of the second part we note that the orthogonal projector R G (t) solves the Riccati equation (1.2) on [α ∞ , ∞), by Proposition 3.9. Moreover, by using formula (4.4) and the first identity in (4. Thus, the matrix R G (t) solves system (4.1) on [α ∞ , ∞).

Remark 4.2.
We remark that the formulas in (4.1) are equivalent with Note that the first equation in (4.7) is the same as (1.2) with A(t) instead of A(t).
Next we derive properties of the solutions of (R), which are based on the projector R G (t) and Lemma 4.1.
Theorem 4.3. Let G be a genus of conjoined bases of (H) with the corresponding matrix R G (t) in (3.11) and let Q(t) be a solution of the Riccati equation Proof. Let R G (t) and Q(t) be as in the theorem. By using (R), the second formula in (4.1), and the identities R Similarly, by the first formula in (4.1) and the identities is a solution of (R) on [α, ∞) and the proof is complete.

Proof. From Theorem 4.3 and Remark 4.4 we know that the matrices
and by the uniqueness of solutions of (R) we obtain the equality Q * (t) = Q(t) on [α, ∞). The latter identity means that the inclusion Im Q(t) ⊆ Im R G (t) holds for every t ∈ [α, ∞). By using the similar arguments the relation Im For our reference we now present an auxiliary result from linear algebra about orthogonal projectors. Lemma 4.6. Let Z ∈ R n×n be an orthogonal projector. Then K, L ∈ R n×n satisfy Im K ⊆ Im Z and Im L ⊆ Ker Z Proof. Let Z be as in the lemma. If the matrices K and L satisfy (4.8), then for E := K + L we have that The opposite implication is trivial. Finally, it is easy to see that in this case we have the equality Ker K ∩ Ker L = Ker E, which completes the proof. In particular, in this case the inclusions in (4.8) are implemented as equalities, i.e., the identities Im K = Im Z and Im L = Ker Z hold. We also note that the condition Ker Let G be a genus of conjoined bases of (H) and let R G (t) be its representing orthogonal projector in (3.11). For a given solution Q(t) of the Riccati equation (R) on a subinterval [α, ∞) ⊆ [α ∞ , ∞) we consider the following system of first order linear differential equations on [α, ∞) together with the initial conditions where the matrices K, L ∈ R n×n satisfy We will study the properties of solutions of system (4.9), which will serve for the formulation and proofs of the main results of this section. The first equation in (4.9) is motivated by the approach in [23, Chapter 2, Lemma 2.1], which is adopted here to the setting of uncontrollable systems (H).
These observations then imply that the initial value problem (4.9)-(4.10) with (4.11) has the solution (Θ, Ω), which is unique up to a right nonsingular multiple. More precisely, if (Θ 0 , Ω 0 ) is another solution of (4.9)-(4.11), then there exists a constant nonsingular matrix , Ω(t), K, and L be as in the proposition. By the uniqueness of solutions of the first equation in (4.9) we have that Θ(t) = Φ α (t) Θ(α), where Φ α (t) is the associated fundamental matrix normalized at the point α, i.e., This implies that Ker Θ(t) = Ker Θ(α) for every t ∈ [α, ∞), i.e., the matrix Θ(t) has constant kernel on [α, ∞). Next we show that the matrices V (t) and Indeed, by using (4.1), (4.6), (4.9), and the inclusion in (4.7) we obtain on [α, ∞) that 14) Moreover, suppose that the matrices K and L satisfy (4.11). Then V (α) = 0 = W (α), which in turn, by uniqueness of solutions of (4.14) and (4.15), implies that V (t) = 0 = W (t) for all t ∈ [α, ∞). Therefore, we have Im Θ(t) ⊆ Im R G (t) and Im Ω(t) ⊆ Ker R G (t) on [α, ∞). And since the matrices Θ(t) and R G (t) have constant ranks on [α, ∞) and the equality rank Θ(α) = rank R G (α) holds by Remark 4.8, we obtain that even Im Θ(t) = Im R G (t) for every t ∈ [α, ∞). Now we shall prove the last condition in (4.12), which is clearly equivalent with the identity Ker Θ(t) ∩ Ker Remark 4.10. The results in Proposition 4.9 and Remark 2.2 imply that for any solution (Θ, Ω) of (4.9)-(4.11) the matrix Θ † ∈ C 1 p and satisfies the equation  (4.13). In particular, the matrix Θ † (t) then satisfies for every t ∈ [α, ∞) the formula On the other hand, by the aid of (4.12) we obtain the equality This equality is an important property of the matrix function Φ α (t), which will be utilized in the proof of Theorem 4.16 below.
In the following remark we introduce an important matrix (called the F -matrix) in terms of a solution Θ(t) of (4.9). For an invertible Θ(t) this matrix was considered in [23, Section 2.2]. Here we allow Θ(t) to be singular.
which will be referred to as the F -matrix corresponding to the solution Q(t) with respect to the genus G. From (4.18) it immediately follows that F α (t) is symmetric and the inclusion Im F α (t) ⊆ Im R G (α) holds for every t ∈ [α, ∞) and F α ∈ C 1 p . Moreover, under (1.1) the matrix F α (t) is nonnegative definite and nondecreasing with F α (t) = Θ † (t) B(t) Θ †T (t) ≥ 0 on [α, ∞). Therefore, the subspace Ker F α (t) is nonincreasing on [α, ∞), and hence eventually constant. Consequently, the properties of Moore-Penrose pseudoinverse displayed in Remark 2.2 imply that F is nonincreasing for large t. And since F † α (t) is nonnegative definite on [α, ∞), it follows that the limit of F † α (t) exists as t → ∞, i.e., Clearly, the matrix D α defined in (4.19) is symmetric and nonnegative definite and the inclusion Im D α ⊆ Im R G (α) holds. In addition, we note that with the aid of (4.16) and the identity R (4.18) can be also represented as for all t ∈ [α, ∞) with Φ α (t) defined in (4.13).
The representation of the matrix F α (t) in (4.20) in terms of the fundamental matrix Φ α (t) of (4.13) allows to apply the original result in [23, Lemma 2.1, pg. 12] for symmetric solutions Q(t) of (R). This yields the following statement, which will be utilized in our further analysis.      Proof. Let F α (t) be the matrix in (4.18) associated with Q(t) on [α, ∞) and set G :=Q(α) − Q(α). We will show that the matrix I + F α (t) G is nonsingular on [α, ∞). Fix t ∈ [α, ∞) and let v ∈ Ker [I +F α (t) G], i.e., the equality v = −F α (t) Gv holds. Since the matrix G is symmetric and satisfies G ≥ 0 and from Remark 4.11 we know that under the Legendre condition (1.1) the matrix F α (t) is nonnegative definite, we have that 0 ≤ v T Gv = −v T G T F α (t) Gv ≤ 0. Thus, v T Gv = 0 and consequently, Gv = 0. Therefore, the vector v = −F α (t) Gv = 0 and the matrix I + F α (t) G is nonsingular. Finally, according to Remark 4.13 and Corollary 4.14 this then means that the matrixQ(t) solves the Riccati equation (R) on the whole interval [α, ∞) and satisfies the inequalityQ(t) ≥ Q(t) for every t ∈ [α, ∞).
In the next result we show further properties of the solutions of the Riccati equation (R). Namely, we characterize a certain class of the values K of the initial conditions at some point β, which guarantee that the corresponding solutioñ Q(t) of (R) withQ(β) = K exists on the whole interval [β, ∞). Another interpretation of the following statement is that any symmetric solution Q(t) of (R) on [α, ∞) ⊆ [α ∞ , ∞) satisfying inclusion (1.4) can be decomposed into the product R G (t)Q(t) R G (t) for a suitable, in general nonsymmetric, solutionQ(t) of (R). This result can be regarded as a partial converse to Theorem 4.3 and it will be utilized for the classification of solutions of (R) in Remark 4.20 below.
Proof. First we note that assertion (i) implies (ii) trivially. Therefore, suppose that (ii) holds, i.e., the matrix K satisfies R G (β) KR G (β) = Q(β). Let Φ α (t) and F α (t) be the matrices in (4.13) and (4.18) associated with Q(t) and put We observe that from Remark 4.11(i), inclusion (1.4) with symmetric Q(t), and (4.17) at t = β we have Then the matrix E in (4.24) satisfies (4.26) We will show that the matrix E is nonsingular. Let v ∈ Ker E. This means according to (4.24) that which proves the nonsingularity of E. In particular, formula (4.26) is then equivalent with the equality R G (α) E −1 = R G (α). Now set and consider the solution Q * (t) of (R) satisfying the initial condition Q * (α) = Q(α) + G. We shall prove that the solution Q * (t) is defined on the whole interval [α, ∞) such that Q * (β) = K. First observe that with the aid of (4.24) and (4.28) we get the identity which implies immediately the formula Furthermore, by using the equality R G (α) E −1 = R G (α) and (4.25) we obtain that Fix t ∈ [α, ∞) and let v ∈ Ker [I + F α (t) G], i.e., the equality v = −F α (t) Gv holds.
In particular, the vector v ∈ Im F α (t) ⊆ Im R G (α), by Remark 4.11(i). This means that v = R G (α) v, which in turn together with the equality F α (t) R G (α) = F α (t) and equation (4.30) yields Thus, the matrix I + F α (t) G is nonsingular on [α, ∞) and by Remark 4.13 the solution Q * (t) exists on the whole interval [α, ∞) such that for all t ∈ [α, ∞), by (4.22). In particular, it follows for the matrix Q * (β) that Therefore, by the uniqueness of solutions of (R) the matrixQ(t) solves (R) on the whole interval [α, ∞) withQ(t) = Q * (t) for every t ∈ [α, ∞). In turn, the matrix is also a solution of (R) on [α, ∞), by Theorem 4.3. Finally, since R G (β)Q(β) R G (β) = R G (β) KR G (β) = Q(β), we conclude by using the uniqueness of solutions of (R) once more that R G (t)Q(t)R G (t) = Q(t) for all t ∈ [α, ∞). This shows (i) and the proof is complete.
We are now ready to formulate the main results of this section (Theorems 4.18 and 4.21), in which we connect the solutions Q(t) of the Riccati equation (R) on [α, ∞) with conjoined bases (X, U ) with constant kernel on [α, ∞) from the genus G. These results extend the well known correspondence between the symmetric solutions Q(t) of (R) on [α, ∞) and conjoined bases (X, U ) of (H) with X(t) invertible on [α, ∞), i.e.,  Proof. Let R(t) be the orthogonal projector in (2.4) associated with (X, U ). From Proposition 3.11 we know that R(t) = R G (t) on [α, ∞). By (2.7) the matrix Q(t) then satisfies the equality Q(t) = R G (t) U (t) X † (t) for every t ∈ [α, ∞). Moreover, using the identities in (1.3), (3.12), and (4.2) we obtain on [α, ∞) that Thus, the matrix Q(t) solves the Riccati equation (R) on [α, ∞) and condition (1.4) holds. Furthermore, according to (3.12) in Proposition 3.11 the matrix Θ(t) = X(t) satisfies the first equation in (4.9) on [α, ∞) while applying (3.13) and (R) yields for the matrix Ω(t) the equality
for all t ∈ [α, ∞), where Φ α (t) is the fundamental matrix in (4.13). On the other hand, with the aid of the equalities X(α) X † (α) = R G (α) = X †T (α) X T (α) and for every on t ∈ [α, ∞). Furthermore, let P , P Sα (t), and P Sα∞ be the matrices in (2.4) and (2.12) associated with (X, U ). By combining (4.37) with the identities X T (α) X †T (α) = P and S α (t) P = S α (t) we get F α (t) X †T (α) = X(α) S α (t) P = X(α) S α (t), which in turn through (2.24) implies that where the point τ α, ∞ is defined in (2.23).   Proof. (i) First we show that the pair (X, U ) is a solution of (H) on [α, ∞). From Proposition 4.9 we know that the matrix Θ(t) has constant kernel on [α, ∞) and Im Θ(t) = Im R G (t) for every t ∈ [α, ∞). Thus, the matrix X(t) has constant kernel on [α, ∞) and Im X(t) = R G (t) for all t ∈ [α, ∞). And since Θ(t) solves the first equation in (4.9) on [α, ∞), we have that   Next we derive some additional properties of the matrices Θ(t) and Ω(t), which will simplify our calculations. In particular, by (R) and the first equation in (4.9) we have on [α, ∞) that Now by using (4.44) and (4.45) we obtain that the matrix U (t) in (4.40) satisfies Hence, equalities (4.43) and (4.46) show that the pair (X, U ) solves system (H) on [α, ∞). Moreover, the matrix is symmetric and the subspace Ker for every t ∈ [α, ∞), both by Proposition 4.9. Therefore, the solution (X, U ) is a conjoined basis with constant kernel on [α, ∞). And since the equality Im X(t) = Im R G (t) holds for every t ∈ [α, ∞), we conclude that (X, U ) belongs to the genus G. For the proof of part (ii) we note that the orthogonal projector R(t) in (2.4) associated with (X, U ) satisfies R(t) = R G (t) for all t ∈ [α, ∞). In particular, by using the identities Θ(t) Θ † (t) = R G (t) and R G (t) Ω(t) = 0 on [α, ∞) we obtain that for every t ∈ [α, ∞). Thus, the matrix R G (t) Q(t) R G (t) is the Riccati quotient in (2.7) associated with (X, U ) on [α, ∞) and the proof is complete.

Remark 4.22.
We note that the matrices F α (t) and S α (t) in (4.18) and (2.11) associated with the matrix Q(t) and the conjoined basis (X, U ) in Theorem 4.21, respectively, satisfy the identities in (4.36) and (4.37). This follows directly from Theorem 4.18 and Remark 4.11(ii).

5.
Inequalities for Riccati quotients in given genus. In this section we derive a mutual representation of the Riccati quotients corresponding to conjoined bases of (H) from a given genus G (Theorem 5.3). This representation is then utilized for obtaining inequalities between two Riccati quotients (Corollary 5.5). The results presented in this section essentially generalize the discussion in [7, pg. 54] to possibly uncontrollable systems (H). First we prove an auxiliary property of the image of the matrix F α (t) in (4.18).
Lemma 5.1. Assume (1.1). Let G be a genus of conjoined bases of (H) with the matrix R G (t) in (3.11) and let Q(t) be a solution of the Riccati equation Moreover, let F α (t) be the matrix in (4.18), which corresponds to Q(t), and R Λ∞ (t) be the orthogonal projector defined in (3.7). Then Proof. Let (X, U ) be a conjoined basis of (H) from the genus G, which corresponds to the matrix Q(t) through Theorem 4.21. It follows that (X, U ) has constant kernel on [α, ∞) and the matrix X(t) satisfies the equality Im on (τ α, ∞ , ∞). Moreover, the matrices S α (t) and F α (t) satisfies the identities for every t ∈ (τ α, ∞ , ∞), which can verify by direct computation with the aid of (3.9)-(3.10), (4.36), (4.37), and (5.3).
(ii) Furthermore, upon taking t → ∞ in (5.5) we obtain the formulas where the matrices T α and D α are defined in (2.13) and (4.19), respectively. The equalities in (5.6) then yield that Im D α ⊆ Im R Λ∞ (α) and rank T α = rank D α . In particular, combining the last formula with Remark 2.8(ii) and the fact that d[α, ∞) = d ∞ implies that rank D β is constant with respect to β ∈ [α, ∞).
In the following main result of this section we present a representation of two Riccati quotients corresponding to two conjoined bases of (H) from a genus G. This result will be utilized in the classification of all distinguished solutions of (R) at infinity in Section 7. When G = G max is the maximal genus (in particular, when system (H) is controllable), this representation coincides with the statement in Proposition 4.12. We note that for a given genus G we now compare those solutions Q(t) andQ(t) of (R), which are Riccati quotients according to their definition in (2.7). However, the Riccati equation (R) may also have other solutions, which are not of this particular form.
Theorem 5.3. Let G be a genus of conjoined bases of (H) and let (X, U ) and (X,Ũ ) be two conjoined bases of (H) with constant kernel on [α, ∞) ⊆ [α ∞ , ∞) belonging to G. Moreover, let P and S α (t) be the matrices in (2.5) and (2.11) associated with (X, U ). Suppose that (X,Ũ ) is expressed in terms of (X, U ) via matrices M and N as in Proposition 2.10. Then the Riccati quotients Q(t) and Q(t) in (2.7) corresponding to (X, U ) and (X,Ũ ), respectively, satisfỹ Proof. Let R G (t) be the orthogonal projector in (3.11) and let R(t) andR(t) be the matrices in (2.4), which correspond to the conjoined bases (X, U ) and (X,Ũ ), respectively. According to Proposition 3.11 and the second identity in (2.4) we then have the equalities for all t ∈ [α, ∞). Moreover, the symmetry of the matrices Q(t),Q(t), R(t), and R(t) on [α, ∞), the fact that the matrix N is the Wronskian of (X, U ) and (X,Ũ ) by Remark 2.11(i), and the equations in (5.8) imply that (we omit the argument t) on [α, ∞). Finally, inserting the expression for the matrixX † (t) in (2.37) into the equality in (5.9) yields formula (5.7) on [α, ∞) and the proof is complete.
Remark 5.4. By substituting the matrix X † (t) instead ofX † (t) in (5.9) we get another formula for the differenceQ(t)−Q(t). Namely, inserting the second identity in (2.37) into (5.9) and using the equality P N = N and the symmetry of S α (t) on [α, ∞) yields the formulã for every t ∈ [α, ∞). In addition, ifP is the projector in ( for every t ∈ [α, ∞). In particular, since S α (α) = 0, by evaluating (5.12) at t = α we obtain the equalities Formula (5.7) in Theorem 5.3 yields the following inequalities between two Riccati quotients associated with two conjoined bases from the genus G.
Corollary 5.5. With the assumptions and notation of Theorem 5.3, the Riccati quotients Q(t) andQ(t) satisfy the formulas on [α, ∞). Moreover, the following statements hold. 6. Implicit Riccati matrix differential equation. In this section we study solution spaces of the implicit Riccati equations (1.6) and . These implicit Riccati equations were used in [13,Section 6] in several criteria characterizing the nonnegativity and positivity of the associated quadratic functional. The main contributions of this section (Theorem 6.3 and Corollary 6.4) show that under certain assumption we can transfer the problem of solving the implicit Riccati equations (6.1) and (1.6) into a problem of solving the explicit Riccati equation (R).
In the first result we prove that the two implicit Riccati equations (6.1) and (1.6) are equivalent in terms of their solutions spaces. Proof. Let G and R G (t) be as in the lemma and fix α ∈ [α ∞ , ∞]. Moreover, let Q(t) be an n × n piecewise continuously differentiable matrix-valued function on [α, ∞) and define the functions (we omit the argument t) on [α, ∞). By using (1.3) and (4.3) together with the identity [R G (t)] 2 = R G (t) for every t ∈ [α, ∞) we then obtain (suppressing the argument t) , which proves directly the statement of the lemma.
Following the above remark, we now establish the opposite relation between the solutions of the implicit Riccati equation (6.1) and the Riccati equation (R). Theorem 6.3. Let G be a genus of conjoined bases of (H) with the orthogonal projector R G (t) in (3.11). Let Q(t) be a solution of the implicit Riccati equation Proof. Let R G (t) and Q(t) be as in the theorem. With the aid of (4.1), (4.2), (6.1), and the equalities (suppressing the argument t) R G CR G = C and B = R G BR G on [α, ∞) we get The results in Theorem 6.3 and Lemma 6.1 yield the following. Proof. The implication (i) ⇒ (ii) follows by Remark 6.2. The equivalence of the assertions in (ii) and (iii) is a direct consequence of Lemma 6.1. Now assume (ii), i.e., suppose that the matrix Q(t) is a solution of (6.1) on [α, ∞). The result of Theorem 6.3 and the identities for t ∈ [α, ∞) then imply that Q(t) solves (R) on [α, ∞), showing (i).

7.
Distinguished solutions at infinity. In this section we study, for a given genus G, symmetric solutions of the Riccati equation (R), which correspond to principal solutions of (H) at infinity belonging to the genus G. This correspondence is based on the results in Theorems 4.18 and 4.21 and in Remark 4.20. We introduce the notion of a distinguished solution of (R) at infinity (Definition 7.1) and prove its main properties. In particular, we establish the results about distinguished solutions of (R) at infinity regarding their relationship to principal solutions at infinity (Theorems 7.4 and 7.5) and to the nonoscillation of system (H) at infinity (Theorem 7.8), their interval of existence (Theorem 7.13), their mutual classification within the genus G (Theorem 7.15), and their minimality in a suitable sense (Theorems 7.16 and 7.18). It may be surprising that these results comply with the known theory of distinguished solutions of the Riccati equation (R) for a controllable system (H) only partially. In many aspects the presented theory for general uncontrollable system (H) is substantially different. This is related to the nature of the problem, since for each genus G of conjoined bases of (H) there is a different Riccati equation (R), but even within one genus G there may be many distinguished solutions of (R) at infinity. We discuss these issues in Remark 7.25 at the end of this section. We note that the true uniqueness and minimality of the distinguished solution of (R) at infinity is satisfied only in the minimal genus G min (see Theorem 7.23).
The following definition extends the notion of a distinguished solution (also called a principal solution) of (R) at infinity for a controllable system (H) in [7, pg. 53].
Definition 7.1. Let G be a genus of conjoined bases of (H) with the orthogonal projector R G (t) in (3.11). A symmetric solutionQ(t) of the Riccati equation (R) is said to be a distinguished solution at infinity if the matrixQ(t) is defined on an interval [α, ∞) ⊆ [α ∞ , ∞) and its corresponding matrixF α (t) in (4.18) satisfieŝ F † α (t) → 0 as t → ∞. The notion in Definition 7.1 also extends the distinguished solution of (R) introduced by W. T. Reid in [21, Section IV] and [23, Section 2.7], which in our context corresponds to the maximal genus G = G max (for which R G (t) ≡ I).
Remark 7.2. When it is clear from the context, we will often drop the term "at infinity" in the terminology in Definition 7.1. We also remark that a distinguished solution of the Riccati equation (R) associated with the genus G is also defined by the propertyD α = 0 with the matrixD α in (4.19) corresponding toF α (t).
In the next auxiliary statement we show that the property of being a distinguished solution of (R) is invariant under the multiplication by the orthogonal projector R G (t). This property will be utilized in the proofs of the subsequent main results. Proof. From Theorem 4.3 we know that the matrix R G (t) Q(t) R G (t) solves (R) on [α, ∞). And since by Remark 4.11(ii) the matrices Q(t) and R G (t) Q(t) R G (t) have the same F -matrices in (4.18) with respect to the interval [α, ∞), the statement follows directly from Definition 7.1.
The following two results show that in the context of Theorems 4.18 and 4.21 the distinguished solutions of (R) correspond to the principal solutions of (H) at infinity from the genus G.
Theorem 7.4. Let G be a genus of conjoined bases of (H) and R G (t) be the projector in (3.11). Moreover, letQ(t) be a distinguished solution of (R) at infinity with respect to the interval [α, ∞) ⊆ [α ∞ , ∞). Then every conjoined basis (X,Û ) of (H), which is associated withQ(t) on [α, ∞) via Theorem 4.21, is a principal solution of (H) at infinity with respect to [α, ∞) belonging to the genus G.
Proof. Let R G (t) andQ(t) be as in the theorem. According to Remark 7.2 the ma-trixD α in (4.19) corresponding toQ(t) satisfiesD α = 0. Let (X,Û ) be a conjoined basis of (H), which is associated with the matrixQ(t) on [α, ∞) via Theorem 4.21. Then (X,Û ) belongs to the genus G such that (X,Û ) has constant kernel on [α, ∞). Moreover, ifT α is the T -matrix in (2.13) associated with (X,Û ) through the ma-trixŜ α in (2.11), then we have rankT α = rankD α = 0, by Remark 5.2(ii). Hence, T α = 0 and (X,Û ) is a principal solution at infinity. Remark 7.6. We note that according to Theorem 4.18 the distinguished solu-tionQ(t) at infinity in Theorem 7.5 satisfies the additional property (1.4), i.e., the inclusion ImQ(t) ⊆ R G (t) for all t ∈ [α, ∞). In particular, the latter relation together with the symmetry of the matrixQ(t) on [α, ∞) yields the identitŷ Q(t) = R G (t)Q(t) R G (t) for every t ∈ [α, ∞). Moreover, from Lemma 7.3 it follows that every symmetric solution Q(t) of (R) on [α, ∞), for which the matrix R G (t) Q(t) R G (t) is the Riccati quotient in (2.7) associated with (X,Û ), is also a distinguished solution at infinity with respect to [α, ∞). In general, however, such a matrix Q(t) does not need to satisfy the inclusion in (1.4). From Theorems 7.4 and 7.5 it follows that the property of the existence of a principal solution of (H) at infinity in the genus G, as stated in [29,Theorem 7.12], transfers naturally to the existence of a distinguished solution at infinity of the associated Riccati equation (R).
Corollary 7.7. Let G be a genus of conjoined bases of (H) with the orthogonal projector R G (t) in (3.11). Then there exists a principal solution of (H) at infinity belonging to the genus G if and only if there exists a distinguished solution of the Riccati equation (R) at infinity. In this case, the set of all Riccati quotients in (2.7), which correspond to the principal solutions (X,Û ) of (H) at infinity from the genus G, coincides with the set of all matrices R GQ R G , whereQ is a distinguished solution of (R) at infinity.
Proof. The statement follows directly from Theorems 7.4 and 7.5 and from Remark 7.6.
In the following result we characterize the nonoscillation of system (H) in terms of the existence of a distinguished solution of the Riccati equation (R) in a given (or every) genus G. This corresponds to [29,Theorems 7.6 and 7.12] regarding the principal solutions of (H) at infinity. The proof of Theorem 7.8 is displayed below after the following two remarks.
Remark 7.9. The result in Theorem 7.8 justifies the development of the theory of genera of conjoined bases for possibly oscillatory system (H). Of course, assuming that system (H) is nonoscillatory, then it is sufficient to use the theory of genera of conjoined bases from [29,Section 6] and [31,Section 4] for the construction of distinguished solutions of the Riccati equation (R) for a genus G. It is the converse to this implication, which requires a more general approach, since in this case we need to define the coefficients of equation (R) without the assumption of nonoscillation of system (H). This natural requirement was the initial motivation for the study presented in [27]. Proof of Theorem 7.8. If (H) is nonoscillatory, then by Remark 3.13 for any genus G of conjoined bases of (H) there exists a principal solution of (H) at infinity belonging to G. In turn, there exists a distinguished solution of the Riccati equation (R) at infinity for every genus G, by Corollary 7.7. Moreover, assertion (iii) implies (ii) trivially. Finally, by using Corollary 7.7 once more, assertion (ii), that is, the existence of a distinguished solution of (R) at infinity for some genus G, means that there exists a principal solution of (H) at infinity, which belongs to G. Since every principal solution is a nonoscillatory conjoined basis, system (H) is nonoscillatory, by Proposition 2.1. This shows the validity of (i) and completes the proof.
The next two results deal with the interval of existence of distinguished solutions of (R). In particular, we determine the maximal interval of existence for each particular distinguished solution of (R). Moreover, we show that this maximal interval is the same for all distinguished solutions of (R) as well as for all genera G. Then the matrixQ(t) is a distinguished solution of (R) at infinity also with respect to the interval [β, ∞) for every β ≥ α.
Proof. Let (X,Û ) be a conjoined basis of (H) corresponding toQ(t) on [α, ∞) via Theorem 4.21. In particular, the matrix R G (t)Q(t) R G (t) is the Riccati quotient in (2.7) associated with (X,Û ) on [α, ∞). Moreover, from Theorem 7.4 we know that (X,Û ) is a principal solution of (H) at infinity with respect to [α, ∞), which belongs to the genus G. Fix now β ≥ α. Then (X,Û ) is a principal solution of (H) at infinity with respect to [β, ∞), by Proposition 3.2(i). Consequently, the matrix R G (t)Q(t) R G (t) is a distinguished solution of (R) at infinity with respect to [β, ∞), by Theorem 7.5. Finally, by using Lemma 7.3 we conclude that also the matrixQ(t) is a distinguished solution of (R) at infinity with respect to the interval [β, ∞). Theorem 7.13. Assume that (1.1) holds and system (H) is nonoscillatory witĥ α min defined in (3.1). Let G be a genus of conjoined bases of (H) and let R G (t) be its corresponding orthogonal projector in (3.11). Moreover, letQ(t) be a distinguished solution of the Riccati equation (R) at infinity with αQ defined in (7.1). Then the equality αQ =α min holds.
Then (X,Û ) is a principal solution of (H) at infinity with respect to the interval [α, ∞), by Theorem 7.4. Moreover, from Theorem 3.4 we know that (X,Û ) is a principal solution with respect to the maximal open interval (α min , ∞). Thus, we have the inequalityα min ≤ α. And since α ∈ [α ∞ , ∞) was chosen arbitrarily with regard toQ(t), we obtain that αQ ≤α min , by (7.1). Now we show that the last inequality is implemented as the equality. Suppose that αQ <α min . According to (7.1) there exists β ∈ (αQ,α min ) such thatQ(t) is a distinguished solution of (R) at infinity with respect to the interval [β, ∞). In turn, the conjoined basis (X, U ) in Theorem 4.21 applied to Q(t) :=Q(t) on [β, ∞) is a principal solution of (H) at infinity with respect to [β, ∞), by Theorem 7.4. Applying formula (3.3) and Theorem 3.4 with (X,Û ) := (X, U ) then yields the inequality β ≥α min , which is a contradiction. Therefore, αQ =α min holds and the proof is complete.
Remark 7.14. Given a genus G of conjoined bases of (H) with the matrix R G (t) in (3.11), from Theorem 7.13 it follows that any distinguished solutionQ(t) of (R) is defined on the maximal interval (α min , ∞) and the corresponding matrix F α (t) in (4.18) satisfiesF † α (t) → 0 as t → ∞ for every α >α min .
In the following result we present a mutual classification of all distinguished solutions of the Riccati equation (R). This classification is formulated in terms of the initial values of the involved distinguished solutions at some point α from the maximal interval (α min , ∞).
Theorem 7.15. Assume that (1.1) holds and system (H) is nonoscillatory witĥ α min and R Λ∞ (t) defined in (3.1) and (3.7), respectively. Let G be a genus of conjoined bases of (H) and let R G (t) be the matrix in (3.11). Moreover, letQ(t) be a distinguished solution of the Riccati equation (R) at infinity. Then a symmetric solution Q(t) of (R) defined on a neighborhood of some point α ∈ (α min , ∞) is a distinguished solution at infinity if and only if Proof. Fix α ∈ (α min , ∞) and letQ(t) be as in the theorem. From Definition 7.1 and Remark 7.14 we know thatQ(t) is a distinguished solution of (R) at infinity with respect to the interval [α, ∞). Moreover, let (X,Û ) be a conjoined basis of (H), which corresponds toQ(t) on [α, ∞) via Theorem 4.21. Then (X,Û ) is a principal solution of (H) at infinity with respect to [α, ∞) belonging to the genus G, by Theorem 7.4. LetŜ α (t) be the S-matrix in (2.11) associated with (X,Û ). Suppose that Q(t) is a distinguished solution of (R) at infinity. Thus, Q(t) is a distinguished solution of (R) with respect to [α, ∞) and the corresponding conjoined basis (X, U ) of (H) in Theorem 4.21 is a principal solution with respect to [α, ∞), which belongs to the genus G. From Proposition 3.14 it then follows that there exist matriceŝ M ,N ∈ R n×n such that In order to simplify the notation we set Z := R Λ∞ (α) Q(α) R Λ∞ (α),Ẑ := R Λ∞ (α)Q(α) R Λ∞ (α). (3.10), formula (7.5) and the last condition in (7.4) imply Thus, with respect to (7.6) the matrix Q(α) satisfies equality (7.2). Conversely, assume that Q(t) is a symmetric solution of (R) defined on a neighborhood of α such that the condition in (7.2) holds. We will use the notation in (7.6). Let F α (t) be the F -matrix corresponding toQ(t) in (4.18) and setĜ := Q(α) −Q(α).
Then we have R Λ∞ (α)ĜR Λ∞ (α) = 0, by (7.2). Next we will show that the matrix Consequently, according to Remark 4.13 with Q :=Q,Q := Q, and G :=Ĝ, we conclude that the symmetric matrix Q(t) solve equation (R) on the whole interval [α, ∞). Let (X, U ) be a conjoined basis of (H) associated with Q(t) on [α, ∞) via Theorem 4.21. Then (X, U ) has constant kernel on [α, ∞) and belongs to the genus G. Therefore, the identities Im X(t) = Im R G (t) = ImX(t) hold for all t ∈ [α, ∞), by Remark 3.13. Hence, from Proposition 2.10 and Remark 2.11(i) with (X, U ) := (X,Û ) and (X,Ũ ) := (X, U ) it then follows that there exists matricesM ,N ∈ R n×n such that the formulas in (7.3) and the first three conditions in (7.4) hold. Similarly as in the first part of the proof, the matrix R G (t) Q(t) R G (t) is the Riccati quotient in (2.7) associated with (X, U ) on [α, ∞) and the equality in (7.5) holds. Moreover, multiplying (7.5) by the matrix R Λ∞ (α) from the both sides and using the identities In turn, by using (7.6) and (7.2) we have Z =Ẑ and hence formula (7.7) becomes From Remark 3.7 it follows that ImX † (α) R Λ∞ (α) = Im PŜ ∞ , which means that we haveX † (α) R Λ∞ (α) K = PŜ ∞ for some invertible matrix K. By using (7.8) we then obtain that which is the last condition in (7.4). Thus, according to Proposition 3.14 the conjoined basis (X, U ) is a principal solution of (H) at infinity with respect to [α, ∞). Finally, with the aid of Remark 7.6 the matrix Q(t) is then a distinguished solution of (R) at infinity with respect to [α, ∞). The proof is complete.
In the next three results we study the minimality of distinguished solutions of (R). This minimality property needs to be understood in the following sense. For every symmetric solution Q(t) of (R) there exists a distinguished solution of (R), which exists on the same interval and is at the same time smaller than Q(t) on this interval (Theorems 7.16 and 7.18). On the other hand, any symmetric solution of (H), which is smaller than a distinguished solution of (H) on some interval, is a distinguished solution itself with respect to this interval (Theorem 7.20). However, in general there is no universal "smallest" distinguished solution of the Riccati equation (R), see Remark 7.21. We also note that in the first result we consider the case when the solutions satisfy condition (1.4), while in the second and third result this assumption is removed.
Proof. Let Q(t) be as in the theorem and let (X, U ) be its associated conjoined basis of (H) in Theorem 4.21 or Remark 4.23. Then (X, U ) has constant kernel on [α, ∞) and belongs to the genus G. Moreover, through (1.4) the matrix Q(t) = R G (t) Q(t) R G (t) is the Riccati quotient in (2.7) corresponding to (X, U ) on [α, ∞). Let T α be the T -matrix in (2.13) associated with (X, U ) on [α, ∞) and consider the solution (X,Û ) of (H) in (3.4). From Theorem 3.5 and Remark 3.6 we know that (X,Û ) is a principal solution of (H) at infinity belonging to the genus G. LetQ(t) be its corresponding Riccati quotient in (2.7) on [α, ∞). According to Theorem 7.5 and Remark 7.6 the matrixQ(t) is a distinguished solution of (R) at infinity with respect to [α, ∞) satisfying condition (1.4). Moreover, since the matrix T α is nonnegative definite, by Remark 2.5, with the aid of Corollary 5.5(ii) withQ :=Q, N := −T α , and M := I we then have that Q(α) ≥Q(α). Finally, this inequality implies through Corollary 4.15 that Q(t) ≥Q(t) for all t ∈ [α, ∞), which completes the proof.
Remark 7.17. (i) By applying (5.7) in Theorem 5.3 we obtain an exact relation between the Riccati quotients Q(t) andQ(t) on [α, ∞). Namely, the formulâ holds for every t ∈ [α, ∞). In particular, for t = α the equality in (7.9) becomeŝ (ii) According to Remark 7.14, the point α ∈ [α ∞ , ∞) in Theorem 7.16 satisfies α >α min . Moreover, from Theorems 4.3 and 7.16 it follows that the last inequality holds even when condition (1.4) regarding the matrix Q(t) is dropped. Hence, we conclude that for any genus G the open interval (α min , ∞) is the maximal set such that there exists a symmetric solution of the Riccati equation (R) on (α min , ∞).
Remark 7. 19. We note that the converse to Theorem 7.18 also holds. More precisely, ifQ(t) is a distinguished solution of (R) at infinity with respect to the interval [α, ∞), then every symmetric solution Q(t) of (R), which satisfies the condition Q(α) ≥Q(α), exists on the whole interval [α, ∞) and the inequality Q(t) ≥Q(t) holds on [α, ∞). This observation is a direct application of Corollary 4.15 with the choice Q :=Q andQ := Q.