Characterizations of equilibrium controls in time inconsistent mean-field stochastic linear quadratic problems. I

In this paper, a class of time inconsistent linear quadratic optimal control problems of mean-field stochastic differential equations (SDEs) is considered under Markovian framework. Open-loop equilibrium controls and their particular closed-loop representations are introduced and characterized via variational ideas. Several interesting features are revealed and a system of coupled Riccati equations is derived. In contrast with the analogue optimal control problems of SDEs, the mean-field terms in state equation, which is another reason of time inconsistency, prompts us to define above two notions in new manners. An interesting result, which is almost trivial in the counterpart problems of SDEs, is given and plays significant role in the previous characterizations. As application, the uniqueness of open-loop equilibrium controls is discussed.

Inspired by the formulation of mean-variance portfolio selection problems, it is reasonable to keep the state process of above optimal control problem stable with respect to possible variation of random factors. One effective way is to add the variation of X(·), i.e.
into the cost functional, and we end up with the form of Under proper conditions, optimal control of the formū = Θ 1X + Θ 2 E tX exists with appropriate Θ i , see e.g. Section 3 of [20]. Plugging it into (1.1), we arrive at one conditional mean-field SDEs for optimal stateX, Here and after, the time reference may be omitted for simplicity. The solvability of (1.5) is easy to see if moreover, A, B, C, D are bounded and deterministic, b, σ are proper processes. We also consider the following quadratic cost functional (1.6) J(u(·); t, X(t)) = 1 2 E t T t QE t X, E t X + RE t u, E t u + QX, X + Ru, u ds which is obviously well-defined. Our linear quadratic optimal control problem can be stated as follows.
If t = 0, Problem (LQ) was studied in [19], (see also [3], [10], [11]) and the optimal control exists under proper conditions. Returning back to above dynamic setting, any optimal controlū(·) associated with (t, X(t)) satisfying (1.7) will depend on t and demonstrate the time-inconsistency property, i.e. u(s; t 1 ,X(t 1 )) =ū(s; t 2 ,X(t 2 )) for some (t 1 , t 2 , s) with t ≤ t 1 ≤ t 2 ≤ s ≤ T. In other words, to solve Problem (LQ), one has to make the choice between "optimality" and "time consistency". In most existing papers along this line, the time consistency was kept, and the traditional closed-loop optimal controls, open-loop 2 optimal controls were replaced by closed-loop equilibrium controls, open-loop equilibrium controls, respectively. As to closed-loop equilibrium controls, we refer the reader to e.g., [2], [13], [16], [20], where some delicate convergence arguments from discrete time to continuous case were used, and [9], [14], where a new approach based on variational ideas were developed without convergence procedures. We also refer to [5], [6] for the corresponding study of investment and consumption problems with non-exponential discounting. On the other hand, there were also many articles on open-loop equilibrium controls, see e.g., [7], [8], [12], [15], [20], and so on. We point out that almost all the previous literature on time inconsistent stochastic linear quadratic problems focused on the particular case of A = B = C = D = 0, except [20] where the closed-loop equilibrium controls of Problem (LQ) were introduced and studied via multi-person differential games approach. To our best, the investigation on open-loop equilibrium controls of Problem (LQ) is still open. To fill this gap, in this paper we introduce two notions, i.e., open-loop equilibrium controls and their closed-loop representations, of Problem (LQ), and establish their characterizations by the variational ideas in [9], [14]. As application, we discuss the uniqueness of open-loop equilibrium controls.
There are several essential differences between the existing papers and ours. In contrast with [7], [8], [12], [14], [20], our state equation is a general conditional mean-field SDE. The additional mean-field terms is the second reason of time inconsistency, and requires us to propose new definitions of equilibrium controls and new mathematical tricks, see e.g. Lemma 3.4. Even under the particular SDEs case, our obtained secondorder equilibrium conditions did not appear in [7], [8], [20], [1]. For the proof of uniqueness of open-loop equilibrium controls, our result extends the counterparts in [8], and our procedures are different from theirs as well. We emphasize that the characterization viewpoint on time inconsistent stochastic linear quadratic problem were also used in other specific/different frameworks, such as [4], [8], [9], [12], [14]. At last, by our study we also find the following interesting facts: • The open-loop equilibrium controls are characterized by two kinds of conditions: f irst-order, secondorder equilibrium conditions, which is comparable with the f irst-order, second-order necessary optimality conditions in traditional optimal control problems.
• The second-order equilibrium condition is the same as the second-order optimality condition of meanfield SDEs, and it appears in both open-loop equilibrium controls and their closed-loop representations.
• As to the closed-loop representations of open-loop equilibrium controls, the first-order equilibrium condition includes a system of Riccati equations, which appears for the first time and are essentially different from that of closed-loop equilibrium controls in [20].
The article is organized as follows. In Section 2, we introduce some useful spaces, as well the notions of open-loop equilibrium controls, and their closed-loop representations. In Section 3, we characterize both notions by variational approach. In Section 4, we discuss the uniqueness of open-loop equilibrium controls under proper conditions. Section 5 concludes this paper.

Preliminaries
We first introduce the following hypotheses.
In the following, let K be a generic constant which varies in different context, and If the state equation is a particular controlled SDE, we can use similar form of SDE to describe the equilibrium state process. However, since the increment of state process X in (1.5) has the reliance on additional time reference t, the value of X at time s > t also depends on t. As a result, we need to propose an alternative kind of process as the equilibrium state process.
To get some inspirations from existing papers, we first look at one linear quadratic problem associated with state equation Here the increment of state variable in (2.2) also relies on initial time t, and discounting functions Q, R are not necessary to be exponential form. Both facts naturally lead to the time inconsistency of optimal control. According to [16], [17], the equilibrium controlū(·) not only relies on s ∈ [0, T ], Q(s, s), R(s, s) , but also on equilibrium stateX(·) define by In other words, both equilibrium controlū and the equilibrium stateX depends on the diagonal value (i.e., t = s) of coefficients Q, R, A, B. Similar phenomenon also happens in investment and consumption problems with power-type utilities and general non-exponential discounting, see Section 6.2 of [18].
We return back to our state equation (1.5) again. Following the same principle as above (2.3), it is expected that the corresponding equilibrium state process, denoted by X * (·), should satisfy with notations in (2.1) and the equilibrium control u * (·). Keeping above arguments in mind, we introduce the following notion.
We also introduce the closed-loop representation of open-loop equilibrium control u * (·).

Some useful lemmas
, for later convenience we rewrite the state equation as follows We introduce the following BSDEs to deal with the quadratic cost functional (1.6), Remark 3.1. As to X v,ε 1 (·), by some standard calculations one has where K only depends on p > 1 and v ∈ L 2 Ft (Ω; R m ). Given the backward equations in (3.2), for any t ∈ [0, T ) and small ε > 0, we see that they are uniquely solvable with When there is no mean-field terms in (1.5), the pair of processes (Y v,ε 1 , Z v,ε 1 ) of (3.2) appeared in [14], but were absent in [7], [8] and [20].
The following result shows the roles of previous (Y, Similarly we have To sum up, for any t ∈ [0, T ) we obtain that, Similarly we also obtain that, In order to deal with Therefore, our conclusion follows from (3.7), (3.8), (3.9), (3.10).
Proof. For any t ∈ [0, T ) and small ε > 0 such that We define two processes as, it then follows from Itô's formula that As a result, for s ∈ [t, T ] we see that On the other hand, Hence (Y v,ε 1 , Z v,ε 1 ) satisfies the second backward equation of (3.2). The uniqueness of BSDEs show that where P 1 (·), P 2 (·) appeared above and for s ∈ [t, t + ε], Here we observe that On the other hand, according to the definition of ( . Moreover, it is easy to see The uniqueness of BSDEs shows that ( As a result, for any s ∈ [t, t + ε), Then our conclusion is easy to see. Remark 3.3. Notice that (3.12) is consistent with the second-order adjoint equation in optimal control problem of mean-field SDEs, see e.g., [3].

The case of open-loop equilibrium controls
In this part, we give the characterizations of open-loop equilibrium controls. At first, we introduce a representation for (Y, Z) in (3.2). We observe that For t ∈ [0, T ] and s ∈ [t, T ], suppose that where P 1 (·), P 2 (·) are deterministic, P 3 (·), P 4 (·) are stochastic processes satisfying Here Π i (·) are to be determined. Using Itô's formula, we derive that As a result, we have Consequently, it is necessary to see In this case, from (3.20), (3.21), we see that On the other hand, by the previous representations, At this moment, we can choose Π i (·) in the following ways, Next we make above arguments rigorous. Given the notations in (2.1), for s ∈ [0, T ], we consider the following systems of equations For u(·) ∈ L 2 F (0, T ; R m ), under (H1) it is obvious to see the existence and uniqueness of P 1 (·), P 2 (·), (P 3 (·), L 3 (·)), (P 4 (·), L 4 (·)) and By the results of P i (·), we can conclude that where (Y d (s), Z d (s)) ≡ (Y (s, s), Z (s, s)) with s ∈ [0, T ]. We present the following representation for (Y, Z).

Proof. Given (3.23), it is easy to see that
Using Itô's formula, we know that Consequently, after some calculations one has Then for any t ∈ [0, T ], (Y , Z ) ∈ L 2 F (Ω; C([t, T ]; R n )) × L 2 F (0, T ; R n ) satisfies the first backward equation in (3.2). By the uniqueness of BSDEs, we see the conclusion.

To our best, the conclusion of Lemma 3.3 is new in the literature.
To obtain the main result, we need one more result. For t ∈ [0, T ), x 0 ∈ R n , and u(·) ∈ L 2 F (0, T ; R m ), we consider the following SDE Proof. First we observe that (3.28) where Therefore, Moreover, for any t ∈ [0, T ), As a result, On the other hand, by (3.26), for any t ∈ [0, T ), we have Recall the equation of X (·) and X(·), for any t ∈ [0, T ) we know that As a result, we see that E t X(s) − X (s) = 0 with s ∈ [t, t + ε). Hence G 0 (s, X (s))ds.
Remark 3.5. Thanks to the Markovian framework and the appearance of conditional expectation operator E · , we obtain Lemma 3.4 and transform the investigation of (X, Y , Z ) into that of (X , M, N ).
In our context, above (3.34) and (3.33) are named as the f irst-order, second-order equilibrium condition, respectively.
Remark 3.7. To our best, the introduced second-order equilibrium condition (3.33) has not been discussed in other papers on time inconsistent optimal control problems. Moreover, it is the same as second-order necessary optimality condition of mean-field SDEs ( [3]), which takes us by surprise.
Remark 3.8. If the mean-field terms in the state equation disappears, Theorem 3.1 reduces to the counterparts in [14]. If moreover R, Q = 0, Q, R, G are definite, then (3.33) is obvious to see, and Theorem 3.1 becomes consistent with Theorem 3.5 in [8]. We first give the equation satisfied by (M, N ). Recall (3.22), using Itô's formula, we see that,
• If b = σ ≡ 0, R = Q = 0, m = n = 1, ϕ * is deterministic, then Λ 3 = Λ 4 ≡ 0, P * 3 , P * 4 satisfy deterministic backward ODEs. This corresponds to the case in [7], [8]. Notice that the randomness of b, σ leads to above BSDEs which appears for the first time to our best. Remark 3.10. We look at another case of (3.41) when A = B = C = D = 0: Here we use the term P i instead.
• The coefficients of above system of equations relies on the mean-field terms. To our best, such system is new in the related literature.
• We point out one more interesting thing. If we add the first two equations together, we see that It is a direct calculation that (P 1 + P 2 ) in (3.51) satisfies the same equation as (3.52) if we replace A, B, C, D with A, B, C, D. One can also obtain similar conclusion for (P 3 + P 4 ) and ( P 3 + P 4 ).
Remark 3.11. We observe that the introduced terms B, D in (1.5) could bring essential influence on (Θ * , ϕ * ) in (3.45). Here are some special cases.
• In terms of (3.45), both D and D play important roles in allowing R, R to be indefinite. In other words, if R = R = D = 0, equilibrium control u * could still have feedback form under proper conditions.
• Suppose R ≥ δ > 0 and B = D = 0. From (3.45) we see that the feedback form could still make sense by imposing suitable conditions on B, D.
• Suppose R ≥ δ > 0, and B = 0 or D = 0. According to (3.45), the terms B, D could deny the existence of feedback form for u * , if for example, B := −B, D := −D.

Uniqueness of open-loop equilibrium controls
In this section, we study the uniqueness of open-loop equilibrium controls.
The following result shows more explicit structure of solution for (4.5).
We define Y := Y 1 + E t Y 2 , Z := Z 1 . It is easy to verify that (Y, Z) satisfy equation (  As to (4.8), the standard mean-field BSDEs theory shows the unique solvability of (M, N ) ≡ (0, 0). Then our conclusion is easy to see.
The uniqueness of open-loop equilibrium in Markovian setting was also studied in Section 4 of [8]. In contrast, we obtain the similar uniqueness conclusions by a different approach under the general mean-field framework.

Concluding remarks
In this paer, a class of time inconsistent stochastic linear quadratic problems is discussed where the state equation is described by a controlled linear conditional mean-field stochastic differential equations (SDEs).
Since the mean-field terms in state equations also lead to time inconsistency, both open-loop equilibrium controls and their closed-loop representations have to be redefined in new manners. The characterizations are established for previous two notions and several new features are revealed as well. An interesting result, i.e., Lemma 3.4, and several remarks in Section 3 are given to explain the essential difference with the particular case of controlled SDEs. The relevant study on closed-loop equilibrium controls/strategies and related Riccati equations is much more complicated, and we hope to discuss it in our future publications.