INDEFINITE LQ OPTIMAL CONTROL WITH PROCESS STATE INEQUALITY CONSTRAINTS FOR DISCRETE-TIME UNCERTAIN SYSTEMS

. Uncertainty theory is a branch of axiomatic mathematics that deals with human uncertainty. Based on uncertainty theory, this paper dis- cusses linear quadratic (LQ) optimal control with process state inequality constraints for discrete-time uncertain systems, where the weighting matrices in the cost function are assumed to be indeﬁnite. By means of the maximum principle with mixed inequality constraints, we present a necessary condition for the existence of optimal state feedback control that involves a constrained diﬀerence equation. Moreover, the existence of a solution to the constrained diﬀerence equation is equivalent to the solvability of the indeﬁnite LQ problem. Furthermore, the well-posedness of the indeﬁnite LQ problem is proved. Fi- nally, an example is provided to demonstrate the eﬀectiveness of our theoretical results.

1. Introduction. The stochastic linear quadratic (LQ) optimal control problem was pioneered by Wonham [22], which has been extensively studied [25,26,4,24,2,20] in the past decades. It is known that the dynamic programming [4,24] and the stochastic maximum principle [2,20] are two main techniques to solve stochastic optimal control problem. For the stochastic LQ optimal control problem, the early reference assumed that the control weight is positive definite and the state weight is positive semidefinite. It has been shown recently that for stochastic systems, LQ problem with indefinite control weights could make sense [5,11]. Owing to many applications, LQ problem with indefinite control weights have drawn increasing attention ranging from pollution control [5] to portfolio selection [28]. To some extent, the influence of stochastic noises compensate the negative control weights to make the problem well-posed.
As we know, probability theory has been used to deal with stochastic phenomenon for a long time. Before applying it in practice, we should first obtain the probability distribution via statistics, or test the probability distribution to make sure it is close enough to the real frequency, either of which is based on a lot of observed data. However, due to the technological or economical difficulties, we sometimes have no samples. In this case, we have to invite some domain experts to evaluate their belief degree about the chances that the possible events happen. According to Kahneman and Tversky [10], humans tend to overweight unlikely events, so the belief degree generally has a much larger range than the real frequency. As a result, the probability theory is not applicable in this case, otherwise some counterintuitive results may be derived. In order to rationally deal with personal belief degrees, uncertainty theory was founded by Liu [12] in 2007, and refined by Liu [13] in 2010 based on an uncertain measure which satisfies normality, duality, subadditivity and product axioms. So far, the content of uncertainty theory has been developed to a fairly complete system for modeling human uncertainty and has been applied to uncertain programming [21], facility location problem [7], stock model [6], product control problem [16], and so on.
Based on the uncertainty theory, Zhu [29] proposed an uncertain optimal control in 2010, and gave an optimality equation as a counterpart of Hamilton-Jacobi-Bellman equation. After that, there are a lot of uncertain optimal control problems having been solved. For example, Sheng and Zhu [18] studied an optimistic value model of uncertain optimal control problem and proposed an equation of optimality to solve the model, Yan and Zhu [23] established a linear-quadratic control problem for discrete-time switched systems with subsystems perturbed by uncertainty and the analytical expressions are derived for both the optimal objective function and the optimal switching strategy, Shu and Zhu [19] considered an optimal control problem for an uncertain continuous-time singular system and obtained the equation of optimality for the uncertain singular system.
It is worth being mentioned that some constraints are of considerable importance in many practical problems [3,9], so the constrained LQ issue has a concrete application background. Thus, some researchers discussed stochastic LQ optimal problems with indefinite control weights and constraints [27,15]. Inspired by the stochastic indefinite LQ control control and uncertain optimal control, we propose an indefinite LQ optimal control with process state inequality constraints for discrete-time uncertain systems, in which the state and control in dynamics depend on linear uncertain noises. This control model comes from the practical problems where the noise disturbances are lack of observation data to get probability distribution. We have to invite some domain experts to give the belief degree that the noises happen. In this situation, it is more suitable to choose uncertain variable as the noises in the control model. To the best of our knowledge, this problem has not been investigated in the literature and remains open.
The organization of this paper is as follows. In section 2, we give some definitions about uncertainty theory. Section 3 presents an indefinite LQ optimal control with inequality constraints and gives a necessary condition for the existence of optimal controllers. Section 4 shows that the solvability of the constrained difference equation is sufficient for the well-posedness of the indefinite LQ problem. Section 5 applies the result to a numerical example. Section 6 presents the main conclusions.
For convenience, we adopt the following notations. R n is the real n-dimensional Euclidean space; R m×n the set of all m × n matrices; M τ the transpose of a matrix M ; and tr(M ) the trace of a square matrix M . Moreover, M > 0 (resp. M ≥ 0) means that M = M τ and M is positive (resp. positive semi-) definite.

2.
Preliminaries. In this section, we will review some basic concepts and results in uncertainty theory and Moore-Penrose inverse of a matrix.
2.1. Some concepts about uncertainty theory. Uncertain measure [12] M is a real-valued set-function on a σ-algebra L over a nonempty set Γ satisfying the following axioms: The triplet (Γ, L, M) is called an uncertainty space. In order to obtain an uncertain measure of compound event, a product uncertain measure was defined by Liu [14] as follows.
Axiom 4. (Product Axiom) Let (Γ k , L k , M k ) be uncertainty spaces for k = 1, 2, · · · . Then, the product uncertain measure M on the product σ-algebra satisfies where Λ k are arbitrarily chosen events from L k for k = 1, 2, · · · , respectively. Definition 2.1. (Liu [12]) An uncertain variable is a function ξ from an uncertainty space (Γ, L, M) to the set R of real numbers such that for any Borel set B of real numbers, the set {ξ ∈ B} = {γ ∈ Γ|ξ(γ) ∈ B}, is an event in L.
Definition 2.2. (Liu [12])The uncertainty distribution Φ of an uncertain variable ξ is defined by Φ(x) = M{γ ∈ Γ | ξ(γ) ≤ x}, for any real number x. Example 1. (Liu [12]) An uncertain variable ξ is called linear if it has a linear uncertainty distribution denoted by L(a, b) where a and b are real numbers with a < b.

YUEFEN CHEN AND YUANGUO ZHU
Definition 2.4. (Liu [12]) Let ξ be an uncertain variable. Then the expected value of ξ is defined by provided that at least one of the two integrals is finite.
Lemma 2.5. (Liu [13]) Let ξ be an uncertain variable with regular uncertainty distribution Φ. If the expected value exists, then Example 2. Let ξ ∼ L(a, b) be a linear uncertain variable. Then its inverse uncertainty distribution is Φ −1 (α) = (1 − α)a + αb, and its expected value is Independence is an important concept in uncertainty theory.
3.1. Problem setting. Consider the indefinite LQ optimal control with process state inequality constraints for discrete-time uncertain systems as follows: subject to where λ k ∈ R and |λ k | ≤ 1. The vector x k ∈ X(Γ, R n ) is a state vector with the initial state x 0 ∈ R n and u k ∈ U (R n , U k ) is a control vector subject to a constraint set U k ⊂ R m , where X(Γ, R n ) denotes the function space of all uncertain vectors (measurable functions from Γ to R n ), and U (R n , U k ) denotes the function space of all functions from R n to U k . In this paper we assume and B 0 , B 1 , · · · , B N −1 are matrices with appropriate dimensions determined from the context. Besides, the noises ξ 0 , ξ 1 , · · · , ξ N −1 are independent linear uncertain variables L(−1, 1). Notice that the difference equation (1) means that the state x k+1 in the (k + 1)th stage is linear with the state x k in the kth stage and the control u k with the disturbance λ k (A k x k + B k u k )ξ k . Generally speaking, the disturbance is not larger than the main part of the system. For the convenience of computation in the sequel, the disturbance is described by an uncertain variable ξ k (noise) associated with the amplitude λ k (A k x k + B k u k ) which guarantees the disturbance is not larger than the main It is observed that the state and control weighting matrices in the objective functional are not required to be definite. Therefore problem (1) is called indefinite LQ optimal control problem. Next we give the following definitions.
is called an optimal control sequence. 3.2. Equivalent deterministic problem. In this subsection, we will transform the indefinite LQ optimal control problem (1) into an equivalent deterministic optimal control problem with constraints. Let Since state x k ∈ R n , x k x τ k is a n × n matrix which elements are uncertain variables, and X k is a symmetric deterministic matrix (k = 0, 1, · · · , N ). Denote C = (C 0 , C 1 , · · · , C N −1 ), where C k is matrix for k = 0, 1, · · · , N − 1.
where C 0 , C 1 , · · · , C N −1 are constant matrices, then it is equivalent to the following deterministic optimal control problem subject to (2) Proof. Assume the indefinite LQ problem (1) is solvable by a feedback control It is obvious that λ k U k = 2V k . Note that since ξ k and ξ 2 k are not independent, we will calculate E[U k ξ k + V k ξ 2 k ] as follows.
Combined with case (i) and case (ii), it is concluded that Substituting (4) into (3), we have The associated cost function is expressed equivalently as and the constraints E[x τ k x k ] ≤ c k (k = 1, · · · , N ) becomes tr (X k ) ≤ c k , f or k = 1, · · · , N.

3.3.
A necessary condition for state feedback control. The next theorem shows that a necessary condition for the optimal linear state feedback control with deterministic gains of the indefinite LQ problem (1) can be obtained by applying the maximum principle with mixed inequality constraints [8].
Proof. Assume the uncertain LQ problem (1) is solvable by where the matrices C 0 , C 1 , · · · , C N −1 are viewed as the control to be determined.
We know that C 0 , C 1 , · · · , C N −1 are also the optimal solution of problem (2). Thus, we consider the deterministic optimal control problem (2) as follows. Denote The Lagrangian function is defined as follows where γ k ≥ 0 ∈ R (k = 1, 2, · · · , N ) are the Lagrangian multiplier of the inequality constraints g k (X k ) ≤ 0 (k = 1, 2, · · · , N ), and the matrices H k+1 (k = 0, 1, · · · , N − 1) are the Lagrangian multipliers of the equality constraints h k+1 (X k , C k ) = 0 (k = 0, 1, · · · , N − 1). By means of maximum principle with mixed inequality constraints [8], we have and the complementary clackness condition Based on the partial rule of gradient matrices [1], (11) can be transformed into Let Then we have L k L + k M k = M k and L k C k + M k = 0. Applying Lemma 2.8, the general solution of (14) is given by for k = 0, 1, · · · , N − 1. By (12), firstly we have for k = N , which leads to Secondly, we obtain Substituting (16) into (19), it follows The associated objective functional , by applying (18) and (20), we obtain A completion of square implies Substituting (13) into (21) yields that Next, we prove that L k must satisfy If it is not so, there is a L p for p ∈ {0, 1, · · · , N − 1} with a negative eigenvalue λ. Denote the unitary eigenvector with respect to λ as v λ (i.e., v τ λ v λ = 1 and

Special cases.
We have obtained that L k ≥ 0 in the constrained difference equation (9) of Theorem 3.4. The following corollaries are special cases of the above result if we let L k > 0 and L k = 0, respectively.

Corollary 1. The indefinite LQ problem (1) is uniquely solvable if and only if
L k > 0 for k = 0, 1, · · · , N − 1. Moreover, the unique optimal control is given by u k = −L −1 k M k x k , f or k = 0, 1, · · · , N − 1. Proof. By using Theorem 3.4, we immediately obtain the corollary. Corollary 2. If L k = 0 for k = 0, 1, · · · , N − 1, then any admissible control of the indefinite LQ problem (1) is optimal and the constrained difference equation (9) reduces to the following linear system Proof. Firstly, letting L k = 0 in (9), then the constrained difference equation (9) reduces to the linear system (24). Secondly, letting L k = 0 in (21), it is shown that c k γ k for any admissible control. Then any admissible control of the indefinite LQ problem (1) is optimal.
Next we consider the following indefinite LQ control problem with equality constraint on the terminal state as follows: subject to Corollary 3. If the indefinite LQ problem (25) is solvable by a feedback control sequence where C 0 , C 1 , · · · , C N −1 are constant matrices, then there exist symmetric matrix H k and a µ ∈ R, such that for k = 0, 1, · · · , N − 1. Moreover with Y k ∈ R m×n , k = 0, 1, · · · , N − 1, being any given matrices. Furthermore, the optimal cost of the indefinite LQ problem (25) is Proof. Let tr [X N ] = c in the problem (1) and µ be the lagrange multiplier of this equality constraint in Theorem 3.4. According to Theorem 3.4, we known that g(X N ) = tr [X N ]−c = 0. By the similar process as in Theorem 3.3 and Theorem 3.4, we obtain the corollary directly.
4. Solvability and well-posedness of the indefinite LQ problem.

4.1.
Sufficiency of the the constrained difference equation. In this section, it is shown that the solvability of the constrained difference equation (9) is sufficient for the solvability of the indefinite LQ problem (1). Moreover, any optimal control can be obtained via the solution of the constrained difference equation (9).

4.2.
Well-posedness of the indefinite LQ problem. Next we will discuss the well-posedness of the indefinite LQ problem (1).

4.3.
General expression for the optimal control set. In the following, we present a general expression for the optimal control set based on the solution of the constrained difference equation (9).
Assume that H k (k = 0, 1, · · · , N − 1) and γ k ≥ 0 ∈ R (k = 1, 2, · · · , N ) solve the constrained difference equation (9). A sufficient and necessary condition that u k is in the set of all optimal feedback controls for indefinite LQ problem (1) is that where Y k ∈ R m×n and Z k ∈ R m , k = 0, 1, · · · , N − 1, are arbitrary variables with appropriate size.
We see thatũ k solves the following equation By using Lemma 2.8 with L = L k , M = I, N = −L k L + k M k x k , it is easy to verify that LL + N M M + = N.
We obtain the solution of (35) with (16), the optimal control can be represented by 5. Numerical example. In this section, we report our numerical experiments based on the approach developed in the previous sections. In the constrained indefinite LQ control problem for discrete-time uncertain systems, we give out a set of specific parameters of the coefficients as follows: The state weights and the control weights are as follows According to the necessary condition of Theorem 3.4, we obtain γ k [tr(X k ) − c k ] = 0, γ k ≥ 0 and tr(X k ) ≤ 0, k = 1, 2, Firstly, we have Secondly, we solve the equations (36) as the following four cases: (i) γ 1 = 0, γ 2 = 0; (ii) γ 1 = 0, γ 2 > 0; (iii) γ 1 > 0, γ 2 = 0; (iv) γ 1 > 0, γ 2 > 0. The analysis of above cases show that there are no feasible solution about X 1 and X 2 in cases (i), (iii) and (iv). We can obtain a group of feasible solution about X 1 and X 2 by solving out γ 1 = 0 and γ 2 = 2 in case (ii), which are given below by X 1 = 0 0 0 1.5 , X 2 = 2.25 0 0 0 .
Then we obtain the terminal condition In order to find the optimal controls and optimal cost value in this example, we will construct the optimal feedback control law by u k = C k x k (k = 1, 0) stage by stage in reverse order.
Stage 2: For k = 1, we obtain The optimal feedback control is u 1 = C 1 x 1 where The optimal feedback control is u 0 = C 0 x 0 where C 0 = −L −1 0 M 0 = (15, 0). Finally, the optimal cost value is c k γ k = −1.
Remark 5. Note that in this example, the state weight Q 0 is negative definite, Q 1 is negative semidefinite, and Q 2 is positive semidefinite, the control weight R 1 is negative definite.
6. Conclusion. This paper has investigated the constrained indefinite LQ control for discrete-time systems with state and control dependent on uncertain noise. We first transform the indefinite LQ control problem into an equivalent deterministic optimal control problem. Then, we present a necessary condition for the existence of optimal linear state feedback control by means of the maximum principle with mixed inequality constraints. Moreover, the existence of a solution to the constrained difference equation is equivalent to the solvability of the indefinite LQ problem. Furthermore, the well-posedness of the indefinite LQ problem is discussed. For future work, we will consider indefinite LQ control in infinite time horizon for discrete-time uncertain systems.