ON THE GLOBAL OPTIMAL SOLUTION FOR LINEAR QUADRATIC PROBLEMS OF SWITCHED SYSTEM

. The global optimal solution for the optimal switching problem is considered in discrete time, where these subsystems are linear and the cost func- tional is quadratic. The optimal switching problem is a discrete optimization problem. Complete enumeration search is always required to ﬁnd the global optimal solution, which is very expensive. Relaxation method is an eﬀective method to transform the discrete optimization problem into the continuous optimization problem, while the optimal solution is always not the feasible solution of the discrete optimization problem. In this paper, we propose a special class of relaxation method to transform the optimal switching problem into a relaxed optimization problem. We prove that the optimal solution of this modiﬁed relaxed optimization problem is exactly that of the optimal switching problem. Then, the global optimal solution can be obtained by solving the continuous optimization problem easily. Numerical examples are demonstrated to show that the modiﬁed relaxation method is eﬃcient and eﬀective to obtain the global optimal solution.


(Communicated by Saroj Biswas)
Abstract. The global optimal solution for the optimal switching problem is considered in discrete time, where these subsystems are linear and the cost functional is quadratic. The optimal switching problem is a discrete optimization problem. Complete enumeration search is always required to find the global optimal solution, which is very expensive. Relaxation method is an effective method to transform the discrete optimization problem into the continuous optimization problem, while the optimal solution is always not the feasible solution of the discrete optimization problem. In this paper, we propose a special class of relaxation method to transform the optimal switching problem into a relaxed optimization problem. We prove that the optimal solution of this modified relaxed optimization problem is exactly that of the optimal switching problem. Then, the global optimal solution can be obtained by solving the continuous optimization problem easily. Numerical examples are demonstrated to show that the modified relaxation method is efficient and effective to obtain the global optimal solution.
In the past decades, the optimal control has been a hot research topic in various hybrid systems, we refer the readers to [12,13,21,22,29]. As a special class of optimal control problems, the optimal switching problem aims to find a switching law such that the objective function is optimal. It has been studied in the relevant literature. Some of theoretical results include the model formulation in [16] and the maximum principle proposed in [19]. And many computational results are also developed, which mainly focus on the optimization of the switching times for a fixed switching sequence. For example, the gradient descent method was proposed in [1,3,24] and control parameterization enhancing method (CPET) with the timescaling transformation was studied in [17,20,23,26]. If the switching times are fixed, the optimization of switching sequence becomes very difficult. This problem is a discrete optimization problem. The CPET method can also be used to find the local optimal solution [7,8], where the problem becomes more complicated and unstable. In [2,27], the embedding transformation with the relaxation method was explored, in which the switching frequency is infinite.
The decision of switching sequence is NP complete to find the global optimal solution. Basically, it requires a complete enumeration search, which has an exponential computational complexity, or some heuristic techniques mentioned by [14,28] to obtain the global optimal solution. Branch and bound method is also an efficient method to obtain the global optimal solution in [4,5,25], where the optimal switching problem was considered. The efficiency depends on the precision of lower bound and the number of switching. In [6], a discrete filled function method is proposed to find the global optimal solution based on the discrete local search method. Note that the decision of global optimal switching sequence is very expensive for general optimal switching problems. It is difficult to find a general efficient method for all optimal switching problems. Hence, we can consider a special class of optimal switching problem, where the global optimal solution can be derived easily. Since the linear quadratic problem is very common used in optimal control applications, we consider the global optimal solution of this class of optimal switching problem in this paper.
The relaxation method is an effective technique to transform the discrete optimization problem into the continuous optimization problem. Then, the gradient based algorithm could be applied to solve the continuous optimization problem in [9]. However, the optimal solution of the relaxed optimization problem is always not the optimal solution of the original discrete optimization problem. For this, we aim to find a suitable relaxation method such that the optimal solution of the relaxed optimization problem is still the optimal solution of the original discrete optimization problem. In this paper, we consider the optimal switching problem and propose a modified relaxation method such that the global optimal solution can be obtained by solving the relaxed continuous optimization problem.
The rest of this paper is organized as follows. In Section 1, the optimal switching problem is formulated, where the subsystems are linear and the cost functional is quadratic. In Section 2, we first introduce the general relaxation method to solve this problem, where the optimal solution can not be assured. Then, we propose a modified relaxation method to solve this problem such that the global optimal solution can be obtained. In Section 3, the global optimal solution of the modified relaxation method is analyzed. Numerical examples are demonstrated in Section 4 to show the efficiency and effectiveness of this method.
1. Problem formulation. We consider the discrete-time switched system with N linear subsystems as follows: where I = {0, 1, . . . , T − 1} and T is the terminal time. For any t ∈ I, A i (t) ∈ R n×n , i = 1, 2, . . . , N . The state x(t) = (x 1 (t), x 2 (t), · · · , x n (t)) is an n-dimensional vector with initial condition A switching sequence of the switched system is denoted by where for any t ∈ I, σ(t) ∈ {1, 2, · · · , N }. That is, σ(t) = i means that the i-th subsystem is active at time t. Hence, the dynamic system is given by Then, the optimal switching problem aims to find a switching sequence σ such that the cost functional is minimized. We formulate the optimal switching problem as follows: Since σ(t) is discrete taking value from the index set {1, 2, . . . , N }, ∀t ∈ I, Problem 1 is a discrete optimization problem. Apparently, the number of all feasible switching sequences is N T , which increases exponentially as T grows. Hence, this problem is an NP-complete problem which is very difficult to obtain the global optimal solution. Remark 1. These subsystems in Problem 1 can be generalized as where b i (t) ∈ R n . By introducing some new symbols as follows which is the same as Problem 1.

Remark 2.
If the cost functional in Problem 1 is given by: where for each t ∈ I, Q(t) ∈ R n×n , c(t) ∈ R n . Then, by introducing Hence, the optimal switching problem with the generalized cost functional can also be transformed into the same form of Problem 1.
2. Method analysis. The optimal switching problem is an NP-complete problem in discrete time. There is currently no method which can solve it efficiently. In general, the enumeration method can obtain the global optimal solution, while the cost is very expensive. For the convenience of computing, one often transforms this discrete optimization problem into the continuous case through the relaxation method.

Relaxation method.
To apply the relaxation method to transform Problem 1 into a continuous optimization problem, the weight functions are introduced as where the weight function of the i-th subsystem w i : I → [0, 1], i = 1, . . . , N . For each time t ∈ I, these weights subject to the following conditions: Then, we formulate the relaxed dynamic system by Let the weight vector be w(t) = (w 1 (t), · · · , w N (t)) , which is composed of N weight functions at time t. Denote a set W by then, for any t ∈ I, w(t) ∈ W.
Hence, these weight functions can be uniformly denoted by We formulate the relaxed optimization problem as follows: Problem 2 is an optimal control problem in discrete time, where w ∈ W T is the control vector function. Moreover, noting that w is continuous, Problem 2 can be considered as a continuous optimization problem, which can be solved by any gradient based method.
However, the optimal solution of Problem 2 may not be the feasible solution of Problem 1, which can be seen in the example section. This restricts the use of the relaxation method.

2.2.
Modified relaxation method. Note that the optimal solution can not be obtained by the relaxation method above in general. To obtain the global optimal solution of Problem 1, we should transform it into an equivalent optimization problem. Denote where, for each t ∈ I, X(t) ∈ R n×n . Then, we have It can be seen that the cost functional in Problem 1 becomes linear. Thus, we can set X(t) as the new state variables of the system. For this, the i-th dynamic subsystem is given by Hence, Problem 1 can be transformed into the following form: For Problem 3, we transform the quadratic optimal switching problem into the equivalent linear problem. Then we introduce the weight vector function w ∈ W T , and formulate the linear relaxation optimization problem as follows: Problem 4 is a continuous optimization problem. Furthermore, its optimal solution is also the global optimal solution of Problem 1, which can be seen as the analysis below.
3. Global optimal analysis. We need to prove that the optimal solution of Problem 4 is also the optimal solution of Problem 1. First, we show the equivalence between Problem 1 and Problem 3 as follows.
3.1. Equivalence between Problem 1 and Problem 3. We introduce the following lemma.
Lemma 3.1. For each switching sequence σ, if the initial state of Problem 1 is x 0 , and the initial state of Problem 3 is X 0 = x 0 x 0 , then, we have Proof. We use the induction method. First, for the case t = 0, (17) is obvious followed by (15). Suppose that (17) is true when t = k, that is, Then, for the case t = k + 1, it follows by (6) that we have Hence, By (14) and (18), we have Then, (17) is also true when t = k + 1. Thus, we obtain (17). The proof is complete.
By Lemma 3.1, we have the theorem below. Proof. Suppose that σ 1 is an optimal solution of Problem 1, and σ 2 is an optimal solution of Problem 3. By Lemma 3.1, we have Since σ 2 is an optimal solution of Problem 3, we have On the other hand, it follows by Lemma 3.1 that we have Since σ 1 is an optimal solution of Problem 1, we obtain Hence, we haveĴ (σ 1 ) = J(σ 1 ) =Ĵ(σ 2 ) = J(σ 2 ).
The proof is complete.
Hence, it follows by Theorem 3.2 that the optimal solution of Problem 1 can be obtained by solving Problem 3, where the cost functional in Problem 3 becomes linear.
3.2. Global optimal solution. Next, we show that the global optimal solution of Problem 1 or Problem 3 can also be obtained by solving Problem 4. For this, we introduce the lemma below. Lemma 3.3. Suppose that u = (u(1), u(2), · · · , u(N )) ∈ W, and Then, there is only one index i ∈ {1, 2, · · · , N } such that Suppose that for any index i, u(i) = 1, then u(i) = 0, ∀i, and we have which is a contradiction. Suppose that there are two or more than two indices such that the components of u equal to 1, then which is a contradiction. Therefore, there is only one index i such that u(i) = 1, and the values of the other indices are 0. This proof is complete.
Then, we have the following theorem.
Theorem 3.4. There exists an optimal solution w * of Problem 4 such that w * i (t) ∈ {0, 1}, ∀t ∈ I, ∀i ∈ {1, · · · , N }. Proof. Since X(t) is a symmetric matrix function, the upper triangular part of X(t) can be treated as the state variable. That is, we denote Then, Problem 4 is equivalent to the following problem: and y 0 are derived from (13), (14) and (15), respectively. By (19) and (20), we have Then, the cost functional of Problem 4 is rewritten as: Assume that any optimal solution w * of this problem can not take value at {0, 1} for any time. Without loss of generality, we assume that w * does not take value at {0, 1} at time t 1 . Then, we fix the values w * at all time t = t 1 and consider w * (t 1 ) as the decision variable. That is, we only consider N decision variables (w 1 (t 1 ), w 2 (t 1 ), w 3 (t 1 ), · · · , w N (t 1 )). Note that the cost functional (21) becomes linear with respect to w(t 1 ) and it can be rewritten as follows: J r (w(t 1 )) = a 1 w 1 (t 1 ) + a 2 w 2 (t 1 ) + · · · + a N w N (t 1 ) + c, where a 1 , a 2 , · · · a N are the coefficients which are derived from (21), and c is a constant. Hence, this problem becomes a linear programming problem as follows: By linear programming theory, the optimal solution of linear programming problem can only take value at the boundary of the feasible set. Note that the boundary of the set [0, 1] is {0, 1}. Then, the optimal solution of this problem can take value at {0, 1}, which is denoted by w * * (t 1 ). Hence, the solution w = (w * (0), · · · , w * (t 1 − 1), w * * (t 1 ), w * (t 1 + 1), · · · , w * (T − 1)) is better than or equals to any optimal solution w * . This contradicts to the assumption that any optimal solution can not take value at {0, 1} at any time.
Thus, there exists an optimal solution w * of Problem 3 such that all its elements equal to 0 or 1. The proof is complete.
By Theorem 3.4, it is feasible to transform Problem 3 into a continuous relaxed optimization problem, and the optimal switching sequence can be obtained by solving the relaxed optimization problem. The equivalence between the optimal solutions of Problem 3 and 4 is stated as follows.
Theorem 3.5. There exists at least one optimal solution of Problem 4, which corresponds to an optimal solution of Problem 3.
Proof. By Theorem 3.4, there exists an optimal solution w * of Problem 4, such that w * i (t) ∈ {0, 1}, ∀t ∈ I, ∀i = 1, · · · , N. Note that w * (t) ∈ W, it follows by Lemma 3.3 that there is only one index i such that w * i (t) = 1, w * j (t) = 0, ∀j = i. That is, only one subsystem is active at each time. Then, we can define the switching sequence σ * as σ * (t) = i, if w * i (t) = 1. The switching sequence σ * is a feasible solution of Problem 3. Suppose that the optimal solution of Problem 3 is σ * * (t), then, by the optimality of Problem 3, we haveĴ (σ * ) ≥Ĵ(σ * * ). In addition, σ * * corresponds to a solution w * * in Problem 4. Note that w * is the optimal solution of Problem 4, we havê Hence, we obtainĴ (σ * ) =Ĵ(σ * * ). That is, σ * is an optimal solution of Problem 3. The proof is complete. x (t)Q j (t)x(t) ≤ 0, j = 1, 2, · · · , q in Problem 1, where Q j (t) ∈ R n×n . Then, the constraints are also included in the relaxed optimization problem, and Theorem 3.4 and Theorem 3.5 still hold, if the feasible set is not empty. The proof is similarly to those given in Theorem 3.4 and Theorem 3.5.
4. Numerical example. In this section, we apply the proposed method to find the global optimal solution of the numerical example, in which these weights are treated as control variables. For the optimal control software, Miser 3 [26] is an efficient and mature toolbox to solve the optimal control problem. Hence, we apply Miser 3 to solve Problem 2 and 4. All codes are written in MATLAB 7.0 and implemented on a Laptop with a 2.1 GHz CPU and 3.0 GB RAM.
For the following two examples, there are three subsystems in the switched system, which are given by: Example 1: Consider these subsystems with the parameters given by The initial condition of the state is and the cost functional is given by where the terminal time T = 12 and the parameter matrix is To exactly find the global optimal switching sequence, we first apply the enumeration method to search all scenarios, where the number of all feasible switching sequences is 3 12 ≈ 5.31×10 5 . It takes about 58.02 seconds to obtain the global optimal value 2.33 and the global optimal switching sequence {1, 3, 1, 2, 2, 1, 1, 1, 3, 1, 2, 2}. In order to compare the searching cost of two relaxation methods, we apply the general relaxation method to solve this problem, where the problem is relaxed as Problem 2. The optimal value is obtained as 0.03 and the corresponding optimal weights of subsystems are depicted in Figure 1, 2, and 3, respectively. It can be seen that the weights are always not 0 or 1. We can not determine the active subsystem at each time. Hence, we truncate this solution by setting Then, we obtain the truncated switching sequence depicted in Figure 4. The corresponding cost functional value is 2.07887 × 10 4 , which is not optimal.
Next, we implement the modified relaxation method to find the optimal solution, where the problem is relaxed as Problem 4. That is, we transform this problem into     T r(Q 0 (t)X(t)), Its running time is only 1.02 seconds. The optimal value is finally obtained as 2.33 and the corresponding optimal weights are depicted in Figure 1, 2, and 3. It can be seen that all the weights are 0 or 1 at each time. So we can activate the subsystem when the corresponding weight is equal to 1 at each time. Then we obtain a complete switching sequence which is depicted in Figure 4. This is consistent with the consequence obtained by the enumeration method. It shows that our proposed method is more efficient and effective to find the global optimal solution.  The initial condition of the state is and the cost functional is given by where the terminal time T = 12 and the parameter matrix is It takes about 73.52 seconds for the enumeration method to obtain the global optimal value 4.8161 and the global optimal switching sequence {3, 3, 3, 2, 3, 2, 3, 2, 2, 2, 2, 2}. We apply the relaxation method to solve this problem, where the problem is relaxed as Problem 2. The optimal value is obtained as 4.1733 and the corresponding optimal weights of subsystems are depicted in Figure 5, 6, and 7, respectively. It can be seen that the values are always not 0 or 1. We can not obtain the corresponding switching sequence. Hence, we truncate this solution by setting The truncated switching sequence is depicted in Figure 8 and the corresponding cost functional value is 5.1081. Obviously, it is not optimal.
To determine the optimal solution of this problem efficiently, we apply the modified relaxation method to solve this problem, where the problem is relaxed as Problem 4. That is, we transform this problem into the following problem min w∈W TĴ r (w) = T t=0 T r(Q 0 (t)X(t)), It takes only 1.28 seconds to obtain the optimal value as 4.8161. We depict the corresponding optimal weights in Figure 5, 6, and 7. It can be seen that all the values are 0 or 1 and we depict the corresponding switching sequence in Figure 8. We find that it is the global optimal solution verified by the enumeration method. Thus, the modified relaxation method is very efficient and effective to find the global optimal solution of time-varying switched system.

5.
Conclusion. In this paper, we have considered the global optimal solution of the optimal switching problem for linear switched systems in discrete time, where the cost functional is quadratic and these subsystems are linear. The general relaxation method can be used to solve this problem, where the optimal solution of the relaxed problem is not necessarily a solution of the optimal switching problem. For this, we        propose a modified relaxation method to transform the optimal switching problem into a new relaxed optimization problem. The equivalent relation of the optimal solution between the original problem and the transformed problem has also been established. Then, the global optimal solution of the optimal switching problem can be solved by solving a continuous optimization problem easily. Two numerical examples have been implemented to demonstrate the efficiency and effectiveness of the proposed method.