A LINEAR-QUADRATIC CONTROL PROBLEM OF UNCERTAIN DISCRETE-TIME SWITCHED SYSTEMS

This paper studies a linear-quadratic control problem for discretetime switched systems with subsystems perturbed by uncertainty. Analytical expressions are derived for both the optimal objective function and the optimal switching strategy. A two-step pruning scheme is developed to efficiently solve such problem. The performance of this method is shown by two examples.

1. Introduction.Many practical systems operate by switching between different subsystems or modes.They are called switched systems.The optimal control problems of switched systems arise naturally when the control systems under consideration have multiple operating modes.An example of a switched system is a switched-capacitor DC/DC power converter [17] which operates by switching between different circuit topologies so as to produce a steady output voltage.A powertrain system [20] can also be viewed as a switched system which needs switching between different gears to achieve an objective such as fast and smooth acceleration response to the driver's commands, low fuel consumption and low levels of pollutant emissions.By taking the feeding of glycerol as a continuous-time process, Reference [16] introduces a controlled nonlinear multistage dynamical system to describe the microbial fed-batch culture.Subjected to this system, the objective is to maximize the concentration of 1, 3-PD at the terminal time.Thus, this practical problem is converted to an optimal control model of switched systems.Other examples of optimal control problems of switched systems include sensor scheduling [21], air traffic control [19] and cognitive radio networks [10].
For switched systems, the aim of optimal control is to seek both the optimal switching law and the optimal continuous input for optimizing a certain performance criterion.Many successful algorithms have already been developed to seek the optimal control of switched systems.It is worth mentioning that Xu and Antsaklis [23] propose a two-stage optimization strategy for the optimal control of continuoustime switched systems.Stage (a) is a conventional optimal control problem that finds the optimal cost given the sequence of active subsystems and the switching 268 HONGYAN YAN, YUN SUN AND YUANGUO ZHU instants and stage (b) is a constrained nonlinear optimization problem that finds the local optimal switching instants.A general continuous-time switching problem is investigated in [2] based on the maximum principle and an embedding method.Furthermore, Teo et al [18,9] propose a control parameterization technique and the time scaling transform method to find the approximate optimal control inputs and switching instants, which have been used extensively.As for discrete-time models, Bemporad et al [1] study the solution to optimal control problems for discrete time linear hybrid systems.Borrelli et al [3] describe an off-line procedure to synthesize optimal control laws based on the minimization of quadratic and linear performance indices subject to linear constraints on inputs and states.The procedure is based on a combination of dynamic programming and multiparametric quadratic programming.There is no doubt that the quadratic control of switched systems represents one of the most important optimal control problems of switched system.Zhang et al [26] study the discrete-time linear quadratic control problem for switched linear systems (LQCS) based on the dynamic programming approach.Problems of LQCS with a constant switching cost are investigated in [7] based on an efficient branch and bound algorithm.Lincoln and Rantzer [11] derive a sub-optimal control law for LQCS with an infinite horizon.Moreover, motivated by the problems of viral mutation in HIV infection, reference [5] considers discrete-time control for switched positive systems.
Up to now, a majority of methods for optimal control problems of switched systems are based on deterministic models for the subsystems dynamic.However, indeterminacy is ubiquitous in realistic system models.As a consequence, the controller may not be able to serve its purpose or even cause systems to be inactive due to the influence of this indeterminacy.Therefore, it is necessary to discuss linear quadratic optimal control problems of uncertain switched system.Most literature characterize the involved uncertainty as randomness and implement optimal schemes in the stochastic environment.For example, Zhang et al [27] describe stochastic models and designed a control strategy to meet certain design criteria in the probabilistic sense and Wu et al [22] investigate the problem of sliding mode control of a continuous-time switched stochastic system.In reality, due to lack of historical data, however, in many cases, no samples are available to estimate a probability distribution and we have to invite some domain experts to evaluate the degree of belief that each event will happen.As noted by the Nobelist Kahneman and his partner Tversky [8], humans tend to overweight unlikely events.Thus, the degree of belief may have a much larger range than the true frequency as a result.In this case, probability theory does not work [12] and in order to rationally deal with degree of belief, uncertainty theory was founded in 2007 [13].Nowadays, uncertainty theory has become a branch of axiomatic mathematics for modeling degree of belief [14].Theory and practice have shown that uncertainty theory is an efficient tool to deal with some non-deterministic information, such as expert data and subjective estimations, which appears in many practical problems.During the past seven years, there have been many achievements in uncertainty theory, such as uncertain programming, uncertain statistics, uncertain logic, uncertain inference, and uncertain process.Moreover, uncertainty theory has been applied in many fields, including facility location problems [6], project scheduling problems [28] and portfolio selection problems [29].For the purpose of dealing with an uncertain optimal control problem with an uncertain process, Zhu [29] proposed and studied an uncertain optimal control problem in 2010.
For continuous-time switched systems with subsystems perturbed by uncertainty, an optimal control problem is investigated in [24].The aim of such problem is to seek both the switching instants and the optimal continuous input for optimizing a certain performance criterion.For solving the model, we decompose it into two stages.The first stage is to seek the minimum value of the cost function under fixed switching instants and in the second stage the modified golden section method is used to solve optimization problems.Additionally, [25] studied a bang-bang control model with optimistic value criterion for uncertain switched systems.However, many real-world switched systems are discrete-time.For example, the problem of treatment scheduling to minimize the adverse effects of virus mutation in HIV [5] is discrete-time.Moreover, sometimes we need to discretize continuous-time switched systems so as to solve our problems as in [20,4].Therefore, we consider the problem of quadratic optimal control for an uncertain discrete-time linear switched system.
The remainder of this paper is organized as follows.In the next section, some basic concepts of uncertainty theory are reviewed.In Section 3, a linear-quadratic control problem of the uncertain discrete-time switched systems to be studied in this paper will be presented.By using a dynamic programming approach, the analytical solutions of the optimal objective function and the control strategy which are characterized by a sequence of sets of ordered pairs are derived in Section 4. In order to improve the computational efficiency, a two-step pruning scheme is presented to remove as many redundant pairs of matrices as possible in Section 5. Finally, the approach is tested through two numerical examples in Section 6.
Throughout the paper, we use notation Q ≥ 0 (Q > 0) to denote a positive semidefinite (PSD) matrix (positive definite matrix), S n + (S n ++ ) is the set of all n×n positive semidefinite matrices (positive definite matrices), x 2 S is the quadratic term x τ Sx with S being symmetric and positive semidefinite and |H| is the number of the elements in the finite set H.

Preliminary of uncertainty theory.
In this section, we will state some basic concepts in uncertainty theory [13,14].Definition 2.1.(Liu [13]) Let Γ be a nonempty set, and L a σ-algebra over Γ.Each element A ∈ L is called an event.A set function M defined on the σ-algebra The triplet (Γ, L, M) is called an uncertainty space.An uncertain variable is a measurable function from an uncertainty space (Γ, L, M) to the set R of real numbers, and an uncertain vector is a measurable function from an uncertainty space to R n .In order to describe an uncertain variable, the concept of an uncertainty distribution is defined as follows: the uncertainty distribution Φ : R → [0, 1] of an uncertain variable ξ is defined by Φ(x) = M{ξ ≤ x} for any real number x. Definition 2.2.(Liu [13]) An uncertain variable ξ is called linear if it has a linear uncertainty distribution where a and b are real numbers with a < b.Such a linear uncertain variable is denoted by ξ ∼ L(a, b).
The expected value [13] of an uncertain variable ξ is defined by provided that at least one of the two integrals is finite.
We have the following conclusions about the expected value.
Lemma 2.3.(Liu [13]) Let ξ be an uncertain variable with regular uncertainty distribution Φ.If the expected value exists, then ) be a linear uncertain variable.Then its inverse uncertainty distribution is Φ −1 (α) = (1 − α)a + αb, and its expected value is The expected value of monotone function f (ξ) can be obtained by the uncertain distribution of ξ from the above theorem.A scheme is introduced for establishing the uncertainty distribution of function f (ξ) directly from the distribution of ξ without the monotonicity of f (x) for a common uncertain vector ξ in [30], where a common uncertain variable is defined as follows.
Let B be the Borel algebra over R, C be the collection of all intervals of the form (−∞, a], [b, +∞), ∅ and R. The uncertain measure M is provided in such a way: first, Definition 2.5.(Zhu [30]) An uncertain variable ξ with distribution Φ(x) is common if it is from the uncertainty space (R, B, M) to R defined by ξ(γ) = γ, where B is the Borel algebra over R and M is defined by (2).
Lemma 2.6.(Zhu [30]) Let ξ be a common uncertain variable with the continuous distribution Φ(x).For real numbers b and c, denote Then the distribution of the uncertain variable ξ 2 + bξ + c is Example 2. (Zhu [30]) Let ξ be a common linear uncertain variable L(−1, 1).Then 3. Problem formulation.Considering the following class of uncertain discretetime linear switched systems consisting of m subsystems.
where (i) for each where, for any i ∈ M , Q i ≥ 0, R i > 0 and (Q i , R i ) constitutes the cost-matrix pair of the i-th subsystem and Q f > 0 is the terminal penalty matrix.The goal of this paper is to solve the following problem.
to minimize (5) subject to the dynamical system (4) with initial state x(0) = x 0 .4. DP-based approach for Problem 1.By using the dynamic programming approach, we will derive the analytical solution of Problem 1 in this section.However, we should prove the recurrence formula first.4.1.Recurrence formula.For any 0 < k < N − 1, let J(k, x k ) be the optimal reward obtainable in [k, N ] with the condition that at stage k we are in state x(k) = x k .Then we have Applying Bellman's Principle of Optimality, we can obtain the following recurrence equation.
Theorem 4.1.For model ( 6), we have the following recurrence equation: In addition, for any u(i), y(i), k ≤ i ≤ N , we have Taking the minimum of u(k), y(k) in the previous inequality yields The recurrence equation is proved.
4.2.Analytical solution.By using the recurrence equation, the analytical solution of Problem 1 can be derived.As in [26], define the following Riccati operator ρ i (P ) : S + n → S + n for given i ∈ M and P ∈ S + n , Let {H i } N i=0 denote the set of ordered pairs of matrices defined recursively: which means that at each stage k, the disturbance upon each subsystem is comparatively small.Next, we will derive the analytical solution of Problem 1. Firstly, we have For k = N − 1, the following equation holds by Theorem 4.1: (7), we can derive |s| ≥ 2.Moreover, ξ N is a linear uncertain variable and ξ N ∼ L(−1, 1).According to Example 2, the following equations hold Substituting ( 9) into (8) yields The optimal control u * (N − 1)satisfies we have Substituting ( 11) into (10) yields According to the definition of ρ i (P ) and H k , Eq. ( 12) can be written as = min Moreover, according to (13), we have For k = N − 2, we have It follows from a similar computation to (9) that By the similar method to the above process, we can obtain By induction, we can obtain the following theorem.
Remark 1. Theorem 4.2 reveals that, at iteration k, the optimal value and the optimal control law at all the future iterations only depend on the current set H k .The above theorem properly transforms the enumeration over the switching sequences in m N to the enumeration over the pairs of matrices in H k .It will be shown in the next section that the expression given by ( 16) is more convenient for the analysis and the efficient computation of problem 1.
5. Two-step pruning scheme.According to Theorem 4.2, at iteration k, the optimal value and the optimal control law at all the future iterations only depend on the current set H k .However, as k increases, the size of H k grows exponentially.It becomes unfeasible to compute H k when k grows large.A natural way of simplifying the computation is to ignore some redundant pairs in H k .In order to improve computational efficiency, a two-step pruning scheme aimed at removing some redundant pairs will be presented in this section.The first step is a local pruning and the second step is a global pruning.To formalize the above idea, the following definitions are introduced.Therefore, any equivalent subsets of H k define the same J(k, x k ).To ease the computation, we shall prune away as many redundant pairs as possible from H k and obtain an equivalent subset of H k whose size is as small as possible.In order to remove as many redundant pairs of matrices from H k as possible, a two-step pruning scheme is applied here.The first step is a local pruning which prunes away some redundant pairs from Γ k (P, γ) for any (P, γ), and the second step is a global pruning which removing redundant pairs from H k+1 after the first step.5.1.Local pruning scheme.The goal of local pruning algorithm is removing as many redundant pairs of matrices as possible from Γ k (P, γ).However, testing whether a pair is redundant or not is a challenging problem.A sufficient condition for checking whether pairs are redundant or not is given in the following lemma.
where s = |Γ k (P, γ)| and {(P ) ≥ 0 by the condition (17).Thus, for any x ≥ 0, we have α 1 x2 P −P (1) + • • • + α s−1 x 2 P −P (s−1) ≥ 0. So there exists at least one i such that the following formula holds According to γ (i) = γ, we obtain Checking the condition (17) in Lemma 5.3 is a LMI feasibility problem which can be solved with MATLAB toolbox LMI.However, Lemma 5.3 can not remove all the redundant pairs.If the condition in Lemma 5.3 is met, then the pairs under consideration will be discarded, otherwise, the pairs will be kept and get into H k+1 .As we know, the size of H k+1 is crucial throughout the computational process.So, after this step, we apply a global pruning to H k+1 .5.2.Global pruning scheme.A pair in H k being redundant or not can be checked by the following lemma.

Lemma 5.4. A pair
Table 2.The optimal results of Example 3 The numbers of elements in Hk and Ĥk at each step are listed in Table 3.It can be seen that the numbers of Hk and Ĥk do not necessarily increase with the number of subsystems.Additionally, with more subsystems, the effectiveness of local pruning becomes more apparent.Choose x 0 = (3, −1) τ , the optimal controls and the optimal values are listed in Table 4.A quadratic optimal control model for uncertain discrete-time switched linear systems whose subsystems are perturbed by uncertain factors has been presented, together with a method to design a control strategy and a switching law.The analytical solutions of the optimal objective function and the control strategy can be exactly characterized by H k , whose size grows exponentially.A two-step pruning scheme is developed to prune out as many redundant matrices in H k as possible.The first step is a local pruning and the second step is a global pruning.The examples validate the effectiveness of the method and show the fact that the greater the number of subsystems is, the better the effectiveness of the local pruning is.
the state vector with x(0) given and u(k) ∈ R r is the control vector, y(k) ∈ M {1, • • • , m} is the switching control that indicates the active subsystem at stage k; (ii) for each i ∈ M , A i ,B i are constant matrices of appropriate dimension; (iii) for each k ∈ K, σ k+1 ∈ R n and σ k+1 = 0, ξ k is the disturbance and ξ 1 , ξ 2 , • • • , ξ N are independent linear uncertain variables denoted by L(−1, 1).The performance of the sequence u(k)| N −1 k=0 and y(k)| N −1 k=0 can be measured by the following expected value:

Table 3 .k 1
Size of Hk and Ĥk for Example 4

Table 4 .
The optimal results of Example 4