OPTIMAL CONTROL OF SWITCHED SYSTEMS WITH MULTIPLE TIME-DELAYS AND A COST ON CHANGING CONTROL

. In this paper, we consider a class of optimal switching control problems with multiple time-delays and a cost on changing control and subject to terminal state constraints. A computational method involving three stages is developed to solve this class of optimal control problems. First, by param- eterizing the control function with piecewise-constant functions, the optimal switching control problem is approximated by a sequence of ﬁnite-dimensional optimization problems, where the original switching times, the control heights and the control switching times are decision variables. Second, by introduc- ing new variables, the total variation of the control variables is transformed into an equivalently smooth function. Third, we convert the constrained op- timization problem into one only with box constraints by an exact penalty function method. The gradients of the cost functional are then derived, which can be combined with any gradient-based optimization method to determine the optimal solution. Finally, a numerical example is given to illustrate the eﬀectiveness of the proposed algorithm.

1. Introduction. Switched systems are encountered in many real life applications, such as locomotives [5], bio-chemical reactors [9], and hybrid power systems [24]. On the other hands, the existence of time delays must not be ignored in many practical engineering problems [17]. As a consequence, optimal control of switched systems with or without delays has become an important and challenging research topic for many applied mathematicians and engineers [2,18,26]. For conventionally optimal control of switched system, its aim is to determine an optimal control function and an optimal switching sequence of the switched systems involved such that a cost functional is minimized subject to constraints on the state and/or the control. However, the presence of delays in a switched system complicates the search for an optimal operation policy. In particular, the time-scaling transformation [8,12,26], a powerful tool for solving optimal switching control problems, cannot be directly applied to switched systems with time-delays [7]. In fact, optimization techniques for switched systems with time-delays are scarce in the literature. Necessary conditions for determining optimal switching times and/or optimal impulse magnitudes for such systems are derived in [4,21,22] via classical variational techniques. However, these analytical results are only applicable to separable systems with a single delay. Reference [25] presents an algorithm for optimization of switched systems with time-delays, which is based on a parameterization scheme in which the switching times are expressed in terms of the subsystem durations. However, this algorithm is only applicable to switched systems with a single delay and no control input. Recently, an effective computational method is developed to find the optimal switching times and system parameters for a general switched system with multiple time-delays in [11]. However, the control input is not considered as a decision variable in [11].
In this paper, we will consider a new class of optimal control problems, where the dynamical system is governed by a switched system with multiple time-delays and there is a cost on changing control. In a standard optimal control problem, the cost functional is usually expressed in terms of the final state and/or an integral term involving the state and control values at each time. The cost of changing the control input is rarely considered. Thus, two different control laws that result in the same cost value are deemed to perform equally even if one of them is constant while the other fluctuates widely. However, in reality, there will be a cost when the control input changes from one value to another. The cost may be just wear and tear on the system actuators [14]. Some papers are now available in the literature that consider delay-free optimal control problems with costs on changing control input. For example, necessary optimality conditions are derived in [3] for an optimal control problem in which the control input can only assume two possible values and there is a cost associated with changing from one value to another. In [15], further optimality conditions for similar problems (but more general), in which the total variation of the control input is incorporated as a penalty in the cost functional, is investigated. In [20], a computational algorithm is developed to solve a class of optimal control problems in which the cost functional is the sum of the terminal cost, the integral cost and the full variation of control. Recently, in [14], a new computational method based on the control parameterizaiton [19], time-scaling transformation [12] and a smoothing technique is proposed to solve similar problems as in [20]. However, these computational methods are not applicable to optimal switching control problems with multiple time-delays.
The aim of this paper is to propose a computational method, which is applicable to a class of optimal control problems with the following characteristics: (i). The dynamical system is a switched system with variable switching times; (ii). The dynamical system is with multiple time-delays; (iii). The cost functional contains a total variation measuring the changes of control inputs; and (iv). The state variables are required to satisfy the terminal equality constraints. We first convert the optimal switching control problem into a sequence of finitedimensional optimization problems by parameterizing the control input with piecewise-constant functions. In view of the non-smoothness of the cost functional, the smoothing technique [14] is then used to transform the resulting optimization problems into equivalently smooth constrained optimization problems. The exact penalty method [27] is further utilized to transcribe the constrained optimization problems into optimization problems only with box constraints. For these optimization problems, a computational method is developed based on the gradients of the penalized cost functional with respect to decision variables. Finally, a numerical example is given to illustrate the effectiveness of the proposed computational method.
The rest of this paper is organized as follows. The problem formulation is described in Section 2. The transformation procedure, including the control parameterization technique, the smoothing technique and the exact penalty method, is introduced in Section 3. The gradient formulas are derived in Section 4. A numerical example is given in Section 5. Section 6 concludes the paper.
System (1) is controlled by manipulating the switching times and the control. Define where ∆ i > 0 is the minimum duration of the ith subsystem. Any τ ∈ T is called a feasible switching time vector. Define where a h and b h are given real numbers such that a h ≤ b h , h = 1, 2, . . . , r. It is obvious that U is a compact and convex subset of R r . The total variation of the control u : where u h : [0, T ] → R is the hth component of the control u and T 0 u h denotes the total variation of u h defined by Here, the supremum is taken over all finite partitions {t l } p l=0 ⊂ [0, T ] satisfying 0 = t 0 < t 1 < · · · < t p−1 < t p = T. If the total variation of u is finite, then we say that u is of bounded variation. Let u : [0, T ] → U be a function of bounded variation over [0, T ], and let U be the set of all such control functions.
We assume that the following conditions are satisfied throughout this paper. Assumption 1. The functions f i , i = 1, 2, . . . , N, are continuously differentiable. Moreover, the function ϕ is twice continuously differentiable.
where · is the Euclidean norm.
Assumptions 1 and 2 ensure that the switched system (1) has a unique solution x(·|τ , u) corresponding to each τ ∈ T and u ∈ U [1]. Now, we suppose that system (1) is subjected to the following terminal state constraints: where φ l : R n → R, l = 1, 2, . . . , q, are given continuously differentiable functions.
In real practical situations, large fluctuations of the control are clearly undesirable. Thus, the cost functional is expressed as a weighted sum of the function of the terminal state and the total variation of the control, that is, the cost functional can be defined as where ψ : R n → R is a given continuously differentiable function and γ ∈ [0, 1] is a weighting factor. Note that we can easily transform an integral running cost into the form of (7) by introducing an additional state variable. Our object is to choose a switching time vector τ ∈ T and a control u ∈ U such that the cost functional (7) is minimized subject to the terminal constraints (6). Thus, our optimal switching control problem is stated formally as follows.
Problem (P). Find a switching time vector τ ∈ T and a control u ∈ U such that the cost functional (7) is minimized subject to the switched time-delay system (1) and the terminal constraints (6). Problem (P) is an optimal control problem involving a nonlinear switched system with multiple time-delays and a cost on changing control. It is well known that variable switching times pose a significant challenge for conventional numerical optimization techniques [12,14]. One of the most widely used methods for overcoming this challenge is the time-scaling transformation described in [6,13,14]. This transformation involves mapping the variable switching times to fixed points in a new time horizon. Unfortunately, since the time-scaling transformation is not applicable to delay systems [7], it cannot be used to solve Problem (P). Therefore, a new approach is needed to solve Problem (P).
3.1. Control parameterization. To solve Problem (P) numerically, we apply the control parameterization scheme [19]. That is, let each of the time subintervals (τ i−1 , τ i ], i = 1, 2, . . . , N , be partitioned into p i subintervals with p i − 1 partition points, i.e., Then, the control is approximated as a piecewise-constant function defined by is the kth control height vector of the ith switch interval, i = 1, 2, . . . , N , k = 1, 2, . . . , p i ; and for a given interval I, χ I denotes the indicator function defined by Furthermore, the partition times satisfy the following constraints: where p 0 = 0 and ∆ i is as defined in (2). Thus, by the control parameterization, the system (1) becomes and Furthermore, let The approximated control can be rewritten as Define Then, the switched system (10) can be written as Let Γ be the set of all those switching vectors υ such that conditions (11) and (12) are satisfied, Ω be the set of all those control parameter vectors σ = ((σ 1 ) ⊤ , (σ 2 ) ⊤ , . . . , (σN ) ⊤ ) ⊤ ∈ UN ⊂ R rN , and x p (·|υ, σ) be the solution corresponding to the pair (υ, σ) ∈ Γ × Ω. Then, the terminal constraints become φ l (x p (T |υ, σ)) = 0, l = 1, 2, . . . , q, and the cost functional becomes As a result, Problem (P) can be approximated by the following finite-dimensional optimization problem: Problem (P(p)). Find a feasible pair (υ, σ) ∈ Γ × Ω such that the cost functional is minimized subject to switched time-delay system (15) and terminal constraints (16).

3.2.
An equivalent problem. Since the cost functional (17) is non-smooth, the standard gradient-based optimization algorithms cannot be used directly to solve Problem (P(p)). In what follows, we shall show that Problem (P(p)) can be converted into an equivalently smooth optimization problem. Let where β, ς j , ξ j are all vectors in R r . Define the function Furthermore, let Z be the set of ζ ∈ R (2N−1)r in the form of (18) such that for each h = 1, 2, . . . , r, . . ,N − 1. Next, we consider the following dynamic system: Let y(·|υ, ζ) be the unique solution of the system (19) corresponding to (υ, ζ) ∈ Γ × Z and let the terminal constraints be defined by φ l (y(T |υ, ζ)) = 0, l = 1, 2, . . . , q.
Then, we can state a new optimization problem as given below: Problem (Q). Find a feasible pair (υ, ζ) ∈ Γ × Z such that the cost functional is minimized subject to the system (19) and the terminal constraints (20).
Note that Problem (Q) is a smooth optimization problem. For Problems (P(p)) and (Q), we have the following theorem.
By system (15), it is obvious that By differentiatingx(t) and using the chain rule, we obtain that for t ∈ (υ j−1 , υ j ), j = 1, 2, . . . ,N ,ẋ This implies thatx(·) is the solution of system (19). It follows thatx(·) is the unique solution of system (19), as required.  Based on Theorems 3.1 and 3.2 and by similar arguments as given in [14], we can obtain the following result. By Theorem 3.3, we see that the optimal solution of Problem (P(p)) can be generated by that of Problem (Q).

3.3.
A penalty problem. Problem (Q) is a smooth dynamic optimization problem with terminal state constraints. In this subsection, the exact penalty method [27] will be applied to these terminal state constraints. Define . . , r}, It is obvious that Γ ⊂ Υ and Z ⊂ Z 0 . Now, we define a constraint violation function on Υ × Z 0 as Then, Π(υ, ζ) = 0 if and only if (υ, ζ) ∈ Υ × Z 0 is feasible for Problem (Q). On this basis, we define a penalty functionJ p (υ, ζ) as follows: where ǫ > 0 is a new decision variable, ρ > 0 is a penalty parameter, and δ 1 and δ 2 are fixed parameters satisfying 1 ≤ δ 2 ≤ δ 1 .
With the penalty cost functional (23), we define the following penalty problem.
The following convergence results can be obtained based on the result in [27]. Theorem 3.5. Let {ρ k } +∞ k=1 be an increasing sequence of penalty parameters such that ρ k → +∞ as k → +∞, and let (υ k, * , ζ k, * , ǫ k, * ) be the solution of Problem (Q) with ρ = ρ k . Then, the limit point of the sequence {(υ k, * , ζ k, * )} is a solution of Problem (Q).
From the above two theorems, we can solve Problem (Q) where only the box constraints are involved. Furthermore, the solution of Problem (Q) with ρ being sufficiently large is the optimal solution of Problem (Q). In the next section, we will derive the gradient formulas ofJ p (υ, ζ, ǫ) with respect to the decision variables. 4. Gradient formulas. Problem (Q) is an optimization problem involving a nonlinear switched time-delay system, where the switching time vector υ, parameter vector ζ and variable ǫ are decision variables to be optimized. Note that gradientbased optimization algorithms, e.g., sequential quadratic programming (SQP) [16], are effective in solving such optimization problems [19]. For this, the gradients of the cost functional (23) with respect to the decision variables are needed. To this end, defineỹ y(t|υ, ζ),ỹ(t|υ, ζ), ̺ j (ζ)), and for given i and j, For each (υ, ζ, ǫ) ∈ Υ × Z 0 × [0,ǭ], consider the following costate system: with the conditions λ(T ) = ∂ψ(y(T )) ∂y + 2ǫ −δ1 q l=1 φ l (y(T )) ∂φ l (y(T )) ∂y .
The following theorem gives the gradients of the cost functional (23) with respect to the switching times υ.
Proof. The proof is similar to that given for Theorem 4.1.
On the basis of Theorems 4.1 and 4.2, we now propose the following algorithm for solving Problem (Q).
5. Numerical example. Consider the following nonlinear switched system with two time delays: with the initial conditions The time horizon is [0, 1.5]. We assume that the system switches once during the time horizon from subsystem 1 to subsystem 2. Suppose that the switched point τ 1 ∈ [0.1, 1.4], and the control u(t) ∈ [−5, 10] for each t ∈ [0, 1.5]. The terminal state should satisfy the following condition: x 2 (1.5) = 1. Our goal is to find an optimal switching time τ is minimized subject to the switched time-delay system (37)-(39) and the terminal constraint (40). We use our proposed computational method in the previous section with ǫ min = 10 −6 , ρ 0 = 1, δ 1 = 1, δ 2 = 2, and δ 3 = 5 to solve this example for different weighting factors γ = 0, γ = 0.01, γ = 0.05, γ = 0.1, and γ = 0.5, respectively. Our numerical results are summarized in TABLE 1. The optimal control and the corresponding state trajectories are shown in FIGURE 1 and FIGURE 2, respectively. From FIGURE 1, we can see that the optimal controls become more smooth when the weighting factor γ is increased, that is, the total variation is emphasized when the weighting factor γ is increased.
6. Conclusions. This paper has studied optimal control problem involving switched systems with multiple time-delays and subject to terminal state constraints. The objective functional in the problem contains a cost on changing control. A gradient-based optimization method was developed to solve the optimal control problem by the control parameterization, the smoothing technique and the exact penalty method. A numerical example indicates the effectiveness of the proposed method.
In fact, the optimal control problem considered in this paper has wide applications in the practical engineering; see, for example [10,23]. To solve these engineering problems using our developed computational method is our undergoing studies.