A sufficient optimality condition for delayed state-linear optimal control problems

We give answer to an open question by proving a sufficient optimality condition for state-linear optimal control problems with time delays in state and control variables. In the proof of our main result, we transform a delayed state-linear optimal control problem to an equivalent non-delayed problem. This allows us to use a well-known theorem that ensures a sufficient optimality condition for non-delayed state-linear optimal control problems. An example is given in order to illustrate the obtained result.


1.
Introduction. Time delays occur in many dynamical systems such as biological, chemical, mechanical and economical systems (see, e.g., [3,11,16,27,44,[48][49][50][51]). Dynamical systems with time delays, in both state and control variables, play an important role in the modelling of real-life phenomena in various fields of applications [15,16]. For instance, in [42] the incubation and pharmacological delays are modelled through the introduction of time delays in both state and control variables. In [46], Silva, Maurer and Torres introduce time delays in the state and control variables for tuberculosis modelling. They represent the time delay on the diagnosis and commencement of treatment of individuals with active tuberculosis infection and the delays on the treatment of persistent latent individuals, due to clinical and patient reasons. There is a vast literature on delayed optimal control problems, also called retarded, time-lag, or hereditary optimal control problems. See, e.g., [2,4,13,15,18,36] and references cited therein.
Delayed linear differential systems have also been investigated, their importance being recognized both from a theoretical and practical points of view. For instance, in [13] Friedman considers linear hereditary processes and apply to them Pontryagin's method, deriving necessary optimality conditions as well as existence and uniqueness results. Analogously, in [36] delayed linear differential equations and optimal control problems involving this kind of systems are studied. Since these first works, many researchers have devoted their attention to linear quadratic optimal control problems with time delays, see, e.g., [7,10,12,26,37]. It turns out that for delayed linear quadratic optimal control problems it is possible to provide an explicit formula for the optimal controls [7,26,37].
Optimal control problems with a differential system that is linear both in state and control variables have been studied in [7,9,10,12,26,28,29,31,35,37]. In [10,28,37], the system is delayed with respect to state and control variables. In [9,35], the system only considers delays in the state variable. Chyung and Lee derive necessary and sufficient optimality conditions in [9] while Oǧuztöreli only proves necessary conditions [35]. Certain necessary conditions analysed by Chyung and Lee in [9] have been already derived in [23,40,41]. However, the system considered in [9] is different from the previously studied hereditary systems, which do not require a initial function of state. In [12], Eller et al. derive a sufficient condition for a control to be optimal for certain problems with time delay. The problems studied by Eller et al. and Khellat, respectively in [12] and [26], consider only one constant lag in the state. The research done by Lee in [31] is different from ours, because in [31] the aim is to minimize a cost functional, which does not consider delays, subject to a differential system that is linear in state and control variables, and to another constraint. In their differential system, the state variable depends on a constant and fixed delay and the control variable depends on a constant lag, which is not specified a priori. Note that the differential system of the problem considered in [29] is similar to the one of [31]. Although Banks has studied delayed non-linear problems without lags in the control, he has also analyzed problems that are linear and delayed with respect to control [2]. Recently, Cacace et al. studied optimal control problems that involve linear differential systems with variable delays only in the control [7]. The problems analyzed in the present paper are different from those considered in the mentioned works, because here the problems involve differential systems that are linear with respect to state, but not with respect to the control. Furthermore, we consider a constant lag in the state and another one in the control. These two delays are in general not equal.
In [20], Hughes firstly consider variational problems with only one constant lag and derive various necessary and a sufficient optimality condition for them. The variational problems in [20] can easily be transformed to control problems with only one constant delay (see, e.g., [34, p. 53-54]). Hughes also investigate an optimality condition for a control problem with a constant delay, which is the same for state and control. Therefore, the problems investigated in [20] are different from the problems studied by us, because in the present paper the delay of state is not necessarily equal to the delay of control. The problems analyzed by Chan and Yung [8] and by Sabbagh [43] are similar to the first problems studied by Hughes in [20]. So, for the same reason, the problems investigated in [8,43] are different from ours. The problems considered in [20,43] are also considered in [38] by Palm and Schmitendorf. For such problems, they derive two conjugate-point conditions, which are not equivalent. Note that their conditions are only necessary and do not give a set of sufficient conditions [38].
In [22], Jacobs and Kao investigate delayed problems that consist to minimize a cost functional without delays subject to a differential system defined by a non-linear function with a delay in state and another one in the control. Similar to our case, these delays do not have to be equal. In contrast, our cost functional contains also time delays, therefore being more general than the one considered in [22]. Jacobs and Kao transform the problem using a Lagrange-multiplier technique and prove a regularity result in the form of a controllability condition, as well as some necessary optimality conditions. Then, in some special restricted cases, they prove existence, uniqueness and sufficient conditions. Such restricted problems consider a differential system that is linear in state and in control variables. Thus, the sufficient conditions of [22] are derived for problems that are less general than ours.
The delayed optimal control problems analyzed by Schmitendorf in [45] have a cost functional and a differential system that are more general than ours. However, in [45] the control takes its values in all R m while in the present paper the control values belong to a set Ω ⊂ R m , m ∈ N. In [32], Lee and Yung study a problem that is similar to the one considered in [45], where the control belongs to a subset of R m , as we consider here. First and second-order sufficient conditions are shown in [32]. Nevertheless, the conditions of [32] are not constructive and practical for the computation of the optimal solution. Indeed, as hypothesis, it is assumed existence of a symmetric matrix under some conditions, for which is not given a method to calculate its expression. Another similar problem to our is studied by Bokov in [6], in order to arise a necessary optimality condition in an explicit form. Moreover, a solution to the problem with infinite time horizon is given in [6]. In contrast, in the present paper we are interested to derive sufficient optimality conditions. As it is well known, and as Hwang and Bien write in [21], many investigations have directed their efforts to seek sufficient conditions for control problems with delays: see, e.g., [9,12,20,22,32,45]. In [21], Hwang and Bien prove a sufficient condition for problems involving a differential affine time-delay system with the same lag for the state and the control. Thus, the differential system considered in the present article is obviously more general. In 1996, Lee and Yung derived various first and second-order sufficient conditions for non-linear optimal control problems, with only a constant delay in the state, and considering functions that do not have to be convex [30]. As in [8,32], second-order sufficient conditions are shown to be related to the existence of solutions of a Riccati-type matrix differential inequality.
Optimal control problems with multiple delays have also been investigated. In [18], Halanay derive necessary conditions for some optimal control problems with various time lags in state and control variables, using the abstract multiplier rule of Hestenes [19]. In [18], all delays related to state are equal to each other and the same happens with the delays associated to the control. Note that the results of [13,23] are obtained as particular cases of problems considered in [18]. Later, in 1973, a necessary condition is derived for an optimal control problem that involves multiple constant lags only in the control. This delayed dependence occurs both in the cost functional and in the differential system, which is defined by a non-linear function [47]. In [25], Kharatishvili and Tadumadze prove the existence of an optimal solution and a necessary condition for optimal control systems with multiple variable time lags in the state and multiple variable commensurable time delays in the control. Later, an optimal control problem where the state variable is solution of an integral equation with multiple delays, both for state and control variables, is studied by Bakke in [1]. Furthermore, necessary conditions and Hamilton-Jacobi equations are derived. In 2013, Boccia, Falugi, Maurer and Vinter derived necessary conditions for a free end-time optimal control problem subject to a non-linear differential system with multiple delays in the state [4]. The control variable is not influenced by time lags in [4]. Recently, in 2017, Boccia and Vinter obtained necessary conditions for a fixed end-time problem with a constant and unique delay for all variables, as well as free end-time problems without control delays [5].
As Guinn wrote, the classical methods of obtaining necessary conditions for retarded optimal control problems (used, for instance, by Halanay in [18], Kharatishvili in [24] and Oǧuztöreli in [36]) require complicated and extensive proofs [17] (see, e.g., [2,13,18,24,36]). In 1976, Guinn proposed a method whereby we can reduce some specific time-lag optimal control problems to equivalent and augmented optimal control problems without delays [17]. By reducing delayed optimal control problems into non-delayed ones, we can then use well-known theorems applicable for optimal control problems without delays to derive desired optimality conditions for delayed problems [17]. In [17], Guinn study specific optimal control problems with a constant delay in state and control variables. These two delays are equal. Later, in 2009, Göllmann, Kern and Maurer studied optimal control problems with a constant delay in state and control variables subject to mixed control-state inequality constraints [15]. In that research, the delays do not have to be equal. For technical reasons, the authors need to assume that the ratio between these two time delays is a rational number [15]. In [15], the method used by Guinn in [17] is generalized and, consequently, a non-delayed optimal control problem is obtained again. Pontryagin's Minimum Principle, for non-delayed control problems with mixed state-control constraints, is used and first-order necessary optimality conditions are derived for retarded problems [15]. Furthermore, Göllmann, Kern and Maurer discuss the Euler discretization for the retarded problem and some analytical examples versus correspondent numerical solutions are given. Later, in 2014, Göllmann and Maurer generalized the research mentioned before, by studying optimal control problems with multiple and constant time delays in state and control, involving mixed state-control inequality constraints [16]. Again, necessary optimality conditions are derived [16]. Note that the works [15][16][17][18] consider delayed non-linear differential systems.
In the present paper, we consider optimal control problems that consist to minimize a delayed non-linear cost functional subject to a delayed differential system that is linear with respect to state, but not with respect to control. The delay in the state is the same for the cost functional and for the differential system. The same happens with the time lag of the control variable. We derive a sufficient optimality condition for this type of problems. Note that the cost functional does not have to be quadratic, but it satisfies some continuity and convexity assumptions. To the best of our knowledge, this gives answer to an open question. Note that the constant delays on the state and control variables do not have to be equal, but we ensure the commensurability assumption between state and control delays, similarly to Göllmann, Kern and Maurer in [15]. Indeed, we follow the approach of [15] and Guinn [17], that is, we transform the delayed optimal control problem into an equivalent non-delayed optimal control problem and then apply a classical sufficient optimality condition [33, p. 340-343].
The paper is organized as follows. In Section 2, we define the optimal control problem without delays for which the sufficient optimality condition [33, p. 340-343] holds. In Section 3, we define our retarded optimal control problem with constant time delays in state and control variables. Then, in Section 4, we prove a sufficient optimality condition for the problem stated in Section 3. A concrete example is solved in detail in Section 5, with the purpose to illustrate our main result. We end with some conclusions in Section 6.
2. Non-delayed state-linear optimal control problem. We begin by defining a non-delayed state-linear optimal control problem and recall a well-known sufficient optimality result for such class of problems.
Consider the non-delayed state-linear optimal control problem (LP) which consists to subject to the control system in R ṅ with initial boundary condition and final boundary condition Notation. Along the text we use the notation ∂ i f to denote the partial derivative of a certain function f with respect to its ith argument. For example, 1. An admissible process to (LP) is given by a pair of functions (2) and (3).
The following theorem gives a sufficient optimality condition for problem (LP).
Theorem 2.2 (See Theorem 5, Section 5.2 of [33]). Consider problem (LP) and assume that 3. for almost all t ∈ [a, b], u * is a control with response x * that satisfies the maximality condition where and η(t) is any nontrivial solution of the adjoint systeṁ satisfying the transversality condition that ensures that η(b) is an inward normal vector of Π at the boundary point x * (b). Then, u * is an optimal control that leads to the minimal cost C[u * ].
3. Delayed state-linear optimal control problem. In this paper we are interested in state-linear optimal control problems with discrete time delays r ≥ 0 in the state variables x(t) ∈ R n and s ≥ 0 in the control variables u(t) ∈ R m , (r, s) = (0, 0). The delayed state-linear optimal control problem (LP D ) consists in subject to the delayed differential systeṁ with the following initial functions where Definition 3.
1. An admissible process to problem (LP D ) is given by a pair of In what follows, we assume that the time delays r and s respect the following commensurability assumption.
Assumption 4.1 (Commensurability assumption). We consider r, s ≥ 0 not simultaneously equal to zero and commensurable, that is, Remark 2. The commensurability assumption holds for any couple of rational numbers (r, s) for which at least one number is nonzero [15].
3. for almost all t ∈ [a, b], u * is a control with response x * that satisfies the maximality condition where for p ∈ {0, 1}, and η(t) is any nontrivial solution of the adjoint systeṁ that satisfies the transversality condition η(b) = [0 · · · 0] 1×n .
Then, u * is an optimal control that leads to the minimal cost C D [u * ].
Proof. We transform the delayed state-linear optimal control problem (LP D ) into an equivalent non-delayed state-linear optimal control (LP) type problem, using the approach of [15,17], and then we apply Theorem 2.2. Without loss of generality, we assume the first case of Assumption 4.1, that is, r s ∈ Q for r > 0 and s > 0.
Consequently, there exist k, l ∈ N such that Thus, let us divide the interval [a, b] into N subintervals of amplitude h := r k = s l . We can note that r = hk and s = hl. Furthermore, we also assume that a + hN = b and N > 2k + 1, with N ∈ N. = hN ), then we can study problem (LP D ) for t ∈ [a,b], whereb is the smallest multiple of h, which is greater than b. Thus, we also study problem (LP D ) for t ∈ [a, b], because b <b.
As ξ − (t) and θ − (t) are known, we have that Therefore, we have the set of constraints (9) in form (10). To apply Theorem 2.2, we have to ensure that 1. F 0 , ∂ 2 F 0 , G 0 ,Ã andG are continuous for all (t, ξ, θ) ∈ [a, a + h] × R N n+N m ; 2. F 0 (t, ξ) is a convex function in ξ for each fixed t ∈ [a, a + h]; 3. θ * is a control with response ξ * that satisfies the maximality condition such that Λ i (a + h) is an inward normal vector of the closed convex set at the boundary point ξ * i (a + h) for i = 0, . . . , N − 1. Thus, θ * will be an optimal control that leads us to the minimal cost C[θ * ]. From now on, we are going to analyze each hypothesis of Theorem 4.2.
Remark 4. We can note that: (i) problems (LP D ) and (LP ) are equivalent; (ii) the augmented and non-delayed problem (LP ) is defined for t ∈ [a, a + h]. Even more, we can solve problem (LP D ) by solving N sub-problems, each one with respect to each subinterval of [a, b] with amplitude h. Then, we can concatenate the respective N optimal solutions in order to obtain an optimal solution of (LP D ). Thus, we can solve problem (LP D ) by solving N augmented and non-delayed sub-problems (LP i ) associated to problem (LP D ), with i = 0, . . . , N − 1. For i ∈ {0, . . . , N − 2}, the (i + 1)th augmented and non-delayed sub-problem (LP i ) consists to minimize a+h a f 0 (t i , ξ i (t), ξ i−k (t)) + g 0 (t i , θ i (t), θ i−l (t))dt . Theorem 2.2 can be applied and we can find an optimal pair (ξ * i (·), θ * i (·)) in the interval of time [a, a + h] that provides an optimal solution (x * (·), u * (·)) in the interval of time [a + hi, a + h(i + 1)]. The setΠ i has a single point. So, Λ i (a + h) is an inward normal vector ofΠ i at the boundary point ξ * i (a + h) (recall Remark 1). The last augmented and non-delayed sub-problem Again, Theorem 2.2 can be applied and we can find an optimal pair (ξ * N −1 (·), θ * N −1 (·)) in interval of time [a, a+h] that provides an optimal solution (x * (·), u * (·)) in the interval of time [a + h (N − 1), b]. AsΠ N −1 = R n , then by Theorem 2.2 Λ N −1 (a + h) = [0 · · · 0] 1×n .
Such analytical expressions can be obtained with the help of a modern computer algebra system. We have used Mathematica. In Figures 2 and 3, we observe that the numerical solutions for control and state, obtained using AMPL [14] and IPOPT [39], are in agreement with their analytical solutions, given by (19) and (20)  6. Conclusion. We considered a delayed state-linear optimal control problem. We proved a sufficient optimality condition for problems with delays in both state and control variables. The proof is based on the transformation of the delayed statelinear optimal control problem into a non-delayed one, following the approach proposed in [17] and used in [15]. Analogously to [15], we ensure the commensurability assumption between the, possibly different, state and control delays. An example is provided, which illustrates the usefulness of obtained sufficient optimality condition.