PROOF OF THE MAXIMUM PRINCIPLE FOR A PROBLEM WITH STATE CONSTRAINTS BY THE V-CHANGE OF TIME VARIABLE

. We give a new proof of the maximum principle for optimal control problems with running state constraints. The proof uses the so-called method of v − change of the time variable introduced by Dubovitskii and Milyutin. In this method, the time t is considered as a new state variable satisfying the equation d t/ d τ = v, where v ( τ ) ≥ 0 is a new control and τ a new time. Unlike the general v − change with an arbitrary v ( τ ) , we use a piecewise constant v. Every such v − change reduces the original problem to a problem in a ﬁnite dimensional space, with a continuum number of inequality constrains corresponding to the state constraints. The stationarity conditions in every new problem, being written in terms of the original time t, give a weak* compact set of normalized tuples of Lagrange multipliers. The family of these compacta is centered and thus has a nonempty intersection. An arbitrary tuple of Lagrange multipliers belonging to the latter ensures the maximum principle.

1. Introduction. In this paper we propose a new proof of the maximum principle (MP) for optimal control problems with state constraints. There is a vast literature on the subject. The very first results were obtained by R.V. Gamkrelidze at the dawn of the optimal control theory (see e.g. [16,Ch.6]), under some special assumptions on the structure of optimal process. These assumptions were avoided somewhat later in the seminal work [18] by A.Ya. Dubovitskii and A.A. Milyutin, containing a complete form of the MP for state constraints together with a new, quite universal method of its proof. An important part of this method is the v− change of time variable and related v− variations of the control. Later, several other methods of the proof were used by different authors, such as the so-called sliding mode regimes [13,20,14,15], the smoothly-convex structure of the controlled system [11], the Ekeland variational principle [9,17,1], etc. However, all these proofs are technically rather complex and not completely mastered even by specialists, not to mention a wider range of readers, albeit with a good mathematical background.
The idea of v -change of the time variable, proposed in [18], is as follows. Consider the time t, varying in an interval [t 0 , t 1 ], as a new state variable t = t(τ ) that depends on a new time τ, varying in an interval [τ 0 , τ 1 ]. Let the function t(τ ) satisfy the relations Then we can putũ(τ ) ≡ u * on this interval, where u * ∈ U is an arbitrary fixed value, while on the complement to [τ , τ ] we can use the formulaũ(τ ) = u(t(τ )). Thus, we obtain a new controlũ(τ ) and a new trajectorỹ x(τ ) defined asx(τ ) = x(t(τ )) for all τ ∈ [τ 0 , τ 1 ]. It is easy to transform the initial optimal control problem with the independent variable t into a new problem, corresponding to this change, with the independent variable τ. Moreover, one can easily show that, under such a transformation, any optimal process of the initial problem transforms into an optimal process of the new problem. Furthermore, it is possible to return from the variable τ to the original variable t using the "inverse" (to be more precise, right inverse) change. Namely, for any t ∈ [t 0 , t 1 ], let τ (t) be the smallest root of the equation t(τ ) = t. Then, obviously, t(τ (t)) ≡ t,ũ(τ (t)) = u(t), andx(τ (t)) = x(t) on [t 0 , t 1 ].
It could seem that the selected value u * does not play any role in such transition, since the interval [τ , τ ] maps into a single point t * ∈ [t 0 , t 1 ]. However, this is true only for the given control v(τ ). But what happens if the function v(τ ) is perturbed by a uniformly small variationv(τ ) such that still v(τ ) +v(τ ) ≥ 0 a.e. on [τ 0 , τ 1 ]? Then the interval [τ , τ ] maps onto a small interval [t , t ] with u(t) = u * on it, so we obtain a needle shape variation of the original control! It is this fact that was used by Dubovitskii and Milyutin to obtain the MP by means of the v -change. They systematically used it; see e.g. [18,20,14] and other works. The v -change turned out to be a very powerful tool for obtaining necessary optimality conditions in the form of MP.
Note that though the v -variations are much alike the convenient needle variations, they have advantages. Whereas the usage of needle variations requires the assumption of piecewise continuity of the optimal control, the usage of v -variations does not need it, and the optimal control can be an arbitrary measurable bounded function. Moreover, the v -variations generate a smooth control system, well defined for v(τ ) of arbitrary sign, where the requirement v(τ ) ≥ 0 can be regarded as a separate standard constraint, while the needle variations can be considered only for nonnegative widths of the needles, so the terminal value of the state is a function of the needles widths defined just on the positive cone, which is not convenient to differentiate. Thus, the v -change of time is a more preferable tool in research of optimality than the needle variations. Note however, that for an arbitrary nonnegative v(τ ), the v -change is rather complicated technically. To make the proof of MP more simple, A.A. Milyutin proposed in 2001 to use the v -change with a piecewise constant function v(τ ) (obviously, it then suffices to allow only two values: 0 and 1). This idea was realized in [15,21] for a general optimal control problem of the Pontryagin type (i.e. without state constraints), where a quite simple proof of MP was obtained. Such a primitive v -change allowed to pass to a family of smooth optimization problems in finite dimensional spaces, i.e. to problems of mathematical programming, and then to use the well-known necessary optimality conditions in each problem. Proper arranging of the obtained family of these optimality conditions made it possible to pass from them to one universal condition, which had the form of MP. Later it turned out that this approach works as well for problems with state constraints, but then one obtains a family of "semi-infinite" problems in finite dimensional spaces, optimality conditions for which are also known. In the case of one state constraint the passage from a family of finite dimensional optimality conditions to the universal MP is rather simple, see [6], while the case of several state constraints, considered in this article, requires an additional nontrivial trick.
The paper is organized as follows. In Section 2 we give the problem statement and formulate the MP. Section 3 is devoted to the proof of MP. The proof uses some auxiliary facts given in Appendix (Sections A-C). In particular, Section A gives an abstract Lagrange multipliers rule for a nonsmooth optimization problem in a Banach space, Section B presents a generalized formula of integration by parts for functions of bounded variation, and Section C contains the proof of a key fact on the weak* compactness of some special set of Lagrange multipliers.
2. Statement of the problem and the maximum principle. We consider the following optimal control problem on a variable time interval [t 0 , t 1 ] : (4) Here the function x(·) : [t 0 , t 1 ] → R n is absolutely continuous, u(·) : [t 0 , t 1 ] → R r is measurable and essentially bounded; the mappings F 0 : R 2n+2 → R, F : are of class C 1 , while f : R n+r+1 → R n is continuous together with its first derivatives with respect to t and x . Here d(F ), d(K), and d(Φ) are the dimensions of F, K, and Φ, respectively. The set U ⊂ R r is arbitrary. For brevity, the problem (1)-(4) will be called problem A.
In order to formulate the maximum principle for problem A, introduce the Pontryagin function where p x is a row vector of the dimension n, and the endpoint Lagrange function where α 0 is a real number, and α, β are row vectors of the same dimensions as F, K, respectively (the dependence of l on α 0 , α, β is omitted).
We say that, that an admissible process (x(t), u(t)), t ∈ [t 0 , t 1 ] satisfies the maximum principle (MP) if there exist a real number Here p x and p t are costate (adjoint) variables corresponding to the state variables x and t, respectively (so, the subindices x and t do not denote the derivatives) 1 . The adjoint equations (iv x )−(iv t ) can be understood as equalities between measures. From the second complementarity condition (iii) and assumption (5) it follows that all measures dµ j vanish in some neighborhoods of the ends of the interval [t 0 ,t 1 ], and therefore, in view of the adjoint equations, the functions p x (t) and p t (t) are continuous in these neighborhoods.
Observe that conditions (vi) and (vii) imply the maximality condition for the Pontryagin function: Finally note that, in this definition of MP, we may do not care about the left or right continuity of the adjoint variables p x (t) and p t (t), considering them as arbitrary functions of bounded variation, but then condition (vii) should be replaced by the two conditions: . The other conditions do not change.
The necessary conditions for a strong minimum are given in the following theorem, first proved in [18] for a somewhat less general problem.
is a strong minimum in Problem A and satisfies assumption (5), then it satisfies the maximum principle (i)-(vii).
3. Proof of the maximum principle. It is convenient to carry out the proof first for the time-independent case of problem A.
3.1. The autonomous problem B. Consider the following time-independent problem B (a special case of Problem A), still on a non-fixed interval [t 0 , t 1 ] : Assumption (5) reads here For this problem, the adjoint equation (iv t ) implies that p t = const, and then condition (v t ) yields p t ≡ 0, so we will drop the subindex x at p x . Thus, conditions (vi) and (vii) for problem B take the form: In the rest, the formulation of the MP remains unchanged.
3.2. Index θ . Let a process (x(t),û(t)), t ∈ [t 0 ,t 1 ] be a strong minimum in Problem B. With this process, we associate a family of finite-dimensional problems B θ and their optimal solutions, numbered by a certain index θ . By the index , we mean a collection of time instants and control values wheret 0 < t 1 ≤ . . . ≤ t s <t 1 , and the values of u k ∈ U, k = 1, . . . , s are arbitrary. The length s = s(θ) of the index depends on θ .
Next, define an interval [τ 0 , τ 1 ] as follows: take the interval [t 0 ,t 1 ], and at the points t 1 , . . . , t s insert segments of unit length, always preserving the position of the pointt 0 . As a result, we obtain the interval [τ 0 , τ 1 ] with endpoints τ 0 =t 0 , τ 1 =t 1 + s, and the inserted intervals Then Thus, t θ (τ ) is a piecewise linear nondecreasing function that maps [τ 0 , τ 1 ] onto [t 0 ,t 1 ], and ∆ k are intervals of constancy of t θ (τ ). Moreover, t θ (∆ k ) = t k for all k = 1, . . . , s. Define the functions Then u θ (·) is a bounded measurable function, u θ (τ ) ∈ U a.e. in [τ 0 , τ 1 ], and x θ (·) is an absolutely continuous function satisfying the relations , hence the endpoints of the new trajectory x θ (τ ) are the same as those of the original x(t), and therefore, in view of assumption (10), satisfy the inequalities Note that some points t k of the index θ may coincide: t k = . . . = t k = t * , therefore, at such a point t * several consecutive intervals ∆ k are inserted in succession, on each of which we have v θ (τ ) = 0 and u θ (τ ) = u k , where the value u k ∈ U corresponds to the interval ∆ k .
So, the set E 0 consists of a finite number of nonoverlapping intervals ∆ k , k = 1, . . . , s, and the set E + also consists of a finite number of some nonoverlapping intervals. (It does not really matter, whether or not they include their endpoints.) Let us unite all these intervals of the sets E 0 and E + into a general collection, denoted by σ k , k = 1, . . . , m. So, [τ 0 , τ 1 ] = σ 1 ∪ . . . ∪ σ m , and the different σ k do not overlap. Denote by χ k (τ ) the characteristic function of the interval σ k , k = 1, . . . , m.

3.3.
Problem B θ of the index θ . For a given index θ, we fix the constructed interval [τ 0 , τ 1 ] and fix the function u θ (τ ). Consider the space R m+n with the variables (z, Define the following problem B θ in the space R m+n : where x(·) is the solution to the Cauchy problem (15) with the function v(·), defined by (13). We call it the associated problem corresponding to the index θ. In view of (13), the control system of this problem has actually the form: Letẑ k be the value of v θ (τ ) on σ k , k = 1, . . . , m, i.e. v θ (τ ) = ẑ k χ k (τ ). Recall thatẑ k = 0 if σ k ⊂ E 0 , andẑ k = 1 if σ k ⊂ E + . We setẑ = (ẑ 1 , . . . ,ẑ m ) andx 0 =x(t 0 ). Obviously, for each pair (z, x 0 ) in a neighborhood of (ẑ,x 0 ), the value x 1 = x(τ 1 ) is uniquely determined by equation (17), and moreover, depends smoothly on this pair.
Since the process (x(t),û(t)), t ∈ [t 0 ,t 1 ] is a strong minimum in Problem B, it is easy to show that the point (ẑ,x 0 ) is a local minimum in Problem B θ .
All the data of Problem B θ is smooth, but this problem cannot be regarded as smooth, since it contains a continuum of inequality constraints Φ(x(τ )) ≤ 0, τ ∈ [τ 0 , τ 1 ]. In fact, this is a standard problem of "semi-infinite" mathematical programming, for which one can use the known necessary conditions for a local minimum (see Theorem A.1 in Section A). For the point (ẑ,x 0 ) in Problem B θ , these conditions are as follows.
Our nearest goal is to decipher these conditions. Note that in view of (12) and (iii), all dµ j (τ ) = 0 in some neighborhoods of τ 0 and τ 1 .
Thus, the preliminary deciphering of the stationarity conditions in Problem B θ results in the following Theorem 3.2. For any index θ, there exists a tuple (α 0 , α, β, µ θ (·)) with nondecreasing functions µ θ j (·), j = 1, . . . , d(Φ), and the corresponding function of bounded variation p θ (·), both continuous at τ 0 and τ 1 , such that the following conditions hold: Consider in more detail the second condition in (24). Let σ k = [τ , τ ] ⊂ E 0 . Then u θ (τ ) = u k on σ k . Further, let σ k = [τ * , τ * ] be the union of σ k with all segments of E 0 adjoining σ k (if any). Then v θ (τ ) = 0 on σ k , hence t θ (τ ) = t k is constant on σ k , and the value x θ (τ ) is also constant on σ k ; we denote it bŷ x * . Then, according to (23), The second condition in (24) says that Substituting here the value p θ (τ ) from (25), we obtain Then condition (27) implies Let us now summarize our findings for the index θ, returning back to the original time t .
Define the functions Obviously, each µ j (t) is a nondecreasing function having at every point t k a jump equal to the measure of the singleton {t k } : while p(t) is a function of bounded variation satisfying the equation and having the same endpoint values as p θ (τ ). Set µ(t) = (µ 1 (t), . . . , µ d(Φ) (t)).
Recall that by (12) all dµ j (τ ) = 0 in some neighborhoods of τ 0 and τ 1 , hence, all dµ j (t) = 0 in some neighborhoods oft 0 andt 1 , so the function p(t) is continuous at these points.
(Note that if all µ j are continuous at some t k , then (29) reduces to a simple inequality Thus, for any given index θ , we obtain a tuple (α 0 , α, β, µ(·)) of Lagrange multipliers and the corresponding function p(·) such that conditions (i) − (vii) of Theorem 3.3 are satisfied. These multipliers, in general, depend on the index θ. Conditions (i) − (v) are the same for all indices, while conditions (vi) − (vii) are associated with each individual index. Our goal now is to pass to conditions that do not depend on the index θ. Remark 1. The above technically simple but quite nontrivial in its idea trick with the replacement of condition (27) in the time τ by condition (28) including some unknown numbers ε j , that made it possible to pass to the corresponding condition (29) in the time t, was proposed by A.A. Milyutin in his unpublished lectures at Moscow State University in the 1970-s. It is this trick that allows to make the following key step in the proof of MP for problems with several state constraints. In the case of a single state constraint one do not need this trick, since the function p θ (τ )f (x * , u k ) is monotone, see [6].
3.6. Transition to the universal MP. Like in some other proofs of MP [15,5,6], we perform the following procedure 2 . For a given index θ, denote by Λ θ the set of all tuples λ = (α 0 , α, β, µ(t)) with nondecreasing functions µ j (t), for which there exists a function of bounded variation p(·) such that conditions (i) − (vii) hold with some ε j ∈ [0, 1], j = 1, . . . , d(Φ) . This is a set in the space A key fact is that Λ θ is a compact set in some topology (see Lemma C.1 in Appendix).
Thus, taking all possible indices θ, we obtain, for each of them, a nonempty compact set Λ θ . Let us show that the family of all these compacta form a centered system (i.e., have the finite intersection property). To this end, we introduce a partial order in the set of all indices. We say that θ 1 ⊂ θ 2 if each pair (t k , u k ) of θ 1 is contained in θ 2 . It is clear that, for any two indices θ 1 and θ 2 , there is a third one containing each of them, for example, their union. Obviously, when the index θ expands, the set Λ θ narrows, i.e., the inclusion θ 1 ⊂ θ 2 implies the inverse inclusion Λ θ1 ⊃ Λ θ2 . Now, let be given any finite collection of compacta Λ θ1 , . . . , Λ θj . Take any index θ containing all indices θ 1 , . . . , θ j . Then the nonempty compact set Λ θ is contained in each of the compacta Λ θ1 , . . . , Λ θj , and therefore, is contained in their intersection. Hence, the finite intersection property of the family {Λ θ } holds, and then its total intersection is nonempty: Now, take an arbitrary tuple of multipliers λ = (α 0 , α, β, µ) ∈ Λ * and let p(·) be the adjoint function corresponding to this tuple. By definition, conditions (i) − (v) hold for this tuple. The fulfillment of condition (vi) in any index θ means that for any interval (t , t ), t t p(t) f (x(t),û(t)) dt = 0, (because there is an index containing the points t , t ), which is equivalent to the fulfillment of equality Finally, the fulfillment of (vii) for the chosen "universal" tuple λ implies that for any u ∈ U and any point t ∈ (t 0 ,t 1 ) of continuity of all functions µ j (i.e. any point in (t 0 ,t 1 ) except for a countable set), we have Then this inequality holds for any u ∈ U and any interior point of the interval [t 0 ,t 1 ], and then also for its boundary points, since the function p(t) is continuous at these points. Assuming, without loss of generality, that the function p(t) is left continuous, we get: p(t) f (x(t), u) ≤ 0 for all u ∈ U and all t ∈ [t 0 ,t 1 ].
Thus, for the chosen tuple λ and the corresponding p all conditions (i) − (vii) of the maximum principle are satisfied. Theorem 2.1 for the autonomous Problem B is proved. 3.7. MP for problem A. It remains to show that the MP for problem A can be obtained from the MP for problem B, despite the fact that problem B is a particular case of problem A. Let us show that problem A can be reduced to the form (6)- (9). This is achieved by the following simple trick. To the controlled systemẋ = f (t, x, u) we add the equation dt/ dτ = 1, assuming that τ is a new time from a nonfixed interval τ ∈ [τ 0 , τ 1 ], and t = t(τ ) is a new state variable.
The functions x(·) and u(·) are also now considered to depend on the new time: x = x(τ ), u = u(τ ). As a result, we arrive at the following Problem A : where t(τ ), x(τ ) are the state variables and u(τ ) is the control, τ ∈ [τ 0 , τ 1 ].
Obviously, Problem A is of type B . It is easy to see that both the admissible and optimal processes of Problems A and A are in one-to-one correspondence. Therefore, obtaining necessary optimality conditions in Problem B, one can also obtain necessary conditions in Problem A. The Pontryagin function for the "autonomous" system (30) is H = p x f + p t , the "autonomous" conditions H(x, u) = 0 and H(x, u ) ≤ 0 take the form p x f (x, u) + p t = 0 and p x f (x, u ) + p t ≤ 0, respectively, that is, exactly the conditions (vi) and (vii) of MP. The details of this correspondence are left to the reader.
Appendix A. A general Lagrange multipliers rule.
Statement of the problem. Let X, Y, and Z i , i = 1, . . . , ν be Banach spaces, D ⊂ X an open set, and K i ⊂ Z i , i = 1, . . . , ν closed convex cones with nonempty interiors. Let F 0 : D → R, g : D → Y, and f i : D → Z i , i = 1, . . . , ν, be given mappings. Consider the following optimization problem: Here z * i , z i is the duality pairing between Z i and its dual space Z * i . We study the local minimality of an admissible point x 0 ∈ D.
It is worth noting that the inequality constraints f i (x) ≤ 0, where f i : D → R are given functionals, may also be presented in the form f i (x) ∈ K i if we put K i = R − := (−∞, 0]. Then K 0 i = R + := [0, ∞). On the other hand, all the inequality constraints f i (x) ∈ K i can be written as one constraint f (x) ∈ K if we define a mapping f : . . , f ν (x)), and a cone K = K 1 × . . . × K ν in Z. However, we chose the form (31) to keep a visual relation with convenient statements.
We impose the following Assumptions. 1) The objective function F 0 and the mappings f i are Fréchet differentiable at x 0 ; the operator g is strictly differentiable at x 0 (smoothness of the data functions), 2) the image of the derivative g (x 0 ) is closed in Y (weak regularity of the equality constraint).
Though all the mappings in the problem are differentiable, it is not a standard smooth problem, because the cones K i can be given, in general, by infinite numbers of linear inequalities. In this sense, problem (31) can be regarded as semi-smooth. (If the basic space X is finite-dimensional and at least one of Z i is infinite-dimensional, problem (31) is usually called semi-infinite.) The following theorem gives necessary optimality conditions in problem (31).
Theorem A.1. Let x 0 be a local minimum in problem (31). Then there exist Lagrange multipliers α 0 ≥ 0, z * i ∈ K 0 i , i = 1, . . . , ν, and y * ∈ Y * , not all equal zero, satisfying the complementary slackness conditions (i.e., each z * i is an outer normal to the cone K i at the point f i (x 0 ) ), and such that the Lagrange function is stationary at x 0 : L (x 0 ) = 0, i.e., The proof is based on well-known facts from the functional and convex analysis (see [4,Theorem 4], [7,8], and references therein). In our opinion, this theorem is reasonably general and, at the same time, most simple and convenient both in the proof and in the practical usage in many situations.
Remark 2. The functional F 0 in problem (31) can be taken not necessarily smooth; one can take it in the form F 0 (x) = ϕ 0 (f 0 (x)), where f 0 : X → Z 0 , similarly to other f i , is a smooth mapping, while ϕ 0 is a sublinear functional on Z 0 such that ϕ 0 (f 0 (x 0 )) = 0 and the cone K 0 = {z : ϕ 0 (z) ≤ 0} has a nonempty interior. Then the first terms in the functions L and L (x 0 ) should be replaced, respectively, by z * 0 , f 0 (x) and z * 0 f 0 (x 0 ), where z * 0 ∈ K 0 0 with z * 0 , f 0 (x 0 ) = 0, which makes them symmetric with respect to all indices i = 0, 1, . . . , ν. More details can be found in [8].
Appendix B. A generalized integration by parts.
Appendix C. The weak* compactness of the set Λ θ . Fix any index θ. Recall that the set Λ θ is defined by the conditions (i) − (vii) of Theorem 3.3.
Lemma C.1. The set Λ θ is weak* compact (that is, a compact set with respect to the usual convergence of finite-dimensional vectors (α 0 , α, β) and weak* convergence of the measures dµ in the space C * ).
First of all, we show that p is uniquely determined by the tuple λ from the adjoint equation (iv) : dp(t) = −p(t) f x (x(t)),û(t))) dt + Further, since the weak* topology on the bounded subsets of the space Y * is metrizable (because Y is separable), to establish the compactness of Λ θ it suffices to consider an arbitrary sequence of its elements λ n . We can assume that the vectors (α 0 , α, β) n converge to some (α 0 , α, β).
By condition (ii), the norms of all vector measures dµ n = ( dµ n 1 , . . . , dµ n d(Φ) ) are uniformly bounded, and all the initial values p n (t 0 ) = (l x0 ) n are bounded too, hence (37) implies that the functions p n (t) are uniformly bounded by the common constant. Then, according to the Helly theorems (see, e.g., [12, chapter 6]), there is a subsequence of p n (t) converging for each t to a function of bounded variation p(t), while the measures dp n weakly* converge to the measure dp. We retain the previous numbering for this subsequence, and also assume that the vector measures dµ n weakly* converge to some vector measure dµ connected with dp by relations (36) and (37).
It is clear that the conditions (i), (ii), (iv), (v) and the first equality in (iii) remain valid under this transition to the limit. Take any j ∈ {1, . . . , d(Φ)}. The second condition in (iii) means that, on any interval lying between neighboring points t k < t k+1 , on which Φ j (x(t)) < 0, we have dµ j (t) = 0. This is equivalent to the fact that, for any continuous function η(t), whose support is contained in this interval, we have η(t) dµ j (t) = 0. Obviously, this property is preserved when passing to the weak* limit.