(Almost) Everything You Always Wanted to Know About Deterministic Control Problems in Stratified Domains

We revisit the pioneering work of Bressan \&Hong on deterministic control problems in stratified domains, i.e. control problems for which the dynamic and the cost may have discontinuities on submanifolds of R N . By using slightly different methods, involving more partial differential equations arguments, we (i) slightly improve the assumptions on the dynamic and the cost; (ii) obtain a comparison result for general semi-continuous sub and supersolutions (without any continuity assumptions on the value function nor on the sub/supersolutions); (iii) provide a general framework in which a stability result holds.


Introduction
In a well-known pioneering work, Bressan & Hong [14] provide a rather complete study of deterministic control problems in stratified domains, i.e. control problems for which the dynamic and the cost may have discontinuities on submanifolds of R N .In particular, they show that the value-function satisfies some suitable Hamilton-Jacobi-Bellman (HJB) inequalities (in the viscosity solutions' sense) and were able to prove that, under certain conditions, one has a comparison result between sub and supersolutions of these HJB equations, ensuring that the value function is the unique solution of these equations.
The aim of this article is to revisit this work by (i) slightly improving the assumptions on the dynamic and the cost, in a (slightly) more general framework; (ii) obtaining a comparison result for general semi-continuous sub and supersolutions (while in [14] the subsolution has to be Hölder continuous, and this turns into an a priori assumption on the value function that we do not need here) ; (iii) providing a general (and to our point of view, natural) framework in which a stability result holds.
In order to be more specific, even if we are not going to enter into details in this introduction, the first key ingredient is the "stratification", namely writing the whole space as where, for each k = 0 . . .N , M k is a k-dimensional embedded submanifolds of R N , the M k being disjoint.The reader may consider that the M k are the subsets of R N where the dynamic and cost have discontinuities, which may also mean that, as in [14], on certain M k , there is a specific control problem whose dynamic and cost have nothing to do with the dynamic and cost outside M k .But as in our previous papers in collaboration with A. Briani dedicated to co-dimension 1 type discontinuities [6,7], part of the dynamic and cost on M k is some kind of "trace" of the dynamic and cost outside M k .In [14], the regularity imposed on the M k is C 1 , while in our case it depends on the controllability of the system: C 1 is the controllable case, W 2,∞ otherwise.This additional regularity may be seen as the price to pay for having no continuity assumption on either the value function nor the subsolutions for obtaining the equation and proving the comparison result.
The next ingredient is the control problem, i.e. the dynamic and cost.Here we are not going to enter at all into details but we just point out key facts.First, contrarily to [14], we use a general approach through differential inclusions and we do not start from dynamic b k and cost l k defined on M k .This may have the disadvantage to lead to a more difficult checking of the assumptions in the applications but, for example, since most of our arguments are local, the global Lipschitz assumption on the b k can be reduced to a locally Lipschitz one.But the most interesting feature are the controllability assumptions -and we hope to convince the reader that they are natural: for each k, we assume that the system is controllable w.r.t. the normal direction(s) of M k in a neighborhood of each M k , while the dynamic and cost should also satisfy some continuity assumptions in the tangent direction(s).This controllability assumption has a clear interpretation: if, in a neighborhood of M k , the controller wants to go to M k , then he can do it, and in the same way he can avoid M k if this is its choice.This avoids useless discontinuities (which are not "seen" by the system) and cases where the value functions have discontinuities.We point out that this normal controllability is a key assumption to prove that the value function satisfies the right HJB inequalities without assuming a priori that it is continuous but also it is a key argument in the comparison and stability results as this was already the case in [7].
Except our slightly different approach, the viscosity sub and supersolutions inequalities are the same as in [14], even if the formulation coming from the differential inclusion and the set-valued maps for the dynamic and cost changes a litle bit the form of the Hamiltonians.The next step is more important since it concerns the comparison of any semi-continuous sub and supersolutions: here our proof differs from [14] since it involves more partial differential equations (pde for short) arguments and less control ones.A key step, already used but not in a such systematic way in [7], is to completely localize the comparison result, i.e. to reduce to the proof of comparison results in (small) balls.Once this is done, the assumptions on the M k allow to reduce the case when they are just affine subspaces and the key arguments of [7] can be applied (regularisation in the tangent directions to M k combined by a key control-pde lemma).It is worth pointing out anyway that, as in [14], the proof is done by induction on the codimension of the encountered discontinuities: local comparison in M N , then successively in .., the previous comparison result providing the key argument for the next step.We refer to the beginning of Section 5 for a more explicit exposition of the induction argument.
Finally we provide the stability result, which extends the one proved in [7] to the more complicated framework we have here but the idea remains the same: roughly speaking, the normal controllability implies that the half-relaxed limits on M k can be computed by using only the restrictions of the functions on M k , allowing to pass to the limit on the specific inequalities on M k (in particular for the subsolutions).
Recently, control problems in either discontinuous coefficients situations or in stratified domains or even on networks have attracted more and more attention.Of course, we start by recalling the pioneering work by Dupuis [22] who constructs a numerical method for a calculus of variation problem with discontinuous integrand.Problems with a discontinuous running cost were addressed by either Garavello and Soravia [25,26], or Camilli and Siconolfi [17] (even in an L ∞ -framework) and Soravia [38].To the best of our knowledge, all the uniqueness results use a special structure of the discontinuities as in [20,21,27] or an hyperbolic approach as in [3,19].More in the spirit of optimal control problem on stratified domains are the ones of Barnard and Wolenski [11] (for flows invariances), Rao and Zidani [34] and Rao, Siconolfi and Zidani [35] who proved comparison results but with more restrictive controlability assumptions and without the stability results we can provide.For problems on networks which partly share the same kind of difficulties, we refer to Y. Achdou, F. Camilli, A. Cutri, N. Tchou [2], C. Imbert, R. Monneau, and H. Zidani [28], F. Camilli and D. Schieborn [15] and C. Imbert and R. Monneau [29,30] where more and more pde methods are used, instead of control ones.A multi-dimensional version (ramified spaces) for Eikonal type equations is given F. Camilli, C. Marchi and D. Schieborn [16] and for more general equations in C. Imbert and R. Monneau [29].
The article is organized as follows: in Section 2, we describe the control problem in a full generality; this gives us the opportunity to provide all the notations and recall well-known general results which are useful in the sequel (in particular the results related to supersolutions properties).Then we have to revisit the notion of stratification and we take this opportunity to introduce the assumptions we are going to use throughout this article (Section 3).Section 4 contains the (subsolutions) properties which are specific to this context.Then we address the question of the comparison result (Section 5), reducing first to the case of the comparison in (small) balls which allows to flatten the submanifolds M k .Section 6 is devoted to the stability result and we conclude the article with a section collecting typical examples and extensions.In this section, we consider control problems in R N where the dynamics and costs may be discontinuous on the collection of submanifolds M k for k < N .In this first part, we describe the approach using differential inclusions and we recall all the properties of the value-function which are always true, i.e. results where the structure of the stratification does not play any role.This is the case for all the supersolution type properties of the value function.This part is essentially expository and is kept here in order to have a self-contained article for the reader's convenience.On the contrary, the subsolution's properties of the value function are more specific and described in Section 4.

TERMINOLOGY -(AFS) Admissible Flat Stratification (HJB-SD)
We first define a general control problem associated to a differential inclusion.As we mention it above, at this stage, we do not need any particular assumption concerning the structure of the stratification, nor on the control sets.
Dynamics and costs -We treat them both at the same time by embedding the cost in the differential inclusion we solve below.We denote by P(E) the set of all subsets of E.
(H BL ) We are given a set-valued maps BL : R N × [0, T ] → P(R N +1 ) satisfying (i) The map (x, t) → BL(x, t) has compact, convex images and is upper semi-continuous; (ii) There exists M > 0, such that for any x ∈ R N and t > 0, where |•| stands for the usual euclidian norm in R N (which reduces to the absolute value in R, for the l-variable).When necessary, we sometimes decompose BL(x, t) as (B(x, t), Assumption (H BL )-(ii) means that both the dynamics and the running costs are uniformly bounded.
We recall what upper semi-continuity means here: a set-valued map x → F (x) is upper-semi continuous at x 0 if for any open set O ⊃ F (x 0 ), there exists an open set ω containing x 0 such that F (ω) ⊂ O.In other terms, F (x) ⊃ lim sup y→x F (y).
The control problem -as we said, we embed the accumulated cost in the trajectory by solving a differential inclusion in R N × R: we look for trajectories (X, L)(•) of the following inclusion d dt (X, L)(s) ∈ BL X(s), t − s for a.e.s ∈ [0, t) , and (X, L)(0) = (x, 0) .
Under (H BL ), it is well-known that given (x, t) ∈ R N × (0, T ], there exists a Lipschitz function (X, L) : [0, t] → R N × R which is a solution of this differential inclusion.To simplify, we just use the notation X, L when there is no ambiguity but we may also use the notations X x,t , L x,t when the dependence in x, t plays an important role.If (X, L) is a solution of the differential inclusion, we have for almost any s ∈ (0, t), ( Ẋ, L)(s) = (b, l)(s) for some (b, l)(s) ∈ BL(X(s), t − s).However, throughout the paper we prefer to write it this way in order to remember that both b and l correspond to a specific choice in BL(X(s), t − s).
Then, we introduce the value function where T (x, t) stands for all the Lipschitz trajectories (X, L) of the differential inclusion which start at (x, 0) and the function g : R N → R is the final cost.We assume throughout the paper that g is bounded and uniformly continuous.
A key standard result is the Theorem 2.1 (Dynamic Programming Principle) Under Assumptions (H BL ), the value-function U satisfies Next we introduce the "usual" Hamiltonian H(x, t, p) for x ∈ R N , t ∈ [0, T ] and p ∈ R N defined as Using (H BL ), it is easy to prove that H is upper semi-continuous (w.r.t.all variables) and is convex and Lipschitz continuous as a function of p only.
The second (classical) result is the Theorem 2.2 (Supersolution's Property) Under Assumptions (H BL ), the value-function U is a viscosity supersolution of In Theorem 2.2, we use the classical definition of viscosity supersolution introduced by H. Ishii [31].Indeed, under (H BL ), the Hamiltonian H is a locally bounded, usc function which is defined everywhere and therefore H * = H.Here and below we have chosen a formulation which holds up to time T , i.e. on (0, T ] instead of (0, T ), to avoid the use of terms of the form η/(T − t) in comparison proofs or results like [5,Lemma 2.8,p.41]We also point out that both Theorem 2.1 and 2.2 hold in a complete general setting, independently of the stratification we may have in mind.
We conclude this first part by a converse result showing that supersolutions always satisfy a superdynamic programming principle: again we remark that this result is independent of the possible discontinuities for the dynamic or cost.
Proof -For M given by (H BL ), we consider the sequence of Hamiltonians where dist(•, BL(y, s)) denoting here the distance here the set BL(y, s).Noticing that ψ is Lipschitz continuous and that ψ(b, l, x, t) = 0 if (b, l) ∈ BL(x, t), the following properties are easy to obtain (i) For any δ > 0, H δ ≥ H and therefore v is a lsc supersolution of v t + H δ (x, t, Dv) = 0, (ii) The Hamiltonians H δ are (globally) Lipschitz continuous w.r.t.all variables, (iii) H δ ↓ H as δ → 0, all the other variables being fixed.
By using (i) and (ii), it is clear that v satisfies the Dynamic Programming Principle for the control problem associated to H δ , namely To conclude the proof, we have to let δ tend to 0. To do so, we pick an optimal or δ-optimal trajectory, i.e. (X δ , L δ ) such that inf Since Ẋδ = b δ , Lδ = l δ are uniformly bounded, standard compactness arguments imply that up to the extraction of a subsequence, we may assume that X δ , L δ converges uniformly on [0, t ∧ σ] to (X, L).And we may also assume that they derivatives converge in L ∞ weak-* (in particular Lδ = l δ ).
We use the above property of X δ , L δ , namely in two ways: first by multiplying by δ, we get But ψ is convex in (b, l) since the BL(y, s) are convex and if (b δ , l δ ) converges weakly to (b, l) (and X δ converges uniformly), we have Finally we remark that ψ ≥ 0 and ψ(b, l, x, t) = 0 if and only if (b, l) ∈ BL(x, t), therefore (X, L) is a solution of the BL-differential inclusion.
In order to conclude, we come back to (3) and we remark that and we pass to the limit in this inequality using the lower-semicontinuity of v, together with the weak convergence of l δ and the uniform convergence of X δ .This yields and recalling that (X, L) is a solution of the BL-differential inclusion and taking the infimum in the left-hand side over all solution of this differential inclusion gives the desired answer. Q.E.D.
In this section, we define the notion of Admissible Stratification, which specifies the structure of the discontinuity set of BL as was considered in [14].We point out that, besides of the precise regularity we will impose in connection with the control problem, this notion is nothing but the notion of Whitney Stratification, based on the Whitney condition [40,41], see below Lemma 3.2 and Remark 3.3.We first do it in the case of a flat stratification; the non-flat case is reduced to the flat one by suitable local charts.

Admissible Flat Statification
We consider here the stratification introduced in Bressan and Hong [14] but in the case when the different embedded submanifolds of R N are locally affine subspace of R N .More precisely where the M k (k = 0..N ) are disjoint submanifolds of R N .We say that M = (M k ) k=0..N is an Admissible Flat Stratification (AFS), the following set of hypotheses is satisfied We point out that these assumptions are equivalent (for the flat case) to the assumptions of Bressan & Hong [14].Indeed, we both assume a decomposition such that the submanifolds are disjoints and the union of all of them coincide with R N but in order to describe the allowed stratifications we define in a different way the submanifolds M k .The key point is that for us M k is here a k-dimensional submanifold while, in [14], the M j can be of any dimension.In other words, our M k is the union of all submanifolds of dimension k in the stratification of Bressan & Hong.With this in mind it is easier to see that our assumptions (AFS)-(ii)-(iii) are equivalent to the following assumption of Bressan and Hong: if M k ∩ M l = ∅ then M k ⊂ M l for all indices l, k without asking l > k in our case.But according to the last part of (AFS)-(i), M k ∩ M l = ∅ if l < k: indeed for any x ∈ M k , there exists r > 0 such that B(x, r) ∩ M l = ∅.This property clearly implies (AFS)-(iii).
In order to be more clear let us consider a stratification in R 3 induced by the upper half-plane {x 3 > 0, x 2 = 0} and the x 2 -axis (see figure 1. below).The stratification we use in this case requires first to set M 2 = {x 3 > 0, x 2 = 0}.The boundary of M 2 which is the x 1 -axis is included in M 1 ∪ M 0 and of course, we have to set here M 0 = {(0, 0, 0)}.Thus, M 1 consists of four connected components which are induced by the x 1 -and x 2 -axis (but excluding the origin, which is in M 0 ).Notice that in this situation, the x 3 -axis has no particular status, it is included in M 2 .
On the other hand, notice that (AFS)-(ii) FORBIDS the following decomposition of R 3 As a consequence of this definition we have following result which will be usefull in a tangential regularization procedure (see Figure 2 Proof -We first consider the case when l = k +1.We argue by contradiction assuming that there can be in the ball.Therefore x 0 belongs to some M k , a contradiction since B(x, r) ∩ M k = B(x, r) ∩ (x + V k ) which would imply that y ∈ M k .
For l > k + 1, we argue by induction.If we have the result for l, then we use the same proof as above if y ∈ M l+1 : there exists z ∈ B(x, r ) ∩ (y + V k ), z / ∈ M l+1 and we build in a similar way x 0 ∈ M l+1 − M l+1 = M l .But this is again a contradiction with the fact that the result holds for l; In this flat situation, the tangent space at x is T x := x + V k while the tangent space at y is T y := y + V l , where l > k.The previous lemma implies that if (y n ) n is a sequence converging to x, then the limit tangent plane of the T yn is x + V l and it contains T x , which is exactly the Whitney condition -see [40,41].

General Regular Stratification
(ii) for any x ∈ R N , there exists r = r(x) > 0 and a C 1,1 -change of coordinates Remark 3.5 If we need to be more specific, we also say that (M, Ψ) is a stratification of R N , keeping the reference Ψ for the collection of changes of variable (Ψ x ) x .This will be usefull in Section 6 when we consider sequences of stratifications.
Notations -The definition of regular stratifications (flat or not) allows to define, for each x ∈ M k , the tangent space to M k at x, denoted by T x M k , which can be identified to R k .Then, if x ∈ M k and if r > 0 and V k are as in (AFS)-(i), we can decompose At this stage, it remains to connect the stratification with the set-valued map BL.To do so, we first recall that the set function BL is said to be continuous at (x, t) ∈ R N × R + if dist H (BL(y, s), BL(x, t)) → 0 when (y, s) → (x, t), where dist H (•, •) stands for the Haussdorf distance between sets.Now, given a regular stratification

Hamiltonians
Considering a regular stratification M adapted to BL, we introduce the associated Hamiltonians: if x ∈ M k , t ∈ [0, T ] and p ∈ T x M k , the tangential Hamiltonian on the M k -submanifold is defined by The continuity requirements on the maps BL| k (see above) together with the compactness of each BL(x, t) implies the continuity of H k in (x, t, p), for any k.In this definition (where we have implicitly identified T x M k as a subspace of R N ), it is clear that H k depends on p only through its projection on T x M k but we keep the notation p to simplify the notations.
Notice that in the special case k = 0, since T x M 0 = {0} the Hamiltonian reduces to: In order to prove comparison for the complemented problem, we need some assumptions on the Hamiltonians that we formulate first in the case of an (AFS).
For any x ∈ R N , if x ∈ M k and r = r(x) is given by (AFS)-(i), there exist three constants We point out the importance of Lemma 3.2 which implies that this is actually an assumption on any y 1 (or y 2 ) of M j .
(NC) Normal Controllability: for any 0 ≤ k < j ≤ N , for any t ∈ [0, T ], if y ∈ M j ∩ B(x, r) then In particular, in the special case k = 0, we have p = p ⊥ .So, (NC) implies the coercivity w.r.t p of all the Hamiltonians H k , k = 1..N , in a neighborhood of any point x ∈ M 0 (recall that such points are isolated).
(LP) Lipschitz continuity: because of the boundedness of BL, there exists C 3 such that, for any 0 It is worth pointing out that these assumptions (except perhaps (LP)) are local assumptions since they have to hold in a neighborrhood of each point in R N and the different constants or modulus of continuity may depend on the considered point.The strategy of proof for the comparison result will explain this unusual feature and in particular Lemma 5.4.(ii) In the general case, we say that the associated Hamiltonians satisfy the Assumption on the Hamiltonians in the general case (AHG) if the Hamiltonians Hk (y, t, q) := H k (χ(y), t, χ (y)q) satisfy the (LAHF), where χ = (Ψ x ) −1 .
In order to be complete, we give below sufficient conditions in terms of BL for (TC) & (NC) to hold: the first one concerns regularity and the second one ensures the normal coercivity of the Hamiltonians.
(NC-BL) There exists δ > 0 such that, for any 0 Here also, the case k = 0 is particular: we impose a complete controllability of the system in a neighborhood of x ∈ M 0 since the condition reduces to B(0, δ) ⊂ B(y, t) because This normal controllability assumption plays a key role in all our analysis: first, in the proof of Theorem 4.1 below, to obtain the viscosity subsolution inequalities for the value function, in the comparison proof to allow the regularization (in a suitable sense) of the subsolutions and, last but not least, for the stability result.(ii) for any k = 0..(N − 1), U is a subsolution of In this result, we point out -even if it is obvious-that (ii) is a viscosity inequality for an equation restricted to M k and this is why point (i) is an important fact since it allows to restrict everything (including the computation of the usc envelope) to M k .
Proof -We provide the proof in the case of an (AFS), the general case resulting from a simple change of variable.
We consider x ∈ M k , t ∈ (0, T ] and a sequence (x ε , t ε ) → (x, t) such that We have to show that we can assume that x ε ∈ M k .
We assume that, on the contrary, x ε / ∈ M k and we show how to build a sequence of points (x ε , tε ) ε with xε ∈ M k for any k and with U * (x, t) = lim ε U (x ε , tε ).By Theorem 2.1, we have for any solution (X, L) of the differential inclusion starting from (x ε , 0).Let xε be the projection of k and by (NC-BL), there exists b ∈ B(y, s) for any y ∈ B(x, r) (the ball given by (AFS)- Choosing such a dynamic b (with any constant cost l), it is clear that X(s) ∈ B(x, r) for s small enough (independent of ε) and for s Finally since s ε → 0 as ε → 0, we deduce that lim sup To prove (ii), we assume now that x ε ∈ M k and we use again Theorem 2.1 which implies for any solution (X, L) of the differential inclusion starting from (x ε , 0).Using the continuity of BL| k , if (b, l) is in the interior of BL| k (x, t), the trajectory X(s), starting from x ε at time t ε remains on M k for s ∈ [0, τ ] if τ is small enough (but independent of ε).Thus, the viscosity inequality can be obtained as in the standard case and we obtain the inequality for (b, l) is in the whole BL| k (x, t) by a simple passage to the limit. Q.E.D.
The sub and supersolution properties of the value function naturally leads us to the following definition.
for any k = 0..N , i.e if, for any test-function φ ∈ C 1 (M k × [0, T ]) and for any local maximum point In the sequel, we also say that a function is a subsolution or a supersolution of (HJB-SD) in a domain D ⊂ R N × (0, T ] if the above properties hold true either in M k × (0, T ] ∩ D or in D. We also say that u is a strict subsolution of (HJB-SD) in a domain D ⊂ R N × (0, T ] if the inequality ≤ 0 is replaced by ≤ −η for some η > 0. Remark 4.3 As in [6,7], we notice that additional subsolution conditions involving the tangential Hamiltonians (H k ) k=0..N are required on the manifolds M k 's .It might be surprising anyway that we have no subsolution condition related to trajectories which are leaving M k for k < N .In fact, even if we are not going to enter into details here, these conditions can be deduced from the inequalities on M l for l > k in the spirit of [6, Theorem 3.1].

Comparison, Uniqueness and Continuity of the Value-Function
In this section, we provide our main comparison result for (HJB-SD).Since the proof relies on proving comparison properties in different subdomains of R N , we introduce the following definition.

Definition 5.1 We have a comparison result for (HJB-SD
where Ω is an open subset of R N and 0 ≤ t 1 < t 2 ≤ T , if, for any bounded usc subsolution u of (HJB-SD) in Q and any bounded lsc supersolution v of (HJB-SD) in Q, then Our main result is the Theorem 5.2 We have a comparison result for (HJB-SD) in any subdomain In order to guide the reader in the long and unusual proof (despite it has some common features with the global strategy in Bressan & Hong [14] and uses locally the ideas of [7]), we indicate the main steps.
• We first show that, instead of proving a "global" comparison result, we can reduce to comparison results in "small" balls.Essentially this first step allows us to reduce to the case of "flat stratifications", namely (AFS).
• Then we argue by induction on the dimension of the submanifolds which are contained in the small ball: if the small ball is included in M N , this means that there is no no discontinuities and we have a standard comparison result.The next step consists in proving a comparison result in the case when the ball intersects both M N and M N −1 , which is actually already done in [7].Therefore the induction consists in proving that if we have a comparison result for any ball intersecting (possibly) M N , . . ., M k+1 , then it is also true for any ball intersecting M N , . . ., M k+1 , M k .
• To perform the proof of this result, we use three key ingredients: for the subsolution, the regularization by sup-convolution and then by usual convolution in the tangent direction to M k (and this is where the (AFS) structure is playing a key role, see Lemma 3.2) together with the fact that a comparison result in M k+1 ∪ M k+2 • • • ∪ M N implies that subsolutions satisfy a sub-optimality principle in this domain.On the other hand, for the supersolution, the DPP allows us to prove an analogous "magic lemma" as in [6,7].
In order to formulate the induction, let us introduce the following statement, where k ∈ {0, ..., N } Q(k): For any ball B ⊂ M k ∪M k+1 ∪• • •∪M N , for any 0 ≤ t 1 < t 2 ≤ T and for any strict subsolution u of (HJB-SD) in B × (t 1 , t 2 ] and any supersolution v of (HJB-SD) in B × (t  Proof -Let u, v be respectively a bounded usc subsolution u and a bounded lsc supersolution v of (HJB-SD) in Q.We consider M = sup Q(u − v).If M ≤ 0 then we have nothing to prove, hence we may assume that M > 0.
In order to replace the "sup" by a "max" if Ω is unbounded, we argue as in [6,7] and we replace u by Proving the comparison inequality for u α instead of u provides the result by letting α tend to 0.
With this argument, we can consider M α = max Q(u α − v) and we denote by (x, t) a maximum point of u α − v.In addition, we may assume that t is the minimal time for which there exists such a maximum point.If t = 0 or x ∈ ∂Ω then the result is proved, hence we may also assume that x ∈ Ω and t > 0.
Using the assumption, we know that there exists r, h > 0 such that such that B r (x) ⊂ Ω, h ≤ t and one has a comparaison result in B(x, r) × (t − h, t) for any r ≤ r and h ≤ h.Thus, in Q r,h := B(x, r) × (t − h, t) (where r and h will be chosen later), we change u α (y, s) into and as a consequence of the comparison property, we have and, since (r 2 +1) 1/2 −1 > 0, if we choose (and fix) h small enough, we have β(− Ch+(r 2 +1) 1/2 −1) > 0. Therefore, for such h, On the other hand, for s = t − h, since t is the minimal time for which the maximum M α is achieved, we have u α (y, s) − v(y, s) < M α and β( C(s − t) + (|y − x| 2 + 1) 1/2 − 1) ≥ −β Ch.Since h is fixed, choosing β small enough, we have max This shows that max ∂pQ r,h (u α (y, s) − β( C(s − t) + (|y − x| 2 + 1) 1/2 − 1) − v(y, s)) < M α , a contradiction since u α (x, t) − v(x, t) = M α .Therefore the maximum of u α − v is achieved either on ∂Ω or for t = 0 and the complete comparison result is obtained by letting α tend to 0. Q.E.D.
In the direction of getting local comparison, we use below that under Q(k) we have a partial local comparison result for any ball which does not intersect the M j for j < k Proposition 5.5 Let B be a ball in R N such that B ∩M j = ∅ for any j < k.If Q(k) holds, then one has a comparison between sub and supersolutions of (HJB-SD) in B ×(t 1 , t 2 ] for any where Proof -For any η > 0, u − ηt is a strict subsolution of (HJB-SD) in B × (t 1 , t 2 ].Looking at a maximum point of (u − ηt) − v in Q, we see that Q(k) implies that such a maximum point cannot be in B × (t 1 , t 2 ].Therefore all the maximum points are on ∂ p Q and therefore Letting η tends to 0 provides the result. Q.E.D.

Properties of sub and supersolutions
A consequence of the partial local comparison result deriving from Q(k) is a sub-dynamic programming principle for subsolutions Lemma 5.6 Let u be an usc subsolution of (HJB-SD) and assume that Q(k) is true for some k ∈ {0, ..., N }.Then, for any bounded domain Ω ⊂ R N such that Ω ∩ M j = ∅ for any j < k, the subsolution u satisfies a sub-dynamic programming principle in Ω × [0, T ]: namely, for any σ ∈ where S(Ω) is the set of all stopping times θ such that X(θ) ∈ ∂Ω.
Proof -Since u is usc, we can approximate it by a decreasing sequence {u n } of continuous functions.Then we consider initial-boundary value problem (associated to an exit time control problem) on ∂Ω × (0, T ) .
Since u is an usc subsolution of (HJB-SD) and since u ≤ u n on ∂ p Q, then u is a subsolution of this problem.On the other hand, using that Q(k) is true, the arguments of [5] (Section 5.1.2,see Thm 5.7) show that is the maximal subsolution (and solution) of this initial value problem.Therefore u ≤ w n in Q.In order to obtain the result, we choose any (fixed) trajectory X and any cost L and write that, by the above inequality where 0 ≤ θ n ≤ T is the stopping time where the supremum is achieved.But θ n is bounded and X(θ n ∧ σ) ∈ ∂Ω which is compact.Therefore extracting some subsequence we may assume that θ n → θ and X(θ n ∧ σ) → X( θ ∧ σ).But, by using that (u n ) n is a decreasing sequence, it is easy to prove that lim sup and therefore Passing to the supremum in the right-hand and using that this is true for any choice of X, L yields the result. Q.E.D.
Remark 5.7 It is worth pointing out that, if x ∈ Ω then there exists η > 0 such that τ Ω > η for any trajectory X.This is a consequence of the boundedness of BL.Therefore, if we take σ < η, we clearly have a more classical formulation of the Dynamic Programming Principle.
The next step is the Lemma 5.8 Let x be a point in M k and t, h > 0. There exists r > 0 such that if u is a subsolution of (HJB-SD) in B(x, r ) × (t − h, t), then for any a ∈ (0, r ), there exists a sequence of usc functions ( 1) We recall that lim sup * u ε (x, t) = lim sup (y,s)→(x,t) ε→0 u ε (y, s).
Concerning the supersolutions now, a key argument that was used in [6,7] is that they satisfy an alternative: either the trajectories are leaving the discontinuity set, or there is a strategy which allows to remain on this set and we deduce an inequation for the tangential Hamiltonian there.
The situation is more complex here since the discontinuity set is composed of submanifolds with different dimensions.But still, a similar alternative can be derived.In order to formulate it, let us introduce some notations.
Consider a point x 0 ∈ M k for some k ∈ {0, ..., (N − 1)}, t 0 ∈ [0, T ] and a sequence (x n , t n ) → (x 0 , t 0 ).For any j ∈ {0, ..., (N − 1)}, we denote by τ n (j) the reaching time where (X xn,tn , L xn,tn ) is a given solution of the differential inclusion such that (X xn,tn (t n ), L xn,tn (t n )) = (x n , 0).Notice that by (AFS)-(i), τ n (j) > 0 for any j < k.Lemma 5.9 Let v be a bounded lsc viscosity supersolution of (HJB-SD) and φ ∈ C 1 (R N × (0, T ]) be a test-function such that the restriction of v − φ to M k × (0, T ] has a local minimum point at (x, t) ∈ M k × (0, T ].Then the following alternative holds A) either there exists τ > 0, a sequence (x n , t n ) → (x, t) and a sequence of trajectories (X xn,tn , L xn,tn ) satisfying τ n (j) ≥ τ for any j ≤ k and Proof -Since the result is local, we can prove it only in the case of an (AFS), the general result being obtained by changing variables.Therefore, we assume in particular in the sequel that M k is a subspace and even that we can assume that it is a strict local minimum point by standard arguments.As we already noticed, (H M )-(iv) implies that there exists τ 0 > 0 such that τ n (j) ≥ τ 0 for all j < k.In order to "push" the minimum point away from x ∈ M k , we construct the following test-function where ε > 0 and q ∈ (M k ) ⊥ , where (M k ) ⊥ is the vector space which is orthogonal to M k .We point out that (M k ) ⊥ can be identified with R N −k .
In order to choose q, we introduce the function χ : R N → R defined by χ(q) := φ t (x, t) + H x, t, Dφ(x, t) + q , which is convex and coercive in R N .In fact, we are interested in the restriction of χ to (M k ) ⊥ and we denote by ϕ := χ /(M k ) ⊥ .If the minimum of ϕ is achieved at q ∈ (M k ) ⊥ , then the classical property for the subdifferential of a convex function at a minimum point (0 ∈ ∂ϕ(q)) can be reinterpreted here as This fact can easily be proved using the identification between (M k ) ⊥ and R N −k , the fact that, in R N −k , we have 0 in the subdifferential of ϕ can be interpreted as the existence of an element in ∂χ(q) which is in M k .
Finally, taking into account the definition of H, the fact that BL(x, t) is convex and classical result on convex function, namely Danskin's Theorem which has to be translated again from R N −k to (M k ) ⊥ , then there exists (b, l) ∈ BL(x, t) such that But we are in the (AFS) case where T x M k = M k and the above property yields If ϕ(q) ≥ 0, then B) holds and we are done.Hence we may assume that ϕ(q) < 0.
From now on we consider the function φ ε with the choice q = q.Notice that, in this case φ ε = φ on M k × (0, T ]: the distance term clearly vanishes and q is orthogonal to z − x if z ∈ M k .Since (x, t) is a strict local minimum point of v − φ on M k × (0, T ), there exists a sequence (x ε , t ε ) of local minimum points of v −φ ε in R N ×(0, T ) which converges to (x, t).There are two possibilities.
Second case: there exists a subsequence of (x ε , t ε ) denoted by (x n , t n ) such that x n / ∈ M k .
Step 1 -Notice first that necessarily we have τ n (k) → 0. Thus, between times t = 0 and t = τ n (k), X xn,tn (s) remains inside a ball B ⊂ R N such that B ∩ M j = ∅ for any j ≤ k.By Lemma 2.3 we can use the super dynamic programmation principle for v(x n , t n ) between times 0 and τ n ∧ τ B , where we write τ n for τ n (k).Taking n large enough so that τ n < τ B , we get Since (X xn,tn , L) satisfies the differential inclusion, we have for some function b such that for any s ∈ (0, τ n ), b(X xn,tn (s), t n − s) ∈ B(X xn,tn (s), t n − s).Hence, taking the test-function φ ε we have, on one hand and, on the other hand Therefore (7) Combining (6) with the above properties, we get Step 2 -By the properties (H BL ), we claim that there exists a couple (b, l) ∈ BL(x, t) such that, at least along a subsequence By the bounds for b n , l n , we deduce that at least along a subsequence still denoted by b n , l n , we have (b n , l n ) → (b, l) for some (b, l) ∈ R N × R. Now, since the images of the BL(z, s) are convex and since BL is upper semi-continuous, dist H ((BL) εn (x, t), BL(x, t)) → 0 as ε n → 0 and we deduce that (b, l) ∈ BL(x, t).
Step 3 -Passing to the limit in (8) as τ n → 0 yields But this is in contradiction with the assumption that ϕ(q) < 0. Hence, either A) holds or this second case cannot happen and then B) holds.This ends the proof. Q.E.D.

Proof by induction on the dimension of M k
As we already noticed above, Q(N ) necessarily holds true since in this case the ball does not intersect any discontinuity.Moreover, we proved in [7] that Q(N − 1) is also true.Of course, Q(0) means that we have a comparison result without any restriction on the submanifolds M k which intersects B(x, r).Thus, the proof of Theorem 5.2 is reduced to the following backwards induction property T , an usc function u which is a strict subsolution of (HJB-SD) in B × (t 1 , t 2 ] and a lsc supersolution v of (HJB-SD) in B × (t 1 , t 2 ]. In order to check Q(k − 1) we have to show that u − v cannot have a maximum point (x, t) in B × (t 1 , t 2 ].But by Q(k), x cannot belong to any M j for j ≥ k.Therefore, we are left with the case where x ∈ M k−1 .Using (RS) and (AFS)-(i), we consider a smaller ball B such that B ⊂ B still containing x and such that B ∩ M j = ∅ for any j < k − 1.
Using that the Hamiltonians H and H j are Lipschitz continuous in p, we can replace u by ū(x, t) := u(x, t) − δ((t − t) 2 + |x − x| 2 ) for δ > 0 small enough: this new function is still a strict subsolution and (x, t) is a strict local maximum point of ū − v.
Next we use Lemma 5.8 for the subsolution ū and for r, h > 0 small enough: since there exists a sequence (u ε ) ε of subsolutions such that lim sup * u ε = ū, there exists an usc subsolution u defined in B(x, r) × ( t − h, t) ⊂ B × (t 1 , t 2 ) and a maximum point (x , t ) of u − v which is also as close as we want to (x, t).
We can therefore assume that x ∈ B and since Q(k) holds true, necessarily x ∈ M k−1 for the same reason as for x above.
Consider now Lemma 5.9 for v at x .If we are in case A) of the alternative, we get a sequence x n → x which remains in Ω := B \M k−1 , and Ω does not intersect any M j for j ≤ k −1.Moreover, the reaching times of trajectories issued from the x n are controled from below.
Next, we use in conjunction Lemma 5.6 in Ω: the sub-optimality principle satisfied by u in Ω implies that for some σ ∈ (0, h) small enough (but uniform with respect to n) where η comes from the strict subsolution property for u .Passing to the limit as and this contradicts the fact that (x , t ) is a local maximum point of u − v.
In case B), since by Lemma 5.8 u is C 1 on M k , we have But this is also a contradiction since u is a strict subsolution and therefore Hence, such a maximum point (x , t ) cannot exist, which implies that if x exists, it has to be located on M j for some j < k − 1, thus and Q(k − 1) holds true. Q.E.D.

A Stability Result
In this section we prove a stability result when we have a sequence of problems on stratified domains (HJB − SD) ε .An important issue here is that, not only do the corresponding Hamiltonians depend on ε, but also the stratification of space does.More precisely, for each ε > 0 we are given a regular stratification M ε and a notion of convergence is required.This is the purpose of the following definition.Definition 6.1 We say that a sequence (M ε ) ε of regular stratification of R N .converges to a regular stratification M if, for each x ∈ R N , there exists r > 0, an (AFS) M = M (x, r) in R N and, for any ε > 0, changes of coordinates Ψ x ε , Ψ x as in Definition 3.4 such that Ψ We denote this convergence by M ε RS −→ M. Thus, the manifolds M k ε (k = 0..N ) can vary with ε but after suitable changes of variable Ψ x ε , they are flat and constant.The important issue is that we do not want to create/destroy/intersect manifolds when they move.
Then we also consider, for each ε > 0, the associated Hamilton-Jacobi-Bellman problem in the stratified domain M ε , that we denote by (HJB − SD) ε .The meaning of sub and supersolutions is the one that is introduced in Definition 4.2, with the family of Hamiltonians H ε and (H k ε ) that are constructed from M ε and some family BL ε .
In order to formulate the following stability result, we have to define limiting Hamiltonians for the H k ε (x, t, p) but the difficulty is that they are defined for x ∈ M k ε which depends on ε.In order to turn around this difficulty, we use the change of variables of Definition 6.1 which leads to consider the Hamiltonians Hk ε , defined for x ∈ M ∩ Ψ x (B(x, r)), a domain which does not depend on ε.We make a slight abuse of notations by saying that supersolution of (HJB-SD), the HJB problem associated with H = lim sup * H ε .
(ii) If, for ε > 0, u ε is an usc subsolution of (HJB − SD) ε and if the Hamiltonians (H k ε ) k=0..N satisfy (NC) and (TC) with uniform constants, then ū = lim sup * u ε is a subsolution of (HJB-SD) with H k = lim inf * H k ε for any k = 0..N .The map BL satisfies (H BL ) and provided each BL k and BL satisfy (NC-BL), we have an (HJB-SD) which has a unique solution.

Applications & Extensions
The Filippov approximation -a way to build a solution of u t + H(x, u, Du) = 0 in R N in presence of discontinuities consists in using the Filippov approximation for the corresponding control problem: for each ε > 0 we consider The construction of BL ε comes from several considerations (i) for each ε > 0, BL ε is a continuous set-valued map with convex, compact images; (ii) BL ε (x, t) also takes into account the specific dynamic-cost at (x, t); (iii) BL ε (x, t) takes into account dynamics-costs coming from a neighborhood of (x, t).
Notice first that by construction, BL ε is a continuous set-valued map which satisfies (H BL ) and (NC-BL),(TC-BL).Therefore there exists a unique solution U ε of (HJB − SD) ε , associated to BL ε .
Since BL ε is continuous, if M is a stratification adapted to BL, it can be seen as a stratification adapted also to BL ε , for any ε, even if there is no discontinuity for BL ε .Thus, BL ε RS −→ BL and the stability result (Corollary 6.4 yields that U ε converges to the unique solution of (HJB-SD)).This result extends [7,Thm. 6.1] where the convergence of Filippov's approximation was proved for an (N − 1)-dimensional discontinuity set.
Infinite horizon problems -we derived a complete study of parabolic (HJB-SD) which correspond to finite horizon control problems.In the same way, we can handle similarly the case of infinite horizon problems, leading to stationary (HJB-SD) as in [6].This amounts to considering a set-valued map x → BL(x) and introduce the Hamiltonians H k (x, u, p) = sup(λu − p • b − l), where the supremum is taken over BL(x), with b ∈ T x M k .The adaptations are quite straightforward: under (TC-BL),(NC-BL) (which have to be considered as independent of t) we get comparison for the complemented problem; and the value function of the associated control problem is the unique viscosity solution of this complemented problem.
Time-depending stratifications -throughout this paper, we assumed that the discontinuities of the set-valued map BL(•, •) is independent of the time-variable.This is a simplification which can be relaxed at (almost) no cost in some situations: following the ideas of the stability result, we assume that for each t > 0 we have a stratification M(t) adapted to BL(•, t) with the following property for each (x, t) ∈ R N ×[0, T ], there exists r > 0, an (AFS) M in R N and a local change of coordinates Ψ (x,t) : B(x, r) × (−r, r) → R N × R as in Definition 3.4 such that Ψ (x,t) M k ∩ B(x, r) × (−r, r) = (M × R) ∩ Ψ (x,t) B(x, r) × (−r, r) .
This means that, up to the local changes of variables Ψ (x,t) , we are in a flat and time-independent situation.All the constructions and results that we derived thus apply with slight modifications.Notice that of course the dependance of Ψ (x,t) with respect to the time variable should be regular enough so that the rectified equation keeps the suitable properties (TC),(NC),(LP), i.e.C 1 in the t-variable, and W 2,∞ or C 1 in the x-variable (depending on the controllability assumptions).
A word on the maximal solution -by focusing on the complemented (HJB-SD) problem, the unique solution we select is the minimal solution of u t + H(x, t, Du) = 0 in M N , complemented with the Ishii conditions on Γ = M N −1 ∪ • • • ∪ M 0 .This solution is denoted by U − in [6,7].
The maximal solution, U + , was identified in [6,7] but only in the specific case of a (N − 1)dimensional discontinuity set: Γ = M N −1 , i.e.M k = ∅ for any k = 0..(N − 2).The reason is that the identification of U + through a suitable control problem (involving only "regular controls") requires a reflection-type argument on Γ.Thus, the problem is linked to the very definition of this maximal solution and in the context of general HJB problems on stratified domains, the methods used in [6,7] do not seem to be adaptable (except in special cases).This is to our point of view a very interesting problem to identify this maximal solution in the general case (at least in a framework as general as possible).

Figure 1 :
Figure 1: Example of a 3-D stratification

Definition 3 . 7
Let M be a general regular stratification associated to BL and (H k ) k=0..N be the associated Hamiltonians.(i) In the case of an admissible flat stratification, we say that the associated Hamiltonians (H k ) k=0..N satisfy the Local Assumptions on the Hamiltonians in the Flat case (LAHF) if (TC), (NC) and (LP) are satisfied.

4
Control Problems on Stratified Domains (II): Subsolutions and Complemented Hamilton-Jacobi-Bellman Equations For the subsolution's property of U , the behaviour of the dynamic is going to play a key role and we have to strengthen Assumption (H BL ) by adding continuity and controllability assumptions, (TC-BL) & (NC-BL) which are equivalent to (TC) & (NC).The main consequences of (TC-BL) & (NC-BL) is the Theorem 4.1 (Subsolution's Property) Under Assumptions (H BL ), (TC-BL) and (NC-BL), the value-function U satisfies (i) For any k = 0..(N − 1), U * = (U | M k ) * on M k ;

Definition 4 . 2
Let M be a regular stratification of R N associated to a set-valued map BL. (i) A bounded usc function u : R N × [0, T ] → R is a viscosity subsolution of the Hamilton-Jacobi-Bellman in Stratified Domain [(HJB-SD) for short], if and only if it is a subsolution of

5. 1 Lemma 5 . 4
From local to global comparison Our first result consists in showing that we can reduce the global comparison result in R N × [0, T ] to "local" comparison results.Let us introduce the following version of the Local Comparison Principle in a cylinder Ω × [0, T ] ⊂ R N × [0, T ]LCP(Ω): for any (x, t) ∈ Ω × (0, T ], there exists r, h > 0 such that B r (x) ⊂ Ω, h ≤ t and one has a comparaison result in B(x, r) × (t − h, t) for any r ≤ r and h ≤ h.Assume (H BL ).We have a comparaison result in Q := Ω×(0, T ] if and only if LCP(Ω) holds true.