OPTIMAL SYNCHRONIZATION PROBLEM FOR A MULTI-AGENT SYSTEM

. In this paper we investigate a time-optimal control problem in the space of positive and ﬁnite Borel measures on R d , motivated by applications in multi-agent systems. We provide a deﬁnition of admissible trajectory in the space of Borel measures in a particular non-isolated context, inspired by the so called optimal logistic problem , where the aim is to assign an initial amount of resources to a mass of agents, depending only on their initial position, in such a way that they can reach the given target with this minimum amount of supplies. We provide some approximation results connecting the microscopical description with the macroscopical one in the mass-preserving setting, we con- struct an optimal trajectory in the non isolated case and ﬁnally we are able to provide a Dynamic Programming Principle.


1.
Introduction. To include uncertainty features in control problems, researchers used a set of different approaches, such as deterministic [16], random [23] and stochastic [4,25], and applied to different domains as finance [18] and quantum control [6].
In particular, in stochastic approaches the state is represented by a random variable or, alternatively, a probability distribution. The evolution is then given by an equation involving Brownian motion and solution is interpreted in the sense of the Itô or Stratonovich integral [17]. However, in many applications the Brownian motion is not necessarily the correct way of representing uncertainty evolution. For instance uncertainties may be naturally bounded. Even the generalization to other stochastic processes [18] share similar limitations.
An alternative approach is provided by the evolution of probability measures according to transport equations as in optimal transport theory [24].
A time-optimal control problem in the space of probability measures endowed with the topology induced by the Wasserstein metric has been introduced in [9,10,11,12], where the dynamics is given by a controlled continuity equation in the space of probability measures, which naturally arises as an infinite-dimensional counterpart of a finite-dimensional differential inclusion.
In particular, in [11] the initial state is described by a Borel probability measure µ 0 ∈ P(R d ) and its evolution in time for 0 < t ≤ T is given by a continuity equation, where v t (·) is a Borel selection of a given set-valued map F : The resulting admissible mass-preserving trajectories µ := {µ t } t∈[0,T ] , µ |t=0 = µ 0 , are time-depending Borel probability measures on R d . Under mild assumptions on the selection v t and the set-valued map F , the Superposition Principle (Theorem 8.2.1 in [2]) allows to represent µ as a superposition of integral solutions of the following characteristic system γ(t) ∈ F (γ(t)), for L 1 -a.e. t ∈ (0, T ), where the initial data are weighted according to the initial state. This gives a natural link between the underlying classical control problem in R d and the problem in P(R d ).
The main application we have in mind is the so-called multi-agent systems [21]. In the framework of crowd dynamics, one of the key challenge is that of the evacuation problem, where the objective is to drive a crowd of people outside of an environment space in the minimum amount of time. Then the evolving total mass of pedestrians may be not conserved in time, since pedestrians are removed from the system once they get outside the modeled area. Thus the evolving mass solves a continuity equation with sink. To treat cases of transport equation with source/sinks, and more precisely to compare measures with different total masses, the classical Wasserstein distance between probability measures cannot be used, thus in [19,20] a generalized Wasserstein distance between positive finite Borel measures is introduced. A measure theoretic approach for transportation problems can be found also in [22] where the modeling approach relies on the concept of discrete-time evolving measures and in [8] in which authors focus mainly on concentration and congestion effects. A very recent survey on this topic is the monograph [14], providing a new and unified multiscale description based on measure theory for the modeling of the crowd dynamics, which usually follows two main points of view, a microscopic and a macroscopic one, in order to analyze the relations between individual and collective behaviours, respectively.
In this paper we move from the framework presented in [11], but with a different formulation of the time-optimal problem and allowing the loss of mass during the evolution, which turns out to be closer to applications in pedestrian dynamics or general multi-agent systems. More precisely, given an initial state µ 0 ∈ P(R d ), we consider an admissible mass-preserving trajectory µ ⊆ P(R d ), starting from µ 0 , coupled with a density f t which decreases linearly in time working as a countdown.
The initial density f 0 = f |t=0 : R d → [0, +∞], called clock-function, represents the time needed by the particles in the support of µ 0 to reach the given target S ⊆ R d following the trajectory µ. In [11] the authors consider the final time for which all the mass is inside the target, while here we are interested in an averaged time. In particular, the aim is to minimize the average of f 0 w.r.t. the initial distribution of agents, µ 0 , among all the Borel functions keeping nonnegative the density associated to µ along all the evolution. Notice that we ask the target set S to be strongly invariant for F in order to remove the agents once they have achieved their own task. From a macroscopic point of view, this formulation gives us the possibility to study a new class of trajectories in the space of positive Borel measures that we call admissible clock-trajectory for the initial measure µ 0 .
Another possible interpretation of this problem can be given in terms of optimal logistic/equipment. Indeed, the clock-function f 0 can be seen as the initial amount of supplies (e.g. fuel, energy) that has to be assigned to each agent in the support of µ 0 , and which depends only on its initial position. In the particular case in which the initial provided quantity is the time, then we recover the previous description of the problem. Considering the case in which we have a time-linear consumption of the provided supplies, the aim is to find the minimum amount of goods that has to be assigned at the beginning to each agent together with the associated optimal mass-preserving trajectory µ which allows µ 0 to reach the target set S with this minimum amount of supplies f 0 . The minimum of the cost function we are interested in, i.e. R d f 0 (x) dµ 0 (x), has to been taken among all the admissible couples (f 0 , µ) keeping nonnegative the evolving density.
We precise that in this paper we are dealing with the case of non renewable resources and non-interacting particles (see Example 1 for a physic application to a fluid depuration problem). We leave these nontrivial considerations to further improvements of the present work which is also connected with possible applications to the so called irrigation (Gilbert-Steiner) problem [3,5].
We will show also that the best clock-function can be interpreted as the minimum amount of time/goods that has to be assigned at the beginning to each agent in order to reach the target and we will construct the associated macroscopic description µ of the trajectories of the agents. In this sense the optimal vector field for the problem in the space of measures can be seen as a measurable feedback strategy for the underlying finite-dimensional control problem.
In the paper [13], the same problem is addressed aiming to focus more on regularity results of the value function and to provide an Hamilton-Jacobi-Bellman equation solved by the value function in some suitable viscosity sense.
The paper is structured as follows: Section 2 recalls some preliminary results and notation; in Section 3 we define the clock-admissible trajectories involved in our study and prove some approximation results on the mass-preserving trajectories on which our objects are built; finally, in Section 4 we state the time-minimization problem, in Theorem 4.4 we prove the existence of an optimal clock-trajectory constructing it by approximation techniques, and we conclude by stating a dynamic programming principle. 2. Preliminaries and notation. We refer the reader to Chapter 5 in [2] for preliminaries on measure theory.
Let X be a separable metric space. With P(X) we denote the set of Borel probability measures on X endowed with narrow convergence, with M + (X) the set of positive and finite Radon measures on X and with M (X; R d ) the set of vectorvalued Radon measures on X. We recall that we can identify P(X) with a convex subset of the unitary ball of the dual space (C 0 b (X)) , and narrow convergence is induced by the weak * -topology on (C 0 b (X)) . We denote with |ν| the total variation of ν ∈ M (R d ; R d ) and we adopt the notation σ µ to say that σ is absolutely continuous w.r.t. µ, for a pair of measures σ, µ on R d .
We recall that, if X, Y are separable metric spaces, the push-forward of a measure µ ∈ P(X) through a Borel map r : X → Y is defined by r µ(B) := µ(r −1 (B)) ∈ P(Y ), for all Borel sets B ⊆ Y , or equivalently it is defined by and we denote with P p (R d ) the subset of P(R d ) made of measures with finite p-moment.
Definition 2.1 (Wasserstein distance). Given µ 1 , µ 2 ∈ P(R d ), p ≥ 1, we define the p-Wasserstein distance between µ 1 and µ 2 by setting where the set of admissible transport plans Π(µ 1 , µ 2 ) is defined by Proposition 1. P p (R d ) endowed with the p-Wasserstein metric W p (·, ·) is a complete separable metric space. Moreover, given a sequence {µ n } n∈N ⊆ P p (R d ) and µ ∈ P p (R d ), we have that the following are equivalent 2. µ n * µ and {µ n } n∈N has uniformly integrable p-moments.
We recall now some basic definitions about the classical control problem with dynamics represented as a differential inclusion in R d .
Definition 2.2 (Standing Assumptions). We will say that a set-valued function F : R d ⇒ R d satisfies the assumption (F j ), j = 0, 1 if the following hold true (F 0 ) F (x) = ∅ is compact and convex for every x ∈ R d , moreover F (·) is continuous with respect to the Hausdorff metric, i.e. given x ∈ X, for every ε > 0 there exists δ > 0 such that |y − x| ≤ δ implies F (y) ⊆ F (x) + B(0, ε) and if γ(·) is an absolutely continuous function satisfying 1 for a.e. 0 < t ≤ T . Given a target set S ⊆ R d closed and nonempty, and recalling the definition of classical minimum time function T : Given T ∈ [0, +∞[, we set , that we endow with the usual sup-norm, recalling that Γ T is a complete separable metric space for every 0 < T < +∞. We call evaluation operator the map e t : Given a set X, A ⊆ X, we call indicator function of A the function 3. Statement of the problem and preliminary results. We define now the objects considered in the present study, recalling the definition of admissible masspreserving trajectory stated in [9,10,11,12].
a.e. t ∈ [0, T ] and µ t -a.e. x ∈ R d . In this case, we will say also that the admissible mass-preserving trajectory µ is driven by ν.
2. Let T > 0, µ be an admissible mass-preserving trajectory defined on [0, T ] starting fromμ and driven by ν = {ν t } t∈[0,T ] . We will say that µ is represented where γ is an absolutely continuous solution of the underlying characteristic system Remark 1. Definition 3.1(1) is equivalent to say that µ is an admissible masspreserving trajectory starting fromμ if it solves an homogeneous (parametrized ) continuity equation, ∂ t µ t + div(v t µ t ) = 0, in the distributional sense with Borel velocity field v t ranging among L 1 µt -selections of the given multifunction F : where F is driving the underlying differential inclusion in finite dimension. The vector-valued measure ν t represents the averaged Eulerian velocity field v t of the particles/agents weighted w.r.t. the evolving mass. We point out that assuming no regularity on the vector field v t leads to the lack of uniqueness both of the finite-dimensional characteristic system and of the continuity equation. Anyway as precised in the following remark, under mild integrability assumptions, it is still possible to construct a probabilistic representation for µ, which is not unique for the same reason, but still it is concentrated on characteristics driven by the same vector field as µ. On the other side, under mild assumptions, we can build a measure η ∈ P(R d × Γ T ) concentrated on the Caratheodory solutions oḟ γ(t) ∈ F (γ(t)) for a.e. 0 < t < T and starting by points in the support ofμ. Then, from η we can recover the associated admissible trajectory µ which turns out to be driven by an appropriate averaged vector field as we will see for example in the proofs of Propositions 2, 3 and Theorem 4.4. The convexity of F (·) ensures that this average vector field is still an admissible velocity.
isfies the integrability assumption of the Superposition Principle (see for example Theorem 8.2.1 in [2]) then there exists η ∈ P(R d × Γ T ) representing µ.
We now define the concept of clock-trajectory pair, where the first, called f 0 is representing the time left to reach the target or, in an optimal equipment interpretation of the problem, it represents more typically a general amount of supplies (not necessarily the time) given at the beginning to each agent in the support of the initial stateμ in order to reach the target. The fact that the clock is ticking downward is recapitulated by condition 4 of the following definition, where we assume that the provided amount of supplies decreases linearly in time during the evolution. Notice that the average time to reach the target, that is the cost we are interested in, is given by R d f 0 (x) dµ 0 (x). Since we want to define a trajectory defined on a possibly unbounded time interval, and recalling that Γ T is a separable metric space only if T is finite, in order to use results related to separability (such as certain compactness results, or the probabilistic representation η) we will construct a trajectory defined on [0, +∞[ by mean of successive prolongations of mass-preserving trajectories defined on bounded time intervals, as done in item 3 of the following definition. , and represented by η n ; 3. given n 1 , n 2 ∈ N with T n1 ≤ T n2 , we have (Id R d × r n2,n1 ) η n2 = η n1 , where r n2,n1 : Γ Tn 2 → Γ Tn 1 is the linear and continuous operator defined by setting r n2,n1 γ(t) = γ(t) for all t ∈ [0, T n1 ]. Clearly, r n2,n1 γ ∈ Γ Tn 1 for all γ ∈ Γ Tn 2 .

for any
In this case we will say thatμ follows the family of mass-preserving trajectories {µ n } n∈N . Notice that, since we askμ 0 (R d ) < +∞, then we can identify f 0 with µ 0 µ ∈ L 1 µ0 .
Remark 3. Definition 3.2 forces an admissible clock-function f 0 (·) to be bounded below by the classical minimum time function T (·) of the underlying finite-dimensional differential inclusion with target S ⊆ R d , in particular f 0 (x) ≥ T (x) forμ-a.e.
x ∈ R d . This is necessary in order to keep nonnegative the measureμ t for all t ≥ 0. In the following example we provide a motivation for the study of our noninteracting particles problem, describing an application in a recent interest research topic among engineers and physicists communities. Example 1. Let us consider a purifier filter, with cylindrical shape for simplicity, with a polluted fluid flowing inside, without turbulence, in the direction of the axis of the cylinder. Usually such kind of filters are constructed by overlapping of many circular thin layers of different materials.
The initial measure µ 0 represents the initial concentration of polluting substances of the fluid on the first section of the filter. The multifunction F : R 3 ⇒ R 3 evaluated at a point x ∈ R 3 of the filter is defined to be a ball with radius smoothly dependent on x and it describes the filter's action on any polluting particle crossing position x.
This implies that the filter has a possibly non-homogeneous absorption power. We stress that this action affects only contaminating particles of the fluid, which are assumed to be unpolarized, small enough and with low concentration, thuswith a good approximation -there are no interactions between them.
We associate also to the initial condition a function f 0 : R 3 → [0, +∞[, such that f 0 (x) expresses the polluting power of the particles occupying the point x at the beginning of the evolution. We assume the decreasing of the pollution power to be linear w.r.t. time, thus the regions where the radius of the ball is smaller, i.e., the regions where the polluting particles move slower, are the regions where the absorption power is highest.
The aim is then to find the maximum initial level of polluting power (which is the optimal clock-function f 0 ) of the contaminating substances in the fluid to grant that all the contaminating agents will be completely neutralized by the filter when the fluid exits the cylinder. Notice that the worst case occurs when all the polluting particles follows a time-optimal trajectory for the underlying finite-dimensional problem ruled by F (·), where the target set is the last section of the filter.
3.1. Some results in a mass-preserving setting. In order to study the problem with mass loss, we will first discuss some approximation results for the masspreserving trajectories defined in Definition 3.1 on which our objects are built. For later use we state the following technical lemma, whose proof can be found in [9]. Lemma 3.3 (Basic estimates). Assume (F 0 ) and (F 1 ), and let C be the constant as in (F 1 ). Let T > 0, p ≥ 1, µ 0 ∈ P p (R d ) and µ = {µ t } t∈[0,T ] be an admissible masspreserving trajectory driven by ν = {ν t } t∈[0,T ] and represented by η ∈ P(R d × Γ T ). Then we have: (iv) there exist D , D > 0 depending only on C, T, p such that for all t ∈ [0, T ] we have We get immediately the following result.
hence the assumptions of the Superposition Principle (see for example Theorem 8.2.1 in [2]) are satisfied.
Given N ∈ N, N > 0, consider a set of N agents moving along admissible trajectories γ i (·), i = 1, . . . , N of the differential inclusionẋ(t) ∈ F (x(t)). We want to associate to the evolution of such a system an admissible mass-preserving trajectory keeping in mind that we are dealing with non-interacting particles/agents. Proposition 2 (Finite embedding of classical admissible trajectories). Assume hypothesis (F 0 ). Let N ∈ N \ {0}, and consider a set of N admissible trajectories {γ i (·), i = 1, . . . , N } ⊆ Γ T of the differential inclusionẋ(t) ∈ F (x(t)). For any t ∈ [0, T ], we define the empirical measures .
is an admissible mass-preserving trajectory driven by Proof. For any ϕ ∈ C ∞ C (R d ) and for a.e. t ∈ [0, T ] we have since the set in whichγ i (t) exists for all i = 1, . . . , N has full measure in [0, T ]. Defining Fix now x ∈ R d and ε > 0. Recalling that the set-valued map F is convex valued and upper semicontinuous, there exists δ > 0 such that F (y) ⊆ F (x) + εB(0, 1) for all y ∈ B(x, δ). In particular, if We have that I N B(x,δ),t = ∅ for all δ > 0 if and only if x ∈ {γ i (t) : i = 1, . . . , N }, i.e., if and only if x ∈ supp µ N t . So for any x ∈ supp µ N t , by taking the limit for δ → 0 + and then letting ε → 0 + , we have We thus obtain that µ N = {µ N t } t∈[0,T ] is an admissible mass-preserving trajectory driven by ν N = {ν N t } t∈[0,T ] and represented by η N for every N ∈ N.

GIULIA CAVAGNARI, ANTONIO MARIGONDA AND BENEDETTO PICCOLI
We consider now the limit of the above construction as N → +∞ in the case p > 1.
Proposition 3 (Mean Field Limit). Assume hypothesis (F 0 ) and (F 1 ). Let {γ i } i∈N ⊆ Γ T be a sequence of admissible trajectories of the differential inclusionẋ(t) ∈ F (x(t)), p > 1. For any N ∈ N \ {0}, we define Assume that there exists C 1 > 0 such that µ ∞ is an admissible trajectory driven by ν ∞ and represented by η ∞ ; e. for any closed set K ⊆ Γ T such that {γ i } i∈N ⊆ K, we have that We will say also that µ ∞ is a mean field limit associated to {γ i } i∈N ⊆ Γ T .

OPTIMAL SYNCHRONIZATION PROBLEM 287
We have that ψ 2 (·) has compact sublevels if p > 1. We recall that ifγ(t) ∈ F • γ(t) for a.e. t, then by (F 1 ) we have and so by Gronwall's inequality Indeed, for all N ∈ N we have which proves Claim 1.
is tight, thus up to a non relabeled subsequence, it weakly * converges to a measure ν ∞ t ∈ M (R d ; R d ). Since the map e t : R d × Γ T → R d is continuous, we have that for all t ∈ [0, T ]. All the other assertions follow from the fact that the moments of µ N t are uniformly bounded for N ∈ N and t ∈ [0, T ], also the tightness of {ν N t } N ∈N follows from 2. Claim 3. µ ∞ is an admissible trajectory driven by ν ∞ = {ν ∞ t } t∈[0,T ] . Notice that the functionals +∞, otherwise,  [7] or Theorem 2.34 in [1]). Then we have that the equation ∂ t µ ∞ t + div ν ∞ t = 0 holds in the sense of distributions, and for a.e. t ∈ [0, T ] we have |ν ∞ t | µ ∞ t , ν ∞ supp η N for all N ∈ N and γ k − γ ∞ → 0. By definition of η N we haveγ k = γ j k for an index 0 < j k ≤ N , and so {γ k } k∈N ⊆ K, thus γ ∈ K.
Remark 5. We notice that the tightness of {µ N t } N ∈N holds also in the case p = 1 by (2).
The following result provides us with the possibility to construct an admissible mass-preserving trajectory µ := {µ t } t∈[0,T ] , i.e., a curve in P(R d ) that satisfies a continuity equation with velocity field that is an L p µt -selection of the multifunction F , by constructing it on admissible trajectories of the finite-dimensional system of characteristics in a consistent way.
Proof. For the first assertion is enough to notice that the set K := {γ ∈ Γ T : γ(f (γ(0))) ∈ K} is closed in Γ T and then apply Proposition 3. For the second assertion, we have that there exists a sequence of compact sets clearly µ j * µ and m p (µ j ) → m p (µ) as j → +∞ by Dominated Convergence Theorem, thus W p (µ j , µ) → 0. There exists {x i,j } i,j∈N such that x i,j ∈ C j for all i, j ∈ N and Since supp µ k,j 0 ⊆ C j and supp µ j ⊆ C j , we have also m p (µ k,j 0 ) → m p (µ j ) as k → +∞. For any j ∈ N, let k j ∈ N be such that In particular, we have W p µ kj ,j 0 , µ ≤ 1 j + W p (µ, µ j ) → 0 + as j → +∞, and so Consider the countable set of points {x i,j : i = 1, . . . , k j , j = 1, . . . , ∞}. We can order it by stating that (i, j) < (i , j ) if either j < j or j = j and i < i , thus we obtain the sequence of points {x h } h∈N . By assumption, for each h ∈ N there exists γ h ∈ Γ T admissible trajectory of the differential inclusion satisfying γ h (0) = x h and γ h • f (x h ) ∈ K. We then apply item 1 to this sequence to conclude the proof.

Remark 6.
The assumption f ∈ C 0 (R d ) of the previous corollary can be weakened by assuming that f (·) is continuous at x for µ 0 -a.e. x ∈ R d or, equivalently, that the set of discontinuities of f (·) are contained in a µ 0 -negligible closed set.

4.
A time-optimal control problem with mass loss. We state now a timeoptimal control problem in the space of positive finite Borel measures for a nonisolated case using the definition of clock-trajectory given in Definition 3.2.
From now on, we will consider only closed, nonempty and strongly invariant target sets for our dynamics. where, by convention, inf ∅ = +∞.
Given p ≥ 1, we define also a clock-generalized minimum time function τ p : P p (R d ) → [0, +∞] by replacing in the above definitions P(R d ) by P p (R d ) and M + (R d ) by M + p (R d ). Since M + p (R d ) ⊆ M + (R d ), it is clear that τ p (µ 0 ) ≥ τ (µ 0 ). The main task of this section is to prove a Dynamic Programming Principle for our minimization problem. In Theorem 4.4, we will see how to construct an optimal-clock trajectory by approximation techniques, in particular by using Lusin's theorem and Corollary 2. This result will allow us to express τ (µ) as an average of the classical minimum-time function T (·).
The following extension lemma will be used.