Stability of the replicator dynamics for games in metric spaces

In this paper we study the stability of the replicator dynamics for symmetric games when the strategy set is a separable metric space. In this case the replicator dynamics evolves in a space of measures. We study stability criteria with respect to different topologies and metrics on the space of probability measures. This allows us to establish relations among Nash equilibria (of a certain normal form game) and the stability of the replicator dynamics in different metrics. Some examples illustrate our results.


(Communicated by Elvio Accinelli)
Abstract. In this paper we study the stability of the replicator dynamics for symmetric games when the strategy set is a separable metric space. In this case the replicator dynamics evolves in a space of measures. We study stability criteria with respect to different topologies and metrics on the space of probability measures. This allows us to establish relations among Nash equilibria (of a certain normal form game) and the stability of the replicator dynamics in different metrics. Some examples illustrate our results.
1. Introduction. In this paper we are interested in symmetric evolutionary games whose strategies' interactions are modeled with a specific dynamical system known as the replicator dynamics. This type of games models interactions of strategies of a single population, and form part of the so-called population games.
In game theory it is important to consider models with strategies in measurable spaces because this allows us to present in a unified manner essentially all the standard models, namely, with finite strategy sets, compact or unbounded intervals, and so on. This the case for some models in oligopoly theory, international trade theory, war of attrition, and public goods, among others. This paper presents an evolutionary dynamics for games where the set of strategies is a metric space. We have consequently that the dynamical system lives in a Banach space, which in our case is a space of finite signed measures. In particular, if the strategy set is finite, then the dynamical system lives in R m , where m is the number of strategies.
Conditions for stability of the replicator dynamics have been developed by several authors. Bomze [5] provides a stability theorem for different topologies. Oechssler and Riedel [23] provide a stability theorem under the total variation norm for pure strategies that are critical points; in [24] the same authors study stability under the Prokhorov metric in a double symmetric game (named a potential game by evolutionary games. Some important technical issues are also summarized. Section 4 establishes the relation between the replicator dynamics and a normal form game, using the concepts of Nash equilibria and strongly uninvadable strategies. Section 5 presents two of our main stability results, Theorems 5.1 and 5.2. Section 6 establishes a relationship between Nash equilibria and the replicator-dynamics stability. Section 7 proposes examples to illustrate our results. We conclude in section 8 with some general comments on possible extensions. An appendix contains results of some technical facts.
The supremum in (1) We consider the weak topology on M(A) (induced by C(A)), i.e., the topology under which all elements of C(A), when regarded as linear functionals g, · on M(A) are continuous. In this topology, a neighborhood of a point µ ∈ M(A) is of the form for > 0 and H a finite subset of C(A).
for all g in C(A). If M(A) is instead the space P(A) of probability measures on A, then sometimes we say that µ n converges in distribution to µ.

Metrics on P(A).
There are many metrics that metrize the weak topology.
In particular, here we use a Wasserstein metric because of its interesting properties, some of which are described in this section. (For further details see, for instance, Villani [32], chapter 6). Let A be a Polish space, that is, a complete separable metric space with a metric ϑ. Let a 0 be a fixed point in A. For each p with 1 ≤ p < ∞, we define the space P p (A) as

SAUL MENDOZA-PALACIOS AND ONÉSIMO HERNÁNDEZ-LERMA
This definition is independent of a 0 (see Villani [32]). The L p -Wasserstein distance r wp between µ and ν in P p (A) is defined by where Π is the set of probability measures on A × A with marginals µ and ν.
In particular, when p = 1 we write the L 1 -Wasserstein distance r w1 as r w . Moreover, the L 1 -Wasserstein distance coincides with the Kantorovich-Rubinstein metric on P(A) (see Villani [32]). Any Wasserstein distance has important properties. For instance, they preserve the metric ϑ on P(A), i.e., for any a, b ∈ A and p ∈ [1, ∞) the distance between the Dirac measures δ a and δ b is r wp (δ a , δ b ) = ϑ(a, b). This is not true for the total variation norm (1), because, for instance, δ a − δ b = 2 for any a, b ∈ A with a = b. Another important property of the Wasserstein distance (6) is its interpretation (see Villani [32]): the distance ϑ(a, b) can be seen as the cost for transporting one unit of mass π(·, b) from a to one unit of mass π(a, ·) from b .

Remark 1.
There exist several metrics that metrize the weak topology of P(A). Among the most well-known are the Prokhorov metric, the bounded-Lipschitz metric, and the Kantorovich-Rubinstein metric. In the rest of this paper we will denote by r wτ any metric that metrizes the weak topology on P(A) (not to be confused with the notation r w of the L 1 -Wasserstein distance). Moreover, we denote by r any metric on P(A), that includes the total variation norm (1) or any distance that metrizes the weak topology τ wτ . An open ball in the metric space (P(A), r) is defined in the classical form V r α (µ) := ν ∈ P(A) : r(ν, µ) < α where α > 0.

Remark 2. a) Let
A be a separable metric space, and r wτ any distance that metrizes the weak topology τ wτ in P(A). Let µ be any measure in P(A), and consider the family V H (µ) of neighborhoods V H (µ) of the form (4). In addition, consider the family V rwτ (µ) of the open balls V rwτ α (µ) of the form (7). Both families V H (µ) and V rwτ (µ) are neighborhood basis for µ in the space (P(A), τ wτ ). For details see Pedersen [25], chapters I-II. b) A neighborhood V H (µ) of µ is contained in some open ball V rwτ α (µ) with center µ. The converse is also true, i.e., any open ball V rwτ α (µ) is contained in some neighborhood V H (µ) (see Munkres [21], Chapter 2, Lemma 13.3).

Differentiability.
Definition 2.2. Let A be a separable metric space. We say that a mapping µ : 3. The model. in a set A of characteristics (the set of pure strategies or pure actions), which is a separable metric space. Let P(A) be the set of probability measures on A, also known as the set of mixed strategies. Moreover, consider a payoff function J : P(A) × P(A) → R that explains the interrelation between the population, and which is defined as where U : A×A → R is a given measurable function. If δ {a} is a probability measure concentrated at a ∈ A, the vector (δ {a} , µ) is written as (a, µ), and (9) becomes In particular, (9) yields In an evolutionary game, the strategies' dynamics is determined by a differential equation of the form with some initial condition µ(0) = µ 0 . The notation µ (t) represents the strong derivative of µ(t) (see Definition 2.2), and F (·) is a given mapping F : where µ (t, E) and F (µ(t), E) are the measures µ (t) and F (µ(t)) valued at E ∈ B(A). We shall be working with a special class of so-called symmetric evolutionary games which can be described as a quadruple where i) I := {1, 2} is the set of players; ii) for each player i = 1, 2 we have a set P(A) of mixed actions and a payoff function J : P(A) × P(A) → R (as in (9)); and iii) the dynamics (11)-(12) is described by the following replicator function (14), 3.2. Technical issues on the replicator dynamics. For future reference, in the remainder of this section we summarize conditions for the existence of a unique solution to the differential equation (11)- (12) with F as in (14), and an important property of this solution. These results can be traced back to Bomze [5], Oechssler and Riedel [23]. See also Mendoza-Palacios and Hernández-Lerma [20].

4.
The replicator dynamics: NESs and SUSs. Is this section we consider symmetric evolutionary games as in (13) and compare them with normal-form games (19), below. We study the relation between a Nash equilibrium of a normal-form game and the replicator dynamics (Proposition 2) . We also define the important concept of strongly uninvadable strategy (Definition 4.2) and its relation to a Nash equilibrium (Proposition 4). Summarizing, in this section we show that where C is the set of critical points of the replicator dynamics, N is the family of Nash equilibrium strategies for (20), r − SU S is the subfamily of r-strongly uninvadable strategies for any metric r on P(A) (see Definition 4.2). This results will be complemented in Section 6 (see Corollary 1). A normal form game Γ (also known as a game in strategic form) can be described as a triplet where ii) for each player i ∈ I, we specify a set of actions (or strategies) P(A i ); and iii) a payoff function J i : , 2}, we can obtain from (19) a symmetric normal-form game with two players, and sets of actions and payoff functions as P(A) = P(A 1 ) = P(A 2 ) and J(µ 1 , µ 2 ) = J 1 (µ 1 , µ 2 ) = J 2 (µ 2 , µ 1 ) for all µ 1 , µ 2 ∈ P(A). Hence, we can describe a two-player symmetric normal form game as For symmetric normal form games Γ s we can express a symmetric Nash equilibrium (µ * , µ * ) in terms of the strategy µ * ∈ P(A), as follows.
The following definition is a slightly modified version of the strongly uninvadable strategies used in Bomze [5].
When r is the Prokhorov metric r p , Oechssler and Riedel [24] name a r p -SUS an evolutionary robust strategy. If r wτ is any metric that metrizes the weak topology (recall Remark 1), Cressman and Hofbauer [8] call a r wτ -SUS a locally superior strategy.
Proposition 3. Let r wτ be a distance that metrizes the weak convergence on P(A). If a measure µ * ∈ P(A) is r wτ -SUS, then it is · -SUS.
The next lemma is a key fact to provide a general framework for the different stability criteria. Lemma 4.3. Let r wτ be a distance that metrizes the weak convergence on P(A). For every µ, ν ∈ P(A) and > 0, there exist α and α in (0, 1) and η, γ ∈ P(A) such that Proof. Let α n be a sequence in (0, 1) such that α n → 0, and let η n := α n ν Hence, by Proposition 7 in the Appendix, part i) follows.
The following proposition shows that a strongly uninvadable strategy is also a Nash equilibrum strategy. Proposition 4. Let r be a metric on P(A). If µ * is a r-SUS, then µ * is a NES of Γ s .

5.
Stability. This section presents a review of results on the stability of the replicator dynamics. These results include different stability criteria with respect to various metrics and topologies in the space of probability measures. Assume that ν << µ. We define the cross entropy or Kullback-Leibler distance of ν with respect to µ as From Jensen's inequality it follows that 0 ≤ K(µ, ν) ≤ ∞ and K(µ, ν) = 0 if and only if µ = ν. The Kullback-Leibler distance is not a metric, since it is not symmetric, i.e., K(µ, ν) = K(ν, µ). Given µ * ∈ P(A), > 0, and a strictly increasing function ϕ : [0, ∞) → [0, ∞), we define the set The following theorem characterizes the stability of the replicator dynamics with respect to the L 1 -Wasserstein metric r w in (6). This distance metrizes the weak topology and has important relationships with other distances that also metrize the weak topology (see Proposition 7 in the Appendix). Furthermore, the L 1 -Wasserstein metric is closely related to the variation norm (1) and the Kullback-Leibler distance (24); see Proposition 8. The following two propositions give better approximations than those in Bomze [4], Theorem 2.
Theorem 5.2. Let A be a separable metric space and suppose that the conditions i) and ii) of Theorem 3.1 hold. Let δ a * be a Dirac measure and r any metric on P(A). If δ a * is r-SUS, µ(·) is a solution of the replicator dynamics, and µ 0 − δ a * < for some small > 0, then is in some open ball V rwτ α (µ * ) as in (7) for all t ≥ 0, where r wτ is a distance that metrizes the weak topology; iii) if A is a compact Polish space (with diameter C > 0), then for all t ≥ 0, r w (µ(t), δ a * ) < C ; iv) if A is compact (not necessary Polish) and the map µ → J(δ a * , µ) − J(µ, µ) is continuous in the weak topology, then r wτ (µ(t), µ * ) → 0, where r wτ is any distance that metrizes the weak topology.
Proof. Parts i), ii) and iv) follow from Proposition 3 and Theorem 6.2 in Mendoza-Palacios and Hernández-Lerma [20]. Part iii) follows from Proposition 8. Theorem 5.2 is also proved by Oechssler and Riedel [23] with slight changes in the definition of -SUS. 6. NESs and stability. In this section we are interested in the relation between the stability of the differential equation (11)- (12) with F as in (14), and the static evolutionary concepts NES and SUS. Let µ, ν ∈ P(A). By Remark 2 and Proposition 8 we know that if µ and ν are "close" with respect to the Kullback-Leibler distance K, then they are close in the total variation norm · , and consequently they are also "close" in the weak topology. This argument is not true in the opposite direction. Hence we say that the Kullback-Leibler distance is "stronger than" the total variation norm, and that the total variation norm is "stronger than" any distance that metrizes the weak topology.
Consider the Kullback-Leibler distance K, the total variation norm · , and any distance r wτ that metrizes the weak topology. The following diagram gives the natural implications between the different concepts of stability.
The following proposition states the existence of the support of a probability measure on a separable metric space. This concept is used in Proposition 6. Proof. See Royden [28], pag. 408.
The following proposition establishes a partial converse of Proposition 2.
Proposition 6. Let A be a separable metric space, and r 1 , r 2 the Kullback-Leibler distance or some metric in P(A) where r 1 is "equal to" or "stronger than" r 2 .
Suppose that the conditions i) and ii) of Theorem 3.1 are satisfied, and let µ * be a critical point of the replicator dynamics. If µ * is [r 1 , r 2 ]-stable, then µ * is a Nash equilibrium strategy (NES) of Γ s .
Let > 0 and µ 0 := λ δ a + (1 − λ )µ * be the condition initial, where λ ∈ (0, 1) and µ 0 ∈ W ϕ( ) (µ * ), with ϕ( ) = 2 . The number λ indeed exists since and the logarithmic function is continuous, and by Remark 2 and Proposition 8, µ 0 is near µ * in the r 1 -distance. By (32) and Theorem 3. iii) Let r 1 and r 2 be the Kullback-Leibler distance or some metric in P(A), where r 1 is "equal to" or "stronger than" r 2 , The following corollary summarizes our results and complements the diagram (18). Corollary 1. Let A be a compact Polish space, and assume the conditions i) and ii) of Theorem 3.1. Then we have the following relations: Proof. This is a consequence of Theorem 5.1 and Propositions 2, 3, 6.
Remark 3. Let r 1 be a metric on P(A), and let r 2 be the total variation norm on P(A) or some metric equivalent to the weak topology. By Theorem 5.2 and Propositions 2, 6, we have the following implications if a Dirac measure δ a * is a r 1 -SUS.
These facts complement Theorem 5.2.
7. Examples. This class of games could represent a Cournot duopoly or models of international trade with linear demand and linear cost (see Bagwell and Wolinsky [1]). It can also represent some models of public good games (see Mas-Colell, Whinston and Green [18]).
If 2c(a − b) > 0 and 4a 2 − b 2 > 0, then we have an interior Nash equilibrium strategy (NES) On the other hand, letȳ µ := A yµ(dy). If µ is such thatȳ µ < x * , then by Jensen's inequality This is also true ifȳ µ > x * . Hence, for any metric r on P(A), the strategy δ x * is r-SUS. Therefore, by Theorem 5.2, if µ 0 − δ x * = 2(1 − µ 0 ({x * })) < , then Moreover, since the payoff function U (·) is continuous and the set A of strategies is compact, we conclude that µ(t) → δ x * in distribution. 7.2. Graduated risk game. A graduated risk game is a symmetric game (proposed by Maynard Smith and Parker [19]), where two players compete for a resource of value v > 0. Each player selects her "level of aggression" for the game. This "level of aggression" is captured by a probability distribution on A := [0, 1]. In this case, x ∈ A can be interpreted as the probability that neither player is injured, and 1 2 (1 − x) is the probability that player one (or player two) is injured. If the player is injured its payoff is v − c (with c > 0), and hence the expected payoff for the player is where x and y are the "levels of aggression" selected by the player and her opponent, respectively. If v < c, this game has a NES with density function where α = c v . Bishop and Cannings [2] show that if v < c, then the NES satisfies that J(µ * , µ) − J(µ, µ) > 0 ∀µ ∈ P(A), that is, µ * is a r-SUS for any metric r in P(A), with A = [0, 1]. Hence, by Theorem 5.1, if K(µ 0 , µ * ) < ϕ ( ) = 2 2 , then i) µ(t) ∈ W ϕ ( ) (µ * ) for all t ≥ 0; ii) µ(t) − µ * < for all t ≥ 0; iii) r w (µ(t), µ * ) < for all t ≥ 0.
8. Comments and suggestions for further research. In this paper, we introduce a model of symmetric evolutionary games with strategies in metric spaces. The model can be reduced, of course, to the particular case of evolutionary games with finite strategy sets. We provide a general framework to the replicator dynamics that allows us to analyze different stability criteria. Our main results, in Sections 4,5,6 are of three types. The first one concerns the relations between three key concepts: the critical points of the replicator dynamics, the Nash equilibrium strategies and the strongly uninvadable strategies. See, for instance, Propositions 4.2 and 4.6. The second type of results is about stability in different topologies and metrics as, for instance, in Theorem 5.1, relations (30), Proposition 6, and Corollary 1. Finally, the third type of results is about the special case of Dirac measures, as in Theorem 5.2, and Remark 3. Finally, we presented two examples. The first one may be applicable to oligopoly models, theory of international trade, and public good models. The second example deals with a graduated risk game. There are many questions, however, that remain open. For instance, when the set of pure strategies is finite, Cressman [6] shows that under some conditions the stability of monotone selection dynamics is locally determined by the replicator dynamics. Is this true for games with strategies in the space P(A) of probability measures? Another important issue would be to obtain a stability theorem for several evolutionary dynamics of games with continuous strategies similar to the result by Hofbauer and Sigmund [15] (Theorem 14) for games with a finite strategy set A.