An accretive operator approach to ergodic zero-sum stochastic games

We study some ergodicity property of zero-sum stochastic games with a finite state space and possibly unbounded payoffs. We formulate this property in operator-theoretical terms, involving the solvability of an optimality equation for the Shapley operators (i.e., the dynamic programming operators) of a family of perturbed games. The solvability of this equation entails the existence of the uniform value, and its solutions yield uniform optimal stationary strategies. We first provide an analytical characterization of this ergodicity property, and address the generic uniqueness, up to an additive constant, of the solutions of the optimality equation. Our analysis relies on the theory of accretive mappings, which we apply to maps of the form $Id - T$ where $T$ is nonexpansive. Then, we use the results of a companion work to characterize the ergodicity of stochastic games by a geometrical condition imposed on the transition probabilities. This condition generalizes classical notion of ergodicity for finite Markov chains and Markov decision processes.

1. Introduction 1.1. Motivations. In zero-sum stochastic games, introduced by Shapley [Sha53], two agents are facing repeatedly a zero-sum game that depends on a state variable, the evolution of which is governed by a stochastic process jointly controlled by the players. The finite-horizon value v k is an equilibrium payoff achieved when the players are optimizing their average payoff over the k first stages. A significant part of the literature focuses on the study of asymptotic properties of v k as the horizon k goes to infinity. Two main approaches are particularly followed. In the asymptotic approach, one studies the convergence of v k toward some limit called the asymptotic value. In the uniform approach, the problem is the existence of uniform optimal strategies, i.e., strategies that are near optimal in any k-stage game with k large enough, giving rise to the uniform value. We refer the reader to [NS03,MSZ15] for background on stochastic games.
When the state space is finite (which we assume throughout the paper), the existence of the asymptotic value was proved for stochastic games with finite action spaces: first for particular classes of games (recursive games by Everett [Eve57], absorbing games by Kohlberg [Koh74]) and then in general by Bewley and Kohlberg [BK76]. The result still holds for recursive and absorbing games with compact action spaces and continuous payoff and transition functions (see [Sor03] and [RS01], respectively). On the other hand, Mertens and Neymann [MN81] showed the existence of the uniform value for stochastic games with finite action spaces. Bolte, Gaubert and Vigeral [BGV15] extended these results to "definable" stochastic games, i.e., games whose data are definable in some o-minimal structure (e.g., when the action spaces and the payoff and transition functions are semialgebraic). The uniform value also exists for absorbing games with compact action spaces (see [MNR09]). We further mention that the existence of the uniform value is also guaranteed for Markov decision processes (equivalent to one-player stochastic games) with bounded payoffs (see Renault [Ren11]). However, the asymptotic value (hence the uniform value) does not exist for general zero-sum stochastic games, even with standard assumptions on the data, i.e., compact action spaces, continuous payoff and transition functions (see Vigeral [Vig13], see also Ziliotto [Zil16b] for a counterexample in the framework of zero-sum repeated games with signals).
Zero-sum stochastic games have a recursive structure which expresses itself in their dynamic programming operator, a.k.a. Shapley operator. From their analysis, one may infer asymptotic properties of the game (see e.g., Rosenberg and Sorin [RS01], Neymann [Ney03], Sorin [Sor04], Ziliotto [Zil16a]). In this paper, following this so-called "operator approach", we focus on the optimality equation (known as average case optimality equation, Shapley equation or ergodicity equation) T (u) = λe + u, where T : R n → R n is the Shapley operator of a game with n states, and e denotes the unit vector of R n . Indeed, if the latter equation has a solution (λ, u) ∈ R × R n , then the game has a uniform value, which is equal to λ for every initial state. Furthermore, the vector u yields uniform optimal strategies.
Conditions which guarantee the solvability of the ergodic equation are usually referred to as ergodicity conditions: they are recurrence conditions which ensure some stability property of the state process. For finite Markov chains (equivalent to zero-player games), the optimality equation can be seen as an instance of the Poisson equation. It is well known that it has a solution if the transition matrix has a unique invariant probability measure, or equivalently if the Markov chain has a unique ergodic class. For Markov decision processes (equivalent to one-player games), Bather [Bat73] showed that the ergodic equation has a solution when the system is communicating, meaning that for each pair of states (i, j), there is a stationary strategy such that the probability to reach j from i in finite time is positive. However, these accessibility relations do not easily carry over to the case of two players. As a consequence, one usually imposes on the games a strong communication structure, such as being irreducible (resp., unichain), that is, one requires the Markov chains induced by all pairs of stationary strategies to be irreducible (resp., unichain, i.e., to have a unique ergodic class) (see e.g., [HK66,Vri03]).
In [AGH15a], Akian, Gaubert and Hochart obtained milder recurrence conditions as a by-product of their analysis of the solvability of the ergodic equation. However, the results only apply to stochastic games with bounded payoffs. In applications, this boundedness property is restrictive and it is thus desirable to extend the results to a broader framework. This constitutes the purpose of the present work.
We further mention that the ergodic equation is a classical tool in the study of stochastic games (as well as Markov decision processes) with a more general state space. See e.g., [BG93,Sen94,AHS97] for countable state spaces or [GB98,HLL00,JN01,Küe01] for Borel spaces. Due to the technical difficulties inherent of these settings, the different kinds of ergodicity assumptions made on these games are much more involved than those previously discussed in the finite state space setting. However, in all the papers mentioned above, these conditions all boil down to the irreducible or the unichain case when the state space is finite.
1.2. Description of the main results. In this paper, we consider zero-sum stochastic games with a finite state space and possibly unbounded payoffs that satisfy some ergodicity property. Following Akian, Gaubert and Hochart [AGH15a], we formulate the latter condition in operator-theoretical terms, namely, we say that a stochastic game is ergodic if the optimality equation (a.k.a. ergodic equation) has a solution for all perturbations of the payoff function that only depend on the state. Let us note that, for finite Markov chains, this definition is one of several equivalent manifestations of ergodicity, as discussed in [AGH15a]. Our main result (Theorem 2.6) is an analytical characterization of this stability property in terms of boundedness in Hilbert's seminorm of some sets, so called slice spaces, which are invariant by the Shapley operator T of the game. We mention that this result has also applications in nonlinear Perron-Frobenius theory (see [Nus88,GG04], or [LN12] for background).
In a companion work (some of whose results were announced in [AGH15b], see also [Hoc16b, Ch. 6]), combinatorial criteria (of a graph-theoretical nature) for the boundedness of all slice spaces of T are given. Using this result, we then provide a geometrical ergodicity condition which is imposed only to the transition probabilities of the game. This condition involves two disjoint subsets of states, called dominions, each controlled by a different player, in the sense that each player can make one of these subsets invariant for the state process. This communication structure readily generalizes classical ergodicity characterization of finite Markov chains (see e.g., [KS76]) or Markov decision processes (see e.g., [Bat73]). Any Shapley operator T is monotone (that is, order-preserving) and additively homogeneous (that is, commutes with the addition by a multiple of the unit vector). These properties imply in particular that T is nonexpansive with respect to the supremum norm, which makes Id − T an m-accretive mapping. Our results are derived from the study of the latter map. Other results have been obtained using such an "acccretive operator approach" (see Vigeral [Vig10] or Vigeral and Sorin [SV16]), but contrary to them, our analysis exploits the nonexpansiveness of T with respect to Hilbert's seminorm. Furthermore, in addition to the solvability of the ergodic equation, this approach allows us to extend our analysis to the problem of uniqueness (up to an additive constant) of the solutions.
The paper is organized as follows. In Section 2, we recall the definition of zerosum stochastic games, we present the operator approach, and state our main results. In Section 3, after some preliminaries on accretive mappings, we prove a surjectivity condition for these mappings, which generalizes our ergodicity condition. In Section 4, we infer from the previous section a result on the stability (under additive perturbations) of existence of fixed points for nonexpansive maps. We also address the "generic" uniqueness of the fixed point with respect to the space of additive perturbations. Finally, in Section 5, we apply these results to stochastic games and prove the two main results stated in Section 2.
We mention that part of these results were announced in the conference proceedings [Hoc16a].
2. Ergodicity of stochastic games: preliminaries and main results 2.1. Zero-sum stochastic games. In this paper, we consider zero-sum stochastic games with a finite state space and possibly unbounded rewards. They involve two players which we call Max and Min. For the games to be well defined, we shall assume that their sets of actions are Borel spaces, i.e., Borel subsets of a Polish space, and given any such Borel space X, we will denote by ∆(X) the set of Borel probability measures on X, equipped with the weak* topology.
A (zero-sum) stochastic game is a 7-tuple Γ = ([n], A, B, K A , K B , r, p) defined by: • a finite state space [n] := {1, . . . , n}; • Borel action spaces A and B for players Max and Min, respectively; denotes the set of actions that are available to player Max (resp., Min) in state i. We • a Borel measurable payoff function r : K → R; • a Borel measurable transition function p : K → ∆([n]). A stochastic game Γ is played in stages, starting from a given initial state i 0 ∈ [n] known by the players. It proceeds as follows: at each stage ℓ 0, if the current state is i ℓ , the players choose simultaneously and independently some actions a ℓ ∈ A i ℓ and b ℓ ∈ B i ℓ , respectively. This incurs a payoff r(i ℓ , a ℓ , b ℓ ) given by player Min to player Max, and the state i ℓ+1 at the next stage is drawn according to the probability distribution p(· | i ℓ , a ℓ , b ℓ ). Then (i ℓ+1 , a ℓ , b ℓ ) is announced to both players.
Denote by H k = K k−1 × [n] the set of histories of length k 1. A (behavioral) strategy of player Max is a Borel measurable map σ : ∪ k 1 H k → ∪ i∈[n] ∆(A i ) such that, for every k 1 and every history h k = (i 1 , a 1 , b 1 , . . . , i k ) of length k, we have σ(h k ) ∈ ∆(A i k ). In particular, the strategy σ is stationary if σ(h k ) only depends on the final state i k , let alone the history length k. Likewise, a (behavioral) strategy of player Min is defined by a Borel measurable map τ : ∪ k 1 H k → ∪ i∈[n] ∆(B i ). We denote by S (resp., T ) the set of strategies of player Max (resp., Min). Also, for any state i, an element of ∆(A i ) (resp., ∆(B i )) is called a mixed action of player Max (resp., Min), and when this action is a Dirac measure, we call it pure and identify it with its supporting point in A i (resp., B i ).
An initial state i ∈ [n] and a pair of strategies (σ, τ ) of the players induce a probability measure on the set of infinite histories, H ∞ = K N , the expectation of which is denoted by E i,σ,τ . Then, for every length k 1, the k-stage payoff is Throughout the paper, we implicitly assume that all the finite-stage values of the games that we consider exist. We shall sometimes (but not always) make the following standing assumption, which ensures, besides the existence of the finite-stage values, that the players have optimal strategies, that is, the infima and suprema in Equation (1) can be replaced by minima and maxima, respectively (see [MSZ15, Thm. I.2.4]).
Assumption A. For every state i ∈ [n], (i) the action sets A i and B i are nonempty and compact; (ii) the payoff function r(i, ·, ·) is bounded from below or from above; (iii) for every pair of actions (a, b) ∈ A i ×B i , the functions r(i, ·, b) and p(j | i, ·, b), with j ∈ [n], are upper semicontinuous (u.s.c. for short), and the functions r(i, a, ·) and p(j | i, a, ·), j ∈ [n], are lower semicontinuous (l.s.c. for short).
We emphasize that some of our results (in particular our main result, Theorem 2.6) do apply to more general settings, for which the compactness of the action sets and/or the continuity of the payoff and transition functions may not hold. As an example, let us mention the following subclasses of stochastic games: • Markov decision processes (equivalent to one-player games); • games with perfect information, for which each state i is controlled by only one player, i.e., the functions r(i, ·, ·) and p(i, ·, ·) only depend on the actions of one player. In any case, it worth noting that the payoff function may not be bounded, even if Assumption A holds (see Examples 2.8 and 2.14).
A standard problem is to determine if the value vector v k has a limit in R n when the horizon k goes to infinity. When this limit exists, it is called the asymptotic value of the game. Bewley and Kohlberg [BK76] proved that when the action spaces are finite (along with the state space), the asymptotic value exists. But this result cannot be extended to stochastic games with compact action spaces and continuous payoff and transition functions (see [Vig13]).
Loosely speaking, the asymptotic value v = lim k→∞ v k exists when the players are able to guarantee the same payoff, v i , up to an arbitrarily small quantity, in any k-stage game starting in i with k large enough and known in advance. A stronger notion of value, the uniform value, asks for both players to guarantee, up to any small perturbation, the same payoff in any finite-stage game with a sufficiently large but unknown horizon. Precisely, the stochastic game Γ has a uniform value v ∞ ∈ R n if, for all ε > 0, there exist strategies σ * ∈ S, τ * ∈ T , and a horizon k 0 ∈ N such that, for all initial states i ∈ [n], for all strategies σ ∈ S and τ ∈ T , and for all k k 0 , The strategies σ * and τ * are called uniform ε-optimal strategies, or simply uniform optimal if they are uniform ε-optimal for every ε > 0. Mertens and Neyman [MN81] proved that the uniform value exists for stochastic games with finite state and action spaces. This result was extended in [BGV15] to games with a finite state space for which the data are definable in some o-minimal structure. Note that when the uniform value exists, the asymptotic value also exists and they are equal.
2.2. Operator approach. Using a dynamic programming principle, Shapley proved in [Sha53] that the value of a finite-stage stochastic game satisfies a recursive formula. This formula is written by means of the so-called Shapley operator. Given a stochastic game Γ, the latter is a map T : R n → R n whose ith coordinate is defined by where, for any Borel measurable map f : K → R, we introduce its bilinear extension The quantity T i (x) represents the value of a one-stage game with initial state i ∈ [n] and an additional payoff x j if the final state is j ∈ [n]. As mentioned in the latter subsection, we shall always assume -implicitly -that the Shapley operator is well defined. This is particularly the case when Assumption A holds and, if so, the sup and inf operators in (2) can be replaced by max and min, respectively.
Then, the dynamic programming principle introduced by Shapley writes: As an immediate consequence, if the asymptotic value exists, then it is given by The "operator approach" exploits the recursive structure (3) in order to infer convergence properties of v k from the analysis of the Shapley operator T (see e.g., [RS01,Ney03,Sor04,BGV15,Zil16a]). The latter analysis makes extensive use of the following properties, satisfied by any Shapley operator: monotonicity: x y =⇒ T (x) T (y), x, y ∈ R n , additive homogeneity: where R n is endowed with its usual partial order, e denotes the unit vector of R n and · ∞ is the sup-norm of R n . Note that the last property can be deduced from the first two (see [CT80]).
A basic tool to study the asymptotic properties of the k-stage value is the ergodic equation, also known as the average cost optimality equation or sometimes Shapley equation: where e is the unit vector of R n . Intuitively, this equation means that if the players choose optimal strategies in the game with an additional terminal payoff of u i when the last visited state is i, then the payoff at each stage is equal to λ, whatever the horizon and the initial state. From this game-theoretical interpretation or, in the spirit of the operator approach, using (4), the additive homogeneity and the nonexpansiveness of T , one easily deduce that if the ergodic equation has a solution, meaning that there exists a pair (λ, u) ∈ R × R n satisfying Equation (5), then the asymptotic value of the game exists and is a constant vector, whose entries are all equal to λ -we say that a vector is "constant" if it is proportional to the unit vector. Note that there is a unique scalar λ for which (5) holds, and for every α ∈ R, the pair (λ, u + αe) is also a solution to (5): we say that (λ, u) is defined up to an additive constant. When the ergodic equation is solvable the existence of the asymptotic value may in fact be refined by the following known result. We state the proof for the reader's convenience.
Proposition 2.2 (Existence of uniform value). Let Γ be a stochastic game such that the ergodic equation (5) is solvable for its Shapley operator T . Let (λ, u) be any solution. Then, the uniform value exists and is equal to the constant vector λe. Moreover, both players have stationary uniform ε-optimal strategies, which can be obtained from the vector u.
Proof. Let ε be a fixed positive real. For every state i ∈ [n], let µ * i ∈ ∆(A i ) be an action of player Max such that inf ν∈∆(Bi) Then, define the stationary strategy σ * of player Max such that the mixed action µ * i is selected whenever the current state is i. We now introduce the Shapley operator T σ * of a one-player game based on Γ where the strategy of player Max is fixed to σ * . Precisely, the ith coordinate of T σ * is given by Denoting by u ∞ = max i∈[n] |u i | the sup-norm of u, we have, for all positive integers k and all strategies τ of player Min, where the first inequality comes from the recursive property (3) applied to the one-player game with Shapley operator T σ * , and the second inequality stems from the monotonicity and the additive homogeneity of T σ * , along with the fact that 0 u − u ∞ e.
Let k 0 be such that 2 u ∞ k 0 ε. By construction of σ * , we deduce that for all integers k k 0 , for all state i, and for all strategy τ of player Min, we have With dual arguments, we show that there exists a stationary strategy τ * of player Min such that for all k k 0 , for all i ∈ [n], and for all σ ∈ S. This proves that λ is the uniform value of Γ for every initial state, and that (σ * , τ * ) is a pair of stationary uniform 2ε-optimal strategies.
We conclude this subsection with a sufficient condition for the solvability of the ergodic equation (5), hence for the existence of the uniform value. To that purpose, let us introduce the following two definitions. First, Hilbert's seminorm of a vector x ∈ R n is defined by x i .
Like any seminorm, it is a nonnegative function which is subadditive and absolutely homogeneous (i.e., α x H = |α| x H for all α ∈ R and all x ∈ R n ). However it fails to be a norm since x H = 0 if and only if x is proportional to the unit vector of R n . It is a standard result that any monotone and additively homogeneous self-map of R n is nonexpansive with respect to this seminorm (see e.g., [GG04]). Second, given a map T : R n → R n and real numbers α, β, we define the slice space Observe that if T is monotone and additively homogeneous (in particular if T is the Shapley operator of a stochastic game), then any slice space is invariant by T . Then we have the following. Definition 2.4 (Ergodicity of stochastic games). A zero-sum stochastic game Γ with Shapley operator T is ergodic if for all perturbation vectors g ∈ R n , the ergodic equation (5) is solvable for g+T , i.e., for the Shapley operator of the perturbed game where, for every (i, a, b) ∈ K, the payoff is g i + r(i, a, b).
Example 2.5. Let us illustrate our notion of ergodicity with the following very basic Shapley operators defined on R 2 by where ∨ stands for max and ∧ for min.
For g + T , the ergodic equation is solvable if and only if g is a constant vector (and then any vector u is a solution). Hence T is not ergodic (in the sense that the game whose Shapley operator is T is not ergodic).
In [AGH15a], several equivalent characterizations of ergodicity are given: in terms of the recession operator T (x) := lim ρ→+∞ ρ −1 T (ρx) and the asymptotic value of the perturbed games (Theorem 3.1); in graph-theoretical terms (Theorem 5.3); in game-theoretical terms (Proposition 5.1). Note that, as discussed in the latter reference, all this equivalent criteria extend the classical notion of ergodicity for finite Markov chains. However, these results only apply to stochastic games with bounded payoffs. They are all derived from a result that establishes a relation between ergodicity and the set of fixed points of T -a game (with bounded payoffs) being ergodic if and only if this fixed-point set is reduced to a line. In general, this equivalence fails, as illustrated in Example 2.8, and so, the results of [AGH15a] cannot be readily extended, to games with unbounded payoffs for instance. Thus, the wish to find a statement for ergodicity which applies to any kind of stochastic games has motivated the formulation of Definition 2.4. Our main result, which we state hereafter, can be seen as a generalization of [AGH15a, Thm. 3.1].
Theorem 2.6 (Ergodicity and slice spaces). A stochastic game is ergodic if and only if all the slice spaces of its Shapley operator are bounded in Hilbert's seminorm.
We mention that the novelty of this result lies in the necessity of the second statement. Indeed, it is easy to deduce the sufficient part from Theorem 2.3, as the subsequent proof shows.
Proof -"if " part. Let Γ be a stochastic game for which all the slice spaces of its Shapley operator T are bounded in Hilbert's seminorm. Let g ∈ R n be a statedependent perturbation of the payoff function of Γ. We first notice that 0 is in the slice space S β α (g + T ) for β = −α = g + T (0) ∞ . Hence the latter is nonempty. Then, denoting by m = g ∞ , we have for every x ∈ S β α (g + T ), This proves that S β α (g + T ) is included in S β+m α−m (T ), hence bounded in Hilbert's seminorm. It follows from Theorem 2.3 that the ergodic equation (5) is solvable for g + T , and consequently that Γ is ergodic.
Let us now illustrate the result, first with the basic Shapley operators of Example 2.5, and then with a more involved stochastic game for which the results in [AGH15a] do not apply.
Example 2.7. Consider the operators introduced in Example 2.5.
The slice spaces of T are trivial, that is, S β α (T ) is either empty if α > β, or equal to the full space if α β. Hence, since T has slice spaces unbounded in Hilbert's seminorm, it is not ergodic.
For T , the slice space S β α (T ) is nonempty if and only if α 0 β. In that case, it is equal to the set of points x such that |x 1 − x 2 | −α ∧ β. Hence the slice space is bounded in Hilbert's seminorm by −α ∧ β, which proves that T is ergodic.
For T ▽ , if the slice space S β α (T ▽ ) is nonempty, i.e., if α β, then it contains all points x such that x 1 x 2 . Hence it is not bounded in Hilbert's seminorm and T ▽ is not ergodic.
Example 2.8. Consider the perfect-information stochastic game with two states, whose Shapley operator is given by Player Max controls the first state and player Min the second. In state 1, player Max chooses an action p ∈ (0, 1] which yields the current payoff 2(1 − p) + log p and induces a probability p to switch to state 2. The payoff in state 1 is positive for all p ∈ (p 0 , 1) (where p 0 ≈ 0.2032), it attains its maximum at p = 1/2 and tends to −∞ as p tends to 0. Hence, securing a positive stage payoff entails a positive probability to switch to state 2, which is controlled by player Min. A dual interpretation holds for player Min in state 2.
Letting h : R → R be the function defined by we can write T as It is then easy to check that all the slice spaces of T are bounded in Hilbert's seminorm, hence that the game is ergodic. Furthermore, for all x ∈ R 2 such that x 2 x 1 , the recession operator of T is given by Thus, the fixed-point set of T is not reduced to the line Re, which shows that the results of [AGH15a] do not apply to this game.
In the conference paper [AGH15b] -a preprint of a longer version is available in [AGH18], see also [Hoc16b, Ch. 6] -the authors give a combinatorial criterion for the boundedness in Hilbert's seminorm of all the slice spaces of any monotone additively homogeneous self-map of R n . The combination of that result, which involves a pair of directed hypergraphs, with Theorem 2.6 provides a generalization of the graph-theoretical ergodicity condition in [AGH15a] (Theorem 5.3). Similarly to [AGH15a,Prop. 5.1], we can then translate this condition in game-theoretical terms. With this end in view, let us introduce the notion of dominion, informally speaking a subset of states that one player can make invariant for the state process.
Definition 2.9 (Dominion). Given a stochastic game Γ, we call dominion of player Max (resp., Min) a nonempty subset of states D for which player Max (resp., Min) has a strategy such that from any initial position in D, the state remains almost surely in D at all stages, whatever strategy the other player chooses.
We mention that an equivalent notion appeared in the algorithmic game theory literature, first in [GL89], then in [BEGM10]. We further mention that the name "dominion" was introduced in [JPZ08] with a definition slightly stronger that ours.
Example 2.10. Let us illustrate the notion of dominion by considering the following stochastic game Γ. It has 3 states and the action sets are [0, 1] for both players in all states. We next identify probability measures on R 3 with vectors in the standard simplex of R 3 . The transition probabilities of Γ are given, for all actions a ∈ [0, 1] of player Max and b ∈ [0, 1] of player Min, by where γ is a fixed parameter in (0, 1). Since dominions are only defined by the dynamics of the state, it is not necessary to specify here the payoff function of Γ. It is then easy to check that the dominions of player Max are Remark 2.12 (The dominion condition for Markov chains and Markov decision processes). It is instructive to apply the latter theorem to zero-player or one-player games, that is, to finite Markov chains with rewards or Markov decision processes, respectively.
In the first case, the two players are dummies, meaning that they have only one possible action in each state. Any dominion (of any player) then contains at least one ergodic class of the Markov chain and, conversely, any ergodic class is a dominion for both players. Thus, the dominion condition in Theorem 2.11 is equivalent to the classical ergodicity condition for finite Markov chains.
For Markov decision processes, only one player is a dummy, say player Min. Then, a dominion of that player is an absorbing subset of states for player Max (meaning that the state remains almost surely in this set once it has reach it, whatever the strategy of player Max is). Thus, the Markov decision process is ergodic if and only if there is no more than one nontrivial absorbing subset of states and this subset has a nonempty intersection with all the dominions of player Max.
Example 2.13. Let us consider the games associated with the operators introduced in Example 2.5.
For T , this is the trivial Markov chain, where every state is absorbing. Hence, every state is a dominion of both players and thus the game is not ergodic.
For T , this is also a Markov chain, which is irreducible. Hence, each player has only one dominion, which is the set of all state. Thus the game is ergodic.
For T ▽ , the nontrivial dominions are {1} for player Max and {2} for player Min. We deduce from Theorem 2.11 that the game is not ergodic.

Surjectivity of accretive mappings
The solvability of ergodic equation (5) can be seen, up to an additive constant, as a fixed-point problem. In this perspective, it is useful to work in the quotient vector space R n /Re. . This motivates us to study the existence stability of a fixed point, under additive perturbations, for nonexpansive maps in (finite-dimensional) Banach spaces, as well as the related and more general problem of the surjectivity of accretive set-valued mappings. The present section is dedicated to the study of the latter problem, whereas the former is investigated in the next section.
In the remainder, X refers to a finite-dimensional real vector space equipped with a given norm, denoted by · . Its dual space, X * , is equipped with the dual norm, denoted by · * , and ·, · refers to the duality product.

Preliminaries on accretive mappings.
Set-valued analysis. We recall here some basic definitions about set-valued mappings and refer the reader to the monograph [RW09] for more details on the subject.
Given a finite-dimensional real vector space Y, a set-valued mapping A : X ⇒ Y is a map sending each point of X to a subset of Y. The domain of A is the subset of X defined by dom(A) := {x ∈ X | A(x) = ∅}. The range of A is the subset of Y defined by rge(A) := x∈X A(x), and the image of any subset U ⊂ X is the subset of Y given by A(U) = x∈U A(x).
The inverse of A, denoted by A −1 , is the set-valued mapping from Y to X sending any element y ∈ Y to the set {x ∈ X | y ∈ A(x)}, i.e., such that x ∈ A −1 (y) if and only if y ∈ A(x). In particular, we have (A −1 ) −1 = A and rge(A) = dom(A −1 ). Also note that the image of any set V ⊂ Y by A −1 is given by We next define notions of continuity for set-valued mappings. The outer limit, lim sup x→x A(x), and the inner limit, lim inf x→x A(x), of A : X ⇒ Y at any point x ∈ X are subsets of Y defined respectively by the following: Then we can define the notions of outer and inner semicontinuity for set-valued mappings.

Definition 3.1 (Semicontinuity of set-valued mappings). A set-valued mapping
and inner semicontinuous (i.s.c. for short) atx ∈ X if It is continuous atx if it is both outer and inner semicontinuous.
These notions are invoked relative to any subset U of X containingx if the properties hold when restricting the convergence x →x to U, i.e., when all the sequences x k →x are required to lie in U.
Note that if A is i.s.c. at a point x ∈ dom(A), then x ∈ int (dom(A)), the interior of dom(A). Let us further mention that outer semicontinuity differs from upper semicontinuity, another notion commonly found in the literature (see e.g., [AF09]). However, when a set-valued mapping is locally bounded, these two definitions agree. As for lower semicontinuity, it is equivalent to inner semicontinuity. The reader can find a discussion on these aspects in [RW09, Ch. 5] (see in particular the Commentary Section).
Duality mapping. Duality mappings are set-valued mappings that appears in the study of Banach spaces (see e.g., [Pet70]) or in applications involving nonexpansive and monotone-like operators (e.g., evolution equations [Bro76], fixed-point approximation [Rei94]). In this paper, we only consider normalized duality mappings and refer the reader to [Cio90] and the references therein for a general view on the subject.
Definition 3.2 (Duality mapping). The (normalized) duality mapping on the normed space (X , · ) is the set-valued mapping J : X ⇒ X * defined by Note that, by the Hahn-Banach separation theorem, dom(J) = X , i.e., J(x) is nonempty for every vector x ∈ X . Furthermore, Asplund characterized in [Asp67, Thm. 1] the image of any point by a duality mapping as the subdifferential at this point of some convex function. This entails that J(x) is a compact convex subset of X * for every x ∈ X . Also, it readily stems from the definition that J is homogeneous of degree one, i.e., for every x ∈ X and every λ ∈ R, we have J(λx) = λJ(x). Finally, a straightforward application of the definitions leads to the following lemma. Example 3.4. Let X = R n . If X is equipped with the standard Euclidean norm, then J is the identity map. More generally, if X is equipped with an L p -norm with 1 < p < +∞, then J is single-valued and given for all x = 0 by where q is the positive real number defined by p −1 + q −1 = 1.
Example 3.5. Assume that the norm · on X is polyhedral, meaning that there is a finite symmetric family W ⊂ X * of linear forms on X such that x, x * , ∀x ∈ X .
We may assume that W is minimal, in the sense that no element in W may be written as a convex combination of other elements in W. This case includes the standard L 1 -and L ∞ -norms. Then, one may check that we have, for all x ∈ X , where co(U) denotes the convex hull of any subset U of a vector space.
Accretivity. Accretive operators are generalization in Banach spaces of monotone operators in Hilbert spaces. They appear in particular in the study of nonlinear evolution equations (see e.g., [Bro76]), and more recently in game theory ( [Vig10,SV16]).

Definition 3.6 (Accretive mappings). A set-valued mapping
If, in addition, rge(Id + λA) = X for all λ > 0, where Id denotes the identity map on X , then A is m-accretive.

Definition 3.7 (Coaccretive mappings). A set-valued maping
Let us mention that in a Hilbert space, the normalized duality mapping is the identity map, so that the notion of accretivity is equivalent to the one of monotony, whereas m-accretive mappings correspond to maximally monotone operators. In particular, for functions form R to R, accretive means nondecreasing, whereas maccretive means continuous and nondecreasing. To get an intuition about the results in this Section, the reader can think of this special case.
We conclude this subsection with few remarks about the latter definitions.
Remark 3.8. (a) There is an equivalent definition of accretivity: a set-valued mapping A : X ⇒ X is accretive if and only if for every x, y ∈ X , every u ∈ A(x) and v ∈ A(y), and every λ > 0, we have x − y x − y + λ(u − v) . (b) It is known that if A : X ⇒ X is accretive, then rge(Id + λA) = X for every λ > 0 if and only if rge(Id + λA) = X for some λ > 0.

Surjectivity conditions.
Local boundedness of coaccretive mappings. A set-valued mapping A : X ⇒ X is locally bounded at a point x ∈ X if there exists a neighborhood U ⊂ X of x such that A(U) is bounded. It is known that in a reflexive Banach space, an accretive mapping is locally bounded at any point in the interior of its domain (see [FHK72]). The following result (which is new, as far as we know) shows that this property also holds for coaccretive mappings, at least in finite dimension.
Proposition 3.9 (Local boundedness). Let (X , · ) be a finite-dimensional real vector space and let A : X ⇒ X be a coaccretive mapping. Then, for every point x in the interior of dom(A), A is locally bounded at x.
Proof. Toward a contradiction, assume that A is not locally bounded at a point x in the interior of dom(A), that is, for all neighborhoods U of x, A(U) is not bounded. Then, there exists a sequence (x k , y k ) k∈N in X × X such that (x k ) k∈N converges to x, y k tends to infinity, and y k ∈ A(x k ) for all integers k. We may assume that y k > 0 for all k ∈ N. Since the dimension is finite, the bounded sequence (y k / y k ) k∈N has a convergent subsequence. Let (y n k / y n k ) k∈N be such a subsequence, which converges toward some point w ∈ X .
The point x being in the interior of dom(A), there exists a scalar α > 0 such that x + αw ∈ dom(A). Letȳ ∈ A(x + αw). Since A is coaccretive, then for all integers k there is a linear form y * k ∈ J(y k −ȳ), where J is the duality mapping on (X , · ), such that x k − (x + αw), y * k 0. By homogeneity of the duality mapping J, for all integers k we have In particular, the sequence (w * k ) k∈N is bounded. We also have, for all k ∈ N, Let w * ∈ X * be a cluster point of the bounded subsequence (w * n k ) k∈N . Since the subsequence ((y n k −ȳ)/ y n k ) k∈N converges toward w, and since J is o.s.c. (Lemma 3.3), then we deduce that w * ∈ J(w). On the other hand, Equation (7) yields w, w * 0, a contradiction since w, w * = w 2 = 1 by definition of the duality mapping.
By application of the Heine-Borel property for compact sets, we readily get the following.
Corollary 3.10. Let (X , · ) be a finite-dimensional real vector space and let A : X ⇒ X be a coaccretive mapping. Then, the image by A of any compact subset of X included in the interior of dom(A) is bounded.
Characterization of surjectivity. Let us denote by dist(0, U) the distance of the origin to any set U ⊂ X , i.e., dist(0, U) = inf x∈U x . We next give a sufficient condition for an m-accretive mapping to be surjective. Then, rge(A) = X .
We now state the main result of this section.
Theorem 3.12 (Surjectivity of accretive mappings). Let (X , · ) be a finitedimensional real vector space and let A : X ⇒ X be an accretive mapping. If rge(A) = X then, for all scalars α 0, the set is bounded. Moreover, if A is m-accretive, then the two properties are equivalent.
Proof. Assume that rge(A) = X . Equivalently, we have dom(A −1 ) = X . Moreover, since A is accretive, its inverse A −1 is coaccretive (see Section 3.1). Hence, according to Corollary 3.10, the image of any compact set by A −1 is bounded. Let α, α ′ be two nonnegative real numbers such that α < α ′ . If x ∈ D α , then A(x) ∩ B(0; α ′ ) = ∅, where B(0; α ′ ) denotes the closed ball centered at 0 of radius α ′ . This proves that For the converse, it is readily seen that the coercivity condition in Theorem 3.11 is equivalent to the boundedness of all the sets D α . Hence the result, when A is m-accretive..

Fixed point problems for nonexpansive maps
In this section, we use the main result of the previous one to study problems related to the existence stability of fixed points of nonexpansive maps. We use the same notation as before. In particular, (X , · ) shall refer to a finite-dimensional real normed space. We may sometimes identify a map A : X → X with the setvalued mapping sending every x ∈ X to {A(x)}. Furthermore, recall that a map T : X → X is nonexpansive (with respect to · ) if T (x) − T (y) x − y for all x, y ∈ X . 4.1. Existence stability under additive perturbations. Let us first recall the classical link between nonexpansive maps and accretive mappings. We give the proof for the reader's convenience.
Lemma 4.1. If T : X → X is a nonexpansive map, then the mapping A = Id − T is m-accretive.
Proof. We first show that A is accretive. Let x, y ∈ X and x * ∈ J(x − y), where J is the duality mapping on (X , · ). We have where the first inequality stems from the definition of the dual norm, the second inequality from the nonexpansiveness of T , and the equality comes from the definition of J. We deduce that A(x) − A(y), x * = x − y − (T (x) − T (y)), x * 0, which proves that A is accretive. Now, let λ > 0 and z ∈ X . One can easily check that the point z is in the range of Id + λA if and only if the map x → T λ,z (x) = λ 1+λ T (x) + 1 1+λ z has a fixed point. Since T is nonexpansive, T λ,z is a contraction. More precisely, for all x, y ∈ X , we have with λ 1+λ < 1. Hence, by the Banach fixed-point theorem, T λ,z has a (unique) fixed point. This proves that rge(Id + λA) = X for every λ > 0, and so that A is m-accretive.
The following corollary of Theorem 3.12 provides a necessary and sufficient condition for the existence of a fixed point for all additive perturbations of a nonexpansive map.
Corollary 4.2 (Existence stability of fixed points). Let (X , · ) be a finitedimensional real vector space and let T : X → X be a nonexpansive map. Then, the following are equivalent: (i) for every vector g ∈ X , the map g + T has a fixed point; (ii) every nonexpansive map G : X → X such that sup x∈X G(x) − T (x) < ∞ has a fixed point; (iii) for every scalar α 0, the set D α (T ) = {x ∈ X | x − T (x) α} is bounded.
Proof. First, observe that a vector g is in the range of Id − T if and only if the map g + T has a fixed point. Thus, the equivalence between Item (i) and Item (iii) is a mere application of Theorem 3.12 to Id − T , which is m-accretive according to Lemma 4.1. Second, it is straightforward to check that (ii) ⇒ (i). We now prove that (iii) ⇒ (ii). Assume that Item (iii) holds and let G : X → X be a nonexpansive map such that sup x∈X G(x) − T (x) M for some M > 0. One can readily check that D α (G) ⊂ D α+M (T ) for every α 0. Hence all the sets D α (G) are bounded. Since we have already proved that (i) ⇔ (iii), by applying the equivalence to G we deduce in particular that G has a fixed point.

4.2.
Uniqueness condition. Given a nonexpansive map T : X → X , let us introduce the set-valued mapping F ix : X ⇒ X defined by that is, the mapping that sends each vector g ∈ X to the set of fixed points of g + T .
Observe that the inverse mapping of F ix is so that F ix is coaccretive by Lemma 4.1. By a straightforward application of the definition, we have the following.
Lemma 4.3. The fixed-point mapping F ix defined in (8) is outer semicontinuous.
In particular it is closed-valued.
We shall need the following technical lemma, which is a variant of the Hahn-Banach separation theorem.
Lemma 4.4. Let J be the duality mapping on the finite-dimensional normed space (X , · ), and let x be any vector in X . Then, Proof. Let x ∈ X \ {0}. We know that J(x) is a compact convex subset of X * (see Section 3.1). Furthermore, 0 / ∈ J(x) by definition. Hence, according to the Hahn-Banach separation theorem, there exists an affine hyperplane of X * strongly separating the two compact convex sets J(x) and {0}, i.e., there exists some vector w ∈ X \ {0} and a constant ε > 0 such that, for all x * ∈ J(x), we have w, x * ε 0.
We now state the main result of this subsection.
Theorem 4.5 (Uniqueness of fixed point). Let (X , · ) be a finite-dimensional real vector space and let T : X → X be a nonexpansive map. Then, the fixed-point mapping F ix : X ⇒ X defined in (8) is continuous at g ∈ dom(F ix) if and only if g ∈ int (dom(F ix)) and F ix(g) is a singleton, i.e., g + T has a unique fixed point.
Proof. Suppose first that the mapping F ix is single-valued at g ∈ int (dom(F ix)) and denote byx the unique fixed point of g + T . Since F ix = (Id − T ) −1 is coaccretive, we know by Proposition 3.9 that F ix is locally bounded at g. Hence there is a neighborhood U of g such that F ix(U) is bounded. We may further assume that U is included in dom(F ix) since g ∈ int (dom(F ix)).
We next show that F ix is i.s.c. at g. Let (g k ) k∈N be any sequence in U that converges to g. Since U ⊂ dom(F ix), for every k ∈ N there exists some x k ∈ F ix(g k ). The sequence (x k ) k∈N , being in F ix(U), is bounded. Let x be any cluster point. By continuity of T , we necessarily have x ∈ F ix(g), hence x =x. This proves that the sequence (x k ) k∈N converges tox, and so, that F ix is i.s.c. at g. Since F ix is also o.s.c. at g (Lemma 4.3), we deduce that it is continuous at g.
Conversely, suppose that F ix is continuous at a point g ∈ dom(F ix). Then, by definition of inner semicontinuity, g ∈ int (dom(F ix)). Let x and y be two points in F ix(g). Let w ∈ X \ {0} and for every positive integer k, define g k = g − k −1 w. We may assume, without loss of generality, that g k is in dom(F ix) for all k. Since F ix is continuous at g, hence inner semicontinuous, there exists a sequence of elements x k ∈ F ix(g k ) converging to x. Furthermore, since F ix is coaccretive, for all k ∈ N there exists a point x * k ∈ J(x k − y) (where J is the duality mapping on (X , · )) such that g k − g, x * k 0, which yields w, x * k 0. Let x * ∈ X * be a cluster point of the bounded sequence (x * k ) k∈N . From the latter inequality, we get w, x * 0. Moreover, since the duality mapping J is o.s.c. (Lemma 3.3), we also have x * ∈ J(x − y). Thus, we have proved that for any point w ∈ X \ {0}, there exists an element x * ∈ J(x − y) such that w, x * 0. We deduce from Lemma 4.4 that x − y = 0, and consequently that F ix(g) is a singleton.
4.3. Generic uniqueness. Semicontinuous mappings are "generically" continuous. Before making this fact precise, let us explain the terminology. A subset B of a set W ⊂ X is nowhere dense in W if the interior relative to W of the closure relative to W of B is empty: A subset of W ⊂ X is meager in W if it is a countable union of nowhere dense subsets in W. Dually, the complement of a meager set in W is the intersection of countably many subsets with dense interiors relative to W. Note that when W is open or closed in X , a fortiori when W = X , the complement of any meager set in W is dense in W.
Theorem 4.6 (Generic continuity, see [RW09,Thm. 5.55]). Let A : X ⇒ X be a closed-valued mapping. If A is outer (resp., inner) semicontinuous relative to W ⊂ X (see Definition 3.1), then the set of points where A fails to be continuous relative to W is meager in W.
We know from Lemma 4.3 that the fixed-point mapping is o.s.c. and closedvalued. Hence, a straightforward combination of Theorem 4.5 and Theorem 4.6 leads to the following.
Theorem 4.7 (Generic uniqueness of fixed points). Let T be a nonexpansive selfmap on a finite-dimensional real vector space (X , · ). Let F ix : X ⇒ X be the fixed-point mapping defined in (8). Then the set of points g ∈ int (dom(F ix)) where g + T fails to have a unique fixed point is meager in int (dom(F ix)). In particular, when dom(F ix) = X , the set of points g ∈ X where g + T has a unique fixed point is dense in X .

Application to Shapley operators and stochastic games
In this section, we first apply the results of the previous one to the case of monotone additively homogeneous operators. Since this includes in particular the case of Shapley operators, a proof of our main result (Theorem 2.6) readily follows. Then, we use a combinatorial criterion for the boundedness in Hilbert's seminorm of all the slice spaces of any monotone additively homogeneous operator (announced in [AGH15b]) to derive the ergodicity condition in terms of dominions which appears in Theorem 2.11. 5.1. From fixed-point to ergodicity problems. In this subsection, we show how the solvability of the ergodic equation (5) for a monotone additively homogeneous map is equivalent to a fixed point problem involving a nonexpansive map in some finite-dimensional normed space. We then adapt the results of Section 4 to the former problem. This provides in particular a proof of our main theorem (Theorem 2.6).
Let TP n := R n /Re be the quotient space of R n by the subspace Re, i.e., the set of equivalence classes over R n by the following relation: x ∼ y if there exists α ∈ R such that x − y = αe. We denote by [x] the equivalence class of any vector x ∈ R n modulo the relation ∼. Observe that for any x, y ∈ R n such that x ∼ y, we have x H = y H , where Hilbert's seminorm · H is defined in (6). Thus, Hilbert's seminorm can be quotiented into a map, which we denote by q H , over TP n . Furthermore, this quotiented seminorm is now a norm since q H ([x]) = x H = 0 if and only if x ∼ 0, that is, [x] = 0. This makes (TP n , q H ) a finite-dimensional normed vector space.
Any monotone additively homogeneous map T : R n → R n can be quotiented into a map [T ] : TP n → TP n , sending any equivalence class [x] to [T (x)]. Furthermore, it is known that such a map T is nonexpansive with respect to Hilbert's seminorm (see e.g., [GG04]), that is, Hence the quotiented map [T ] is nonexpansive with respect to the norm q H . Observe further that (λ, u) ∈ R × R n is a solution of Equation (5) if and only if the equivalence class [u] ∈ TP n is a fixed point of [T ]. Moreover, the uniqueness of the fixed point of [T ] is equivalent to the uniqueness up to an additive constant of the solution of (5), meaning that any solution of (5) is of the form (λ, u + αe) with α ∈ R.
The latter considerations allow us to adapt the results of Section 4 to the setting of monotone additively homogeneous maps, which includes in particular the case of Shapley operators of stochastic games. The next theorem, from which readily follows our main result (Theorem 2.6) gives stability conditions for the solvability of the ergodic equation (5).
Theorem 5.1 (Solvability of perturbed ergodic equations). Let T : R n → R n be a monotone additively homogeneous map. The following assertions are equivalent. (i) Equation (5) has a solution for all maps g + T with g ∈ R n ; (ii) Equation (5) has a solution for all monotone additively homogeneous maps βe + x} is bounded in Hilbert's seminorm for all α, β ∈ R.
Proof. The equivalence between Items (i) to (iii) is a straightforward application of Corollary 4.2 to the nonexpansive map [T ] on (TP n , q H ).
Finally, we already know that (iv) implies (i). It has been proved in Section 2.3 (this is an easy consequence of Theorem 2.3).
We now address the uniqueness up to an additive constant of the solution to Equation (5). From Theorem 4.7 we get the following corollary.
Corollary 5.2 (Generic uniqueness of solutions to perturbed ergodic equations). Let T : R n → R n be a monotone additively homogeneous map. Assume that the ergodic equation (5) is solvable for all maps g + T with g ∈ R n . Then, the set of vectors g ∈ R n for which the ergodic equation (5) with g + T fails to have a unique solution up to an additive constant is meager.
Proof. Since [g] + [T ] = [g + T ] has a fixed point for all [g] ∈ TP n , we know from Theorem 4.7 that the set of points [g] where [g] + [T ] fails to have a unique fixed point is meager in TP n . Let k∈N B k be this set, where for all k ∈ N, B k is nowhere dense in TP n . Denote by π : R n → TP n the quotient map. Then the set of points g ∈ R n where Equation (5) fails to have a unique solution up to an additive constant for g + T is π −1 k∈N B k = k∈N π −1 (B k ). To complete the proof, we need to show that all the sets π −1 (B k ), k ∈ N, are nowhere dense in R n . To that purpose, we next prove that the preimage by π of any set B nowhere dense in TP n is nowhere dense in R n . Let U be an open set included in cl π −1 (B) , the closure of π −1 (B). Since π is continuous, we have cl π −1 (B) ⊂ π −1 (cl(B)), which yields π(U) ⊂ π π −1 (cl(B)) = cl(B).
Furthermore, since π is a surjective continuous linear operator, then it is an open map according to the Banach-Schauder theorem, and so π(U) is open. We deduce that π(U) is empty, since B is nowhere dense in TP n . As a consequence, we finally have U = ∅, which proves that int cl π −1 (B) = ∅, i.e., π −1 (B) is nowhere dense in R n . 5.2. Geometric ergodicity conditions. In [AGH15b], Akian et al. gave a combinatorial criterion (in terms of hypergraph) for the boundedness in Hilbert's seminorm of all the slice spaces of a monotone additively homogeneous map T : R n → R n (see also the preprint [AGH18] or the author's thesis [Hoc16b,Ch. 6]). This criterion boils down to the following result, where, for any set L ⊂ [n], we denote by e L the vector in R n with entries equal to 1 on L and 0 elsewhere. We next show that the latter conditions can be interpreted in game-theoretical terms, involving dominions (Definition 2.9), when T arises as the Shapley operator of a stochastic game satisfying Assumption A. To that purpose, let us first give a simpler characterization of dominions in terms of pure actions.
To prove the following lemma, we shall use the notion of support of a Borel measure µ on a Borel space X. We recall that if this support exists, it is the unique nonempty closed set, denoted by supp µ, satisfying • µ(X \ supp µ) = 0; • if U ⊂ X is open and U ∩ supp µ = ∅, then µ(U ∩ supp µ) > 0; (see e.g., [AB06]). Note that if µ is a Borel probability measure, then it is regular (see [AB06, Thm. 12.7]) which implies that supp µ exists (see [AB06,Thm. 12.14]).
Lemma 5.4. Let Γ be a stochastic game satisfying Assumption A. A subset of states D is a dominion of player Max (resp., Min) if and only if ∀i ∈ D, ∃ā ∈ A i , ∀b ∈ B i , p(D | i,ā, b) = 1 (9) resp., ∀i ∈ D, ∃b ∈ B i , ∀a ∈ A i , p(D | i, a,b) = 1 .
Proof. First assume that (9) holds and consider any pure stationary strategyσ of player Max that assigns to every current position i ∈ D an actionā i ∈ A i satisfying (9). Let τ be any strategy of player Min and let {i k } k 0 be a sequence of states generated by the pair (σ, τ ) with an initial state i 0 ∈ D. Conditioning on the first exit time, the probability that the state leaves D at some stage is With the particular choice ofσ that we made, we have, for every stage k 0, Hence, the probability that the state leaves D at some stage is 0, that is, D is a dominion of player Max. Conversely, assume that D is a dominion of player Max. Then, by definition of dominions, for every initial state i ∈ D, there exists a mixed actionμ ∈ ∆(A i ) of player Max such that for all mixed actions of player Min, hence for all pure actions b ∈ B i , the probability p(D | i,μ, b) that the next state remains in D is equal to 1.
Takeā ∈ suppμ and assume, working toward a contradiction, that there exists someb ∈ B i such that p(D | i,ā,b) < 1. Since the function a → p( However, since U ∩ suppμ is nonempty (it containsā), it follows from the definition of the support of a measure thatμ(U) μ(U ∩ suppμ) > 0. This implies that p([n] \ D | i,μ,b) > 0, or equivalently p(D | i,μ,b) < 1, a contradiction. So, for all b ∈ B i , we have p(D | i,ā, b) = 1, hence (9) is satisfied. With dual arguments we can show that the dominions of player Min are characterized by (10). Remark 5.6. We easily deduce from the proof of the latter result that a dominion D of player Max (resp., Min) in Γ is also characterized by the following equality: where the minimum and the maximum commutes.
We can now identify the sets of states satisfying one of the asymptotic properties in Proposition 5.3. This characterization (which is established in the following lemma) combined with Proposition 5.3 and Theorem 2.6, yields the dominion condition for ergodicity of stochastic games announced in Theorem 2.11. r(i,ā, ν).
Since the map ν → r(i,ā, ν) is l.s.c. on ∆(B i ) and since the latter set is compact, we deduce that the right-hand side of the above inequality is finite. This shows that (11) is true. Conversely, assume that (11) holds true. Let i ∈ D and let ν ∈ ∆(B i ) be a mixed action of player Min. Since the map µ → r(i, µ, ν) is u.s.c. on ∆(A i ) and