MODELS OF THE POPULATION PLAYING THE ROCK-PAPER-SCISSORS GAME

. We consider discrete dynamical systems coming from the models of evolution of populations playing rock-paper-scissors game. Asymptotic behaviour of trajectories of these systems is described, occurrence of the Neimark- Sacker bifurcation and nonexistence of time averages are proved.

We are interested in the frequencies r n , s n , p n of symbols used by the players in the nth round of the game, i.e.
We may treat these numbers as probabilities that a randomly chosen player (the nth round) uses the corresponding symbol. From our assumptions it follows that i. for the revenge strategy we have    R n+1 = R n r n + R n s n + P n s n = R n (1 − p n ) + P n s n P n+1 = P n p n + P n r n + S n r n = P n (1 − s n ) + S n r n S n+1 = S n s n + S n p n + R n p n = S n (1 − r n ) + R n p n .
Dividing these equalities by 2N we obtain    r n+1 = r n − r n p n + p n s n p n+1 = p n − p n s n + s n r n s n+1 = s n − s n r n + r n p n , ii. and similarly for the treason case we get    r n+1 = r n − r n p n + s n r n p n+1 = p n − p n s n + r n p n s n+1 = s n − s n r n + p n s n . (2) Obviously, if we choose the initial conditions r 0 , p 0 , s 0 such that r 0 , p 0 , s 0 ≥ 0 and r 0 + p 0 + s 0 = 1, then r j , p j , s j ≥ 0 and r j + p j + s j = 1 for each positive integer j. Hence (1) and (2) dene the mappings of the simplex S = {(x, y, z) : x, y, z ≥ 0, x + y + z = 1} ⊂ R 3 into itself. For the revenge strategy this mapping has the form T (x, y, z) = (x − xy + yz, y − yz + zx, z − zx + xy).
We are interested in the problem: for a given nonnegative initial data (r 0 , p 0 , s 0 ) and a xed strategy, what is the likelihood that for a suciently large n the point (r n , p n , s n ) is in a subset U of S? This is the problem of existence of an invariant measure µ for the mapping T (respectively W or V λ ) such that for the characteristic function χ U of a measurable set U , the time average for each x ∈ S. We prove that for the revenge strategy such a measure exists, and this is the measure concentrated on the xed points of the mapping T . The only invariant measure for the mapping W is also concentrated on the set of xed points of W but in this case the time average T W U (x) does not exist for U being a small neighbourhood of a corner of the triangle S. It means that for this strategy, we are not able to predict the number of members of the game which use at some moment a given symbol, so in some sense the evolution of such a population is deterministic but not predictable.
2. History of the problem. There are many papers 1 devoted to study of quadratic map of the 2-dimensional simplex S. Usually they arise in the theory of population genetics [6]. One of the older are [4] and [12], where the authors modelled the Mendel's population using the map of the form H(x, y, z) = (x 2 + xy + 1 4 y 2 , 2xz + xy + zy + 1 2 y 2 , z 2 + zy + 1 4 y 2 ).
The dynamics of this map is very simple. The x points of this mapping form the curve given by the equations x = p 2 , y = 2pq, z = q 2 , p + q = 1, where p, q ≥ 0 (HardyWeinberg principle) and every point H(p) lies on this curve for p ∈ Int S. In [7], [10] the authors studied, using computer simulation, the set of quadratic maps and stated the conjectures about the asymptotic behaviour of their trajectories, starting from some point belonging to the interior of S. In particular, the results of computer experiments suggested that for each map from this family, except one, the trajectories converge asymptotically to x point or periodic orbit. For this exceptional map, W in our notation, the trajectories approach to the boundary of S.
Our conjecture is that all such type of maps with such kind of behaviour are contained in the family of W λ .
Using the Lyapunov function for the mapping W , was proved in [11], that the ω-limit set of each point p = ( 1 3 , 1 3 , 1 3 ) is innite and contained in the boundary of S. The deep results about ω-limit sets of W are in [1]. In particular, it was proved that for residual set of initial points their ω-limit set is the boundary of S.
In [14] the author proved that the sequence 1 N Σ N −1 0 W n (x, y, z) is not convergent. The key point in the proof was the estimate on the residence time of trajectory in a small neighbourhood of vertices of S of special kinds. For such neighbourhoods we can use these estimates to prove the nonexistence of time averages. In our case the neighbourhoods are arbitrary which leads to the diculties in the proof.
Note that models of rock-scissors-paper game with continuous time were studied in [2], [5].
Let x 1 (t), x 2 (t), x 3 (t) denote the frequencies of individuals at the moment t who play the symbols R, P and S, respectively. If the payo a i,j for a player using symbol i against a j-player is given by the matrix A and if we assume that the rate of increase x i xi is the dierence between its payo and the average payo of the population then we obtain the equations on the simplex . For the asymptotic behaviour of the trajectories of (6), and the existence of time averages 3. Asymptotic behaviour of trajectories of the mappings T and W . Note that the mapping T is not injective. In fact, for example for any t ∈ (0, 1 3 ) we have .
1 e re grteful to nonymous referee whih drew our ttention to the ppers TD IID IRF To describe the asymptotic behaviour of trajectories of the mapping T we will use the following auxiliary lemma which is surely folklore.
Lemma 3.1. Let Ω ⊂ R d be a compact set and L : Ω → [0, ∞) be a continuous map. If Φ : Ω → Ω is continuous mapping, A ⊂ Ω is a set of xed points of Φ, and L is the Lyapunov function for the mapping Φ i.e. for all x ∈ Ω \ A the sequence which leads to a contradiction.

The set
consists of all xed points of the mappings T and W . The behaviour of the trajectories of T is described in Proof. For any x = (x, y, z) ∈ S we put L(x) = x−c 2 , where · is the Euclidean distance. A simple calculation leads to Using the AM-GM inequality we get The last inequality becomes equality if and only if x 3 y = y 3 z = z 3 x. This implies that x ∈ A. Hence the L : S → [0, 2 3 ] is the Lyapunov function for the mapping T . Therefore from Lemma 3.1 lim n→∞ x n − c 2 ∈ L(A) = 0, 2 3 .
The value 2 3 can be obtained for x 0 = p i , i = 1, 2, 3, only. Hence for all x 0 = p i the trajectory of x 0 tends to c.
The above Theorem states that for suciently large time R n P n S n . It means that randomly chosen member of population plays strategy R, S or P with the same probability.
Note that the sum of positive numbers x + 2z, y + 2x, z + 2y equals 3, so their product is not greater than 1. Thus L(W (x)) ≤ L(x), and the equality holds only for x ∈ ∂S ∪ {c}.  The boundary of S is the union of six invariant sets: the xed points p 1 , p 2 , p 3 and the segments connecting these points. By l i we denote the segment connecting the point p i to p i+1 , i = 1, 2, 3. If x ∈ l i then W n (x) tends to p i+1 for i = 1, 2, and to p 1 for i = 3.
The next lemma gives additional information about behaviour of the trajectories of the mapping W near the boundary of the domain S.
Lemma 3.4. For given δ 1 , δ 2 , there exist N ∈ N and δ 3 such that and for each x ∈ Z there exists a constant C independent of x such that dist(W n (x), l 1 ) < Cdist(x, l 1 ) for n < N .
Proof. The mapping W has the form W (x, 1 − x, 0) = (x 2 , (1 − x) 2 , 0) on the invariant set l 1 . We choose N such that dist(W N (δ 1 , 1 − δ 1 , 0), p 2 ) < δ 2 . Next for such N we choose a suciently small δ 3 to satisfy (8) and (9). We know by Theorem 3.3 that the trajectory W n (x), x ∈ Z, tends to the boundary of S, but the the distance dist(W n (x), ∂S) is not a decreasing function of n. The inequality (10) says that dist(W n (x), l 1 ) cannot grow very fast for n < N .
Let I 1 (resp. I 2 ) be a small interval perpendicular to l 1 at a point q 1 ∈ I 1 (q 2 ∈ I 2 , resp.), dist(q 1 , p 1 ) < dist(q 2 , p 1 ). The levels of the Lyapunov function dene a mapping T : I 1 → I 2 in the following way: if the level of the Lyapunov function crosses I 1 at a point q then T (q) is the point at which this level crosses the interval I 2 . Obviously, if q ∈ I 1 and W n (q) ∈ I 2 , then dist(W n (q), l 1 ) < dist(T (q), l 1 ). The mapping T is C 1 and |T | < C, where the constant C is universal for q 1 , q 2 belonging to an interval [p, q] ⊂ l 1 . The boundedness of |T | implies the inequality (10). 4. Time averages. Now for a given set U ⊂ S and the mapping T (or W ) we investigate the existence of time average T T U (x) (resp. T W U (x)). It follows from Theorem 3.2 that for a small neighbourhood U of the point c the time average T T U (x) exists for all x ∈ S and For the mapping W the problem of existence of time averages is more complicated. We know by Theorem 3.3 that trajectory of a point x = c tends to the boundary of S, hence if U is a neighbourhood of ∂S then T W U (x) = 1. Note that if U is a set whose distance from the xed points of W is positive then T W U (x) = 0 for all x ∈ D. It means that a randomly chosen point of the trajectory starting from x = c is near ∂S, and most points of its trajectory lie in a neighbourhood of the xed points p i , i = 1, 2, 3, see Lemma 3.4.
Theorem 4.1 below implies that we cannot indicate in a neighbourhood of which of the points p i , an element of the trajectory of x lies. In another words we cannot to predict which strategy will use the randomly chosen player in the nth round of the game. The idea of the proof is simple but the diculties lie in some technical details. We dene the periodic sequence {p j }, j = 1, 2, . . ., p j = p 1 for j = 3k + 1, p j = p 2 for j = 3k + 2, p j = p 3 for j = 3k + 3, k = 0, 1, 2, . . . Let ω(p j ) be a small neighbourhood of p j . For a point x suciently close to ∂S and x / ∈ ω(p 1 ), we can dene recurrently the family of subsets {a k } of N. It follows from Theorem 3.2, the existence of a set a 1 = {n 1 , n 1 + 1, . . . , n 1 + m 1 } such that W n (x) ∈ ω(p 1 ) for n ∈ a 1 and simultaneously W n1−1 (x) / ∈ ω(p 1 ), W n1+m1+1 (x) / ∈ ω(p 1 ). The set a k+1 = {n k+1 , n k+1 + 1, . . . , n k+1 + m k+1 } satises: W n (x) ∈ ω(p k+1 ) for n ∈ a k+1 and W n k+1 −1 (x) / ∈ ω(p k+1 ), W n k+1 +m k+1 +1 (x) / ∈ ω(p k+1 ), n k + m k < n k+1 and W n (x) / ∈ i=3 i=1 ω(p i ) for n satisfying n k + m k < n < n k+1 . Note that by Lemma 3.4 m k → ∞ as k → ∞, and all the dierences n k+1 − n k − m k are bounded. Thus, the elements of the trajectory {W n (x)} with n satisfying n k + m k < n < n k+1 have no inuence on the existence of the time average. Thus, we can consider only the elements of the trajectory {W n (x)} which belong to i=3 i=1 ω(p i ). For simplicity of notation we assume that i = 1, and ω is a small neighbourhood of p 1 . Elements of {W n (x)} with n ∈ a 1 generate in the sequence χ ω (W n (x)) a block of 1's of length m 1 . Next, we have in this sequence a block of 0's of length less than some constant N independent of x, and after that there is a block of 0's of length m 2 . Next, we have a negligible a block of 0's and next the block of 0's of length m 3 . Obviously, the existence of time average depends on the relation between the numbers m i .
Similar problems occur in the study of heteroclinic attractors (cf. [2], [8], [9]) but in this case the problem of the calculation of sojourn times near the saddle xed points is simpler, roughly mi+1 mi ∼ | log µi| log λi+1 , where λ i > 1, 0 < µ i < 1 are the eigenvalues of the linearization at ith xed point. Because lack of hyperbolicity, we have an additional problem how to estimate the ratio mi+1 mi . In Lemma below we prove that it is enough to show that mi+1 mi > 7 6 .
The inequality (11) Therefore Note that From (11)  Hence The inequalities (13) and (14) imply the nonexistence of the limit of the sequence S N .
It seems dicult to describe the dynamics on the invariant curve existence of which we just proved.