CRITICAL TRANSITIONS AND EARLY WARNING SIGNALS IN REPEATED COOPERATION GAMES

. Scanning a system’s dynamics for critical transitions, i.e. sudden shifts from one system state to another, with the methodology of Early Warning Signals has been shown to yield promising results in many scientiﬁc ﬁelds. So far however, such investigations focus on aggregated system dynam- ics modeled with equation-based methods. In this paper the methodology of Early Warning Signals is applied to critical transitions found in the context of Cooperation Games. Since equation-based methods are not well suited to account for interactions in game theoretic settings, an agent-based model of a repeated Cooperation Game is used to generate data. We ﬁnd that Early Warning Signals can be detected in agent-based simulations of such systems.


1.
Introduction. Anticipating critical transitions, i.e. abrupt changes from one system state to a different system state is an important challenge in many scientific fields, like climatology, ecology, medicine, economics, or sociology.
In order to find statistical traces of impending regime shifts, an elaborate apparatus of methods has been introduced recently, comprised under the term Early Warning Signals (EWSs) [24,6,5]. Such methods have been successfully used in the fields of ecology [25], climate change [17,16], and medicine [18,20]. Both measured data, as well as data from computer simulations is investigated. Also in the field of social science (agent-based) computer simulations are performed, mainly in the fields of finance and bubble detection [27,12,13,26] and the results are scanned for EWSs.
In this paper, we introduce an agent-based model of a Cooperation Game that is known to exhibit critical transitions from the state of overall cooperation to the state of no cooperation. We then analyze the data for EWSs and see if it is possible to detect the impending critical transition before it occurs.
The agent-based approach makes it possible to model the system bottom-up, starting from the rules that govern the decisions of each individual agent. On the other hand, an equation-based top-down model would require an equation, ideally a differential equation that gives insight into the cooperation behavior of all the players involved in the game. Such a set of equations is very difficult to find and substantiating it is challenging, while the set of rules that are needed for the agent-based approach can be found intuitively and can be backed up by empirical evidence, which makes this approach well-suited for game theory.
The paper is organized as follows: Section 1 briefly summarizes aspects of stability analysis in dynamical systems and explains the theoretical background of EWSs. The statistical methods that are deployed in this investigation are introduced. Section 2 describes the model subjected to EWS analysis and details on this analysis. Section 3 presents results of both model and data analysis, while Section 4 discusses the significance of these results and implications for future research.
Systems theory describes dynamical systems as having an eigenbehavior [11] which makes them evolve towards stable states, so called equilibria. Internal interaction dynamics can cause systems to have several such equilibria, spanning basins of attraction. In the reach of these basins dynamics tend to return to equilibrium after (small) perturbations [19]. The edges of these attractor basins (or potential wells) mark tipping points (i.e. unstable equilibria) at which transitions to alternative stable states occur. Depending on the strength of positive feedback effects, these transitions can be abrupt, showing sudden shifts of regime [14], so called critical transitions.
With respect to the suddenness of these shifts, attempts have been made to derive signals for the approach of a tipping point at which critical transitions occur. One such indicator is the time a system needs to return to equilibrium after perturbation. The slowing down of recovery time, so called critical slowing down (CSD), has been found to indicate the loss of a system's resilience and thus the approach of a transition [6].
CSD is visible in the form of various statistical signals. One of these signals concerns changes in the correlation structure of a time series, caused by an increase in the short-term memory (i.e. the correlation at low lags) of a system prior to a transition and is measured with lag-1 autocorrelation of consecutive observations [24]. Another indicator can be found in the tendency of a system to drift more widely around its stable state when approaching an unstable equilibrium, i.e. a tipping point. This causes the variance in the time series to increase, measured for example by the standard deviation. Furthermore, variance can show asymmetries and more extreme values when a system's state gets close to the attraction of an alternative equilibrium. This causes skewness and kurtosis to change. All these signals are signs that can be used to predict an imminent regime shift.

Methodology.
2.1. The model. The model simulates a simple Cooperation Game that reproduces the empirical finding that without additional mechanisms such as punishment [8] contribution in Cooperation Games declines towards the free riding Nash equilibrium [22] of overall defection [15,4]. In this model the transition between cooperation and non-cooperation is critical, which makes it ideal for studying EWSs. Agents are set up on a grid with periodic boundary conditions, so that every agent has exactly 8 neighbors, i.e. Moore neighborhood. Each round, all agents get an endowment of 1 monetary unit and are asked if they want to invest their endowment into a common pool or not. After each agent made its decision, the common pool is multiplied by a synergy factor s (1 ≤ s ≤ 2) and is then evenly distributed among all agents, even those who did not invest into the common pool. The only Nash equilibrium of this system is overall defection, since switching to defection always increases personal payoff, while decreasing the sum of all payoffs. So the state with the highest sum of all payoffs is a state in which everyone cooperates, a state completely opposite the Nash equilibrium. The simulation begins with nearly all agents cooperating and continues until the Nash equilibrium is reached.
The calculation of individual payoffs is straight forward. Agents, who decided to invest in the common pool, get the payoff P 0 , which can be calculated via with the synergy factor s, the number of cooperating agents A C and the number of defecting agents A D . The defecting agents get the same share from the common pool P 0 , but they also keep their endowment, increasing their overall payoff by 1 to P 1 : Whether or not an agent decides to cooperate or defect depends on its personal cooperation probability cp. cp is defined as the chance of choosing to cooperate (0 ≤ cp ≤ 1) and is initialized with a value close to 1, e.g. 0.98. This cooperation probability is not constant, but can be influenced by one of two effects. The first effect, which can either increase or decrease an agent's cooperation probability, is called conditional cooperation [10]. If all neighbors do cooperate, the cooperation probability is increased by the change rate c. For each neighbor that does not cooperate, the cooperation probability is decreased by 8 c, with 8 being the number of neighbors that influence the decision (both direct neighbors and diagonal neighbors on the grid are considered). This value is chosen such that the negative influence of one defector can be compensated by the positive influence of all neighbors in a following turn. The numerical value of c is arbitrary and it serves only as a scaling factor. For high c the Nash equilibrium is reached fast, while a low c leads to a longer time series. However, apart from the scaling, the value of c has no influence on the qualitative behavior of the system.
The second effect, which can only decrease an agent's cooperation probability, is called reference switching. Reference switching occurs every time the payoff of an agent is smaller than its investment, i.e. the agent experiences a net loss. In this case, the cooperation probability decreases proportional to the experienced loss: with the payoff P , the investment I, the cooperation probability in round i cp i and the cooperation probability in round i+1 cp i+1 . That way, agents reduce their cooperation probability by 1% of the difference between investment and payoff.
Although more complex agent behaviors like learning [2,23], fairness [9, 1] and social preferences [21,3] would make the model more realistic, these effects are not considered, since then the system would be too complicated to guarantee critical transitions, which are of course essential for scanning for EWSs.

Data analysis.
The EWS analysis that we applied focuses on the number of cooperators at each point in time. For this, a window of investigation was chosen ending ahead of the breakdown of cooperation so that EWSs can be used to predict imminent collapses. Often in literature EWS analysis is applied to pure data [24,5]. In the case of a Cooperation Game, however, the development of cooperation shows visible downward trends well before tipping. Since our interest was on the properties of the fluctuations, not on global shifts to lower values, we decided to detrend the data [7]. For this, a polynomial fit was calculated and subtracted from raw data. The resulting residuals were then analyzed for EWSs by calculating lag-1 autocorrelation, standard deviation, skewness and kurtosis for the data in appropriately sized rolling windows. Carefully choosing the size of these windows is essential. If the interval is too large, the signal gets averaged out; too small intervals lead to a random noise on top of the signal due to random fluctuations. We found that an interval of roughly 100 data points is a suitable window size for our system.

Results.
3.1. Simulation results. The described model is initialized with an agent population of 1000 agents on a grid, an initial cooperation probability of 0.98 and a synergy factor of 1.2. The simulation then runs until the Nash equilibrium is reached, which takes a few thousand turns with these parameters. In every turn the number of cooperating agents is saved, leading to a time series that can be analyzed for EWSs.
A typical result of such a simulation run is shown in Figure 1. The percentage of cooperators is high in the beginning and slowly decreases due to changes in the cooperation probability of the agents. After roughly 3500 turns a critical transition occurs, bringing the system abruptly to the state of overall defection. The important part of this time series is highlighted in orange. Here the critical transition is imminent, yet there is no abrupt change in the number of cooperators yet. This makes this data ideal for scanning for EWSs.    3.2. Scanning for EWSs. Figure 3 shows the results of the EWS analysis for pure data (left column) as well as for detrended data (right column). The first panel in the left column shows a detailed view of the considered data that is highlighted in Figure 1 and the fit that is used to calculate the residuals. The first panel in the right column shows the residuals. The second row of panels reports on autocorrelation, the third row on standard deviation, the fourth row on skewness and the fifth row on kurtosis.
EWSs can be detected in both the pure data (left column) as well as in the detrended data (right column) for lag-1 autocorrelation, standard deviation and skewness. Only kurtosis shows no signal. For lag-1 autocorrelation there is more noise for the residuals, yet the sharp increase in autocorrelation shortly before the critical transition is more pronounced. Standard deviation shows a similar increase in both pure and detrended data, only the value of the standard deviation is shifted downwards for the detrended data, which is to be expected, since the polynomial fit was subtracted. The skewness shows qualitatively similar signals in both pure and detrended data, with a visible peak before the transition. Only the kurtosis does not produce an EWS. The main reason for this may be that an increase in kurtosis is a way of detecting flickering, i.e. the occurrence of more extreme states prior to the critical transition. Since the investigated system exhibits no such flickering the kurtosis does not produce an EWS.

Conclusion.
We report on investigations of the possibility of analyzing the resilience of cooperation in simulated Cooperation Games. The analysis aims at predicting regime shifts with the concept of Early Warning Signals. An agent-based Cooperation Game model was designed to generate rapid regime shifts in response to accumulating positive feedback, and was used as an example case on which the possibilities of EWS analysis were tested. The tests considered two kinds of data: the easily observable bare number of cooperators and the detrended number of cooperators, filtering out slow changes in cooperation, thus focusing on fluctuations. In both cases EWS analysis showed clear indications of an approaching regime shift.
Thus it was possible to show that the concept of EWSs can be applied to system dynamics derived from interaction on component level and is thus apt to capture dynamics in games, here in particular the development of simulated social dilemmas. The EWS concept thus could serve as a powerful tool to predict regime shifts and critical transitions in this context. This result is not specific to the investigated system, but a universal property of such interactions. Considering the overall aim to derive possibilities of countering negative dynamics in the development of cooperation in social dilemmas or at least to derive warning signals, our results substantiate that EWSs in the form of variance or autocorrelation of observables can be used and that such an approach yields promising results in the context of an agent-based simulation of a Cooperation Game.