GAME THEORETICAL MODELLING OF A DYNAMICALLY EVOLVING NETWORK I: GENERAL TARGET SEQUENCES

. Animal (and human) populations contain a ﬁnite number of individuals with social and geographical relationships which evolve over time, at least in part dependent upon the actions of members of the population. These actions are often not random, but chosen strategically. In this paper we in- troduce a game-theoretical model of a population where the individuals have an optimal level of social engagement, and form or break social relationships strategically to obtain the correct level. This builds on previous work where individuals tried to optimise their number of connections by forming or breaking random links; the diﬀerence being that here we introduce a truly game-theoretic version where they can choose which speciﬁc links to form/break. This is more realistic and makes a signiﬁcant diﬀerence to the model, one consequence of which is that the analysis is much more complicated. We prove some general results and then consider a single example in depth.

place on the class of graphs with a fixed number of vertices, the set of edges changing according to choices made by the vertices. Details are given in Section 1.3.
1.2. Background literature. A simple graph G = (V, E) is a set of vertices and a set of unordered pairs E ⊆ V * V with (i, i) / ∈ E. Vertices are considered indistinguishable if they have no special type over and above properties resulting from the graph itself, e.g. degree. When considering some evolutionary process for graphs where vertices represent individuals and the edges are links there are several levels of complexity, depending on whether the vertices are distinguishable, whether the edges are fixed, whether the numbers of vertices and or edges changes. There can be various dependencies between the vertices and edges.
It is often desired to generate a random sample of members of some class of graphs. For example, one would like to generate elements of the set of r-regulargraphs on n nodes. Here the vertices are indistinguishable and the number of vertices fixed. Discussion of this can be found in [4]. In [3] the graph at some time t grew by the addition of vertices and edges. New vertices were added one at a time, the one added at time t + 1 was linked to some "c" of those at time t, these latter vertices being chosen with probabilities proportional to the degrees of the vertices at time t. This gives rise to a power law distribution of degrees, see [5] for rigourous derivations.
A third possibility is that the graph grows by some reproductive process. Graph theory has introduced a number of products which form a new graph from two earlier ones. For example, the tensor product of graphs G(V, E) and H(W, F ) is the graph M = (V * W, G) where ((v1, w1), (v2, w2)) ∈ G, if, and only if, (v1, v2) ∈ E and (w1, w2) ∈ F . Another approach is that of [44,45,46]. In their model each vertex at time t produces an "offspring", current edges are retained and then edges are formed between the vertices at t and those at time t + 1 according to some rule. This process grows indefinitely, and the authors track various properties such as chromatic number and diameter through time.
In contrast to the models above, the vertices may possess a type, which may change during time. In many of these latter models there are two types in the population (resident and mutant), and the state of the population, the set of mutant individuals, say, evolves according to an evolutionary dynamics. As per Moran's [33] model individuals are selected, according to some fitness dependent on type, and they replace one of their neighbours chosen at random. The most important feature of such populations is the fixation probability, the probability that a randomly placed mutant will eventually replace the resident population [2,9].
We note that for real populations both of these features change, and there has been much research considering the way in which the interactions of the individuals at the vertices affect not only their type but also the structure of the network, see for example [47]. The growth and structure of the graph can be dictated by an evolutionary game, and in particular by the prior interactions of individuals, as in [36,37]. Here links are formed or broken at rates which depend upon the types of the individuals, and the authors consider an evolving population where evolutionary dynamics happen on a slower timescale than the linking dynamics. Alternatively [43] considered various dynamic models of network formation assuming reinforcement learning. A model where it is not past interactions but reputation that influences structure is given by [17], and one where prosperity influences the structure is given in [12]. A good review of work in this area up to 2010 is given in [39], while a more recent but less specific review is in [1]. An example of more recent work is [40] who discussed such co-evolutionary models examining the prisoner's dilemma and the snowdrift game, together with the Birth-Death process. As stated in [1], while the details of the above models vary, a common theme to many is that cooperative behaviour is easier to achieve when cooperators can group themselves together and exclude defectors effectively.
In this paper, following on from [6], we consider networks of individuals represented, as in evolutionary graph theory, by a simple graph. The population itself will not evolve, i.e. its composition does not change since there is no birth or death of individuals, but the connections between individuals will change according to their preferences and strategic decisions. The emphasis of our paper is the evolution of the structure itself, and although there may be many types of individuals (a type being its targeted number of neighbours) our process is thus not co-evolutionary, as the types do not change. Our process could be considered as a detailed examination of a snapshot in evolutionary time of a more complicated version of the type of model from [36,37], and it would be possible to embed our model into such an evolutionary scenario.
The kind of networks that we consider arise naturally in many contexts including in biology, economics and sociology, and this is the subject of a lot of recent research interest. Example networks are companies which trade with each other in economics, individuals who are friends in sociology or the owners of neighbouring territories or food webs ( [15]) in biology. In social animals there are dominance and mutualist interactions and, for example, primate social structures can be complex and influence key behaviours such as the level of cooperation ( [49,50]).
Populations can also change in important ways over short periods of time. A population of animals may contain individuals with different degress of desire to interact with others. This phenomenon is called "sociability" and has been investigated in various species. Examples of such differences include (non-human) primates [11], bottlenose dolphins [51], [13] and sheep, where different individuals differ in how close they want to be to other flock members [42].
In these examples there are temporary links between individuals. The probability of a link existing between a pair of individuals will in reality often be affected by the relatedness of those individuals, by their genders, by dominance relationships or by spatial factors. In the bottlenose dolphin case the links are reciprocal, whereas in others they might be initiated, or broken, by the action of one individual only. Similarly the absence of a connection may benefit one but not the other (for example a female and a poor quality male). An important related area of research is that concerning biological markets and partner choice [34,35].
In this paper we do not model such complex behaviours, but simply the network of interactions. Individuals (vertices) in our networks are distinguishable only by the number of links they would like to form with others. Each individual will want to make changes which improve its number of links, but since all links involve two individuals, the actions of others can make an individual's situation worse. The key difference with previous work [6] is that individuals will not just choose a random action which improves their situation in the short term, but will strategically choose which individuals are best to link to/ break from. As we shall see, this makes the situation much more complex.

1.3.
A dynamic network population model. In [6] we considered a population of individuals represented by the set V = {1, 2, . . . , n} and the simple graph G = (V, X) where X = (x ij ) i =j=1,...n described the links/edges between pairs 288 MARK BROOM AND CHRIS CANNINGS of individuals with x ij = 1 if there is a link and x ij = 0 otherwise. In particular we considered a random process in discrete time on the evolving edge set X t = (x ij,t ) i,j=1,...n , where the subscript t indicates that this is the edge set at time t. Throughout the paper we shall use the same terminology, but we shall often drop the subscript t when we describe features of the process that are not time-dependent and this causes no ambiguity. In [7] we investigated the possible paths and end states of the process. We describe the process below.
At any given time t individual i had a number of edges e i,t to other individuals, and the vector e t = (e 1,t , e 2,t , . . . , e n,t ) was referred to as the sequence e t .
At each time point an individual was chosen and allowed to add or remove an edge. Each vertex had an acceptable range [m i , M i ] of edges to other vertices, where 0 ≤ m i ≤ M i ≤ n − 1. In much of the work m i = M i = t i , with t i denoted as the unique target of i, and this is the situation in the current paper.
If i was selected with e i < m i (such a vertex is referred to as a Joiner) then it formed a new edge, connecting to one of the other vertices it was not connected to at random. If e i > M i (a Breaker) then it broke one of its edges at random. Otherwise, it neither created nor broke an edge (a Neutral vertex).
At successive time points, a vertex was chosen at random, with i being selected with probability p i > 0, and an edge (potentially) changed following the above, yielding a homogeneous Markov chain.
The transitions at time t depended upon t only through the state, i.e. the process was homogeneous, and were defined as follows: 1) For any x * which differs from x in a single entry, where x ij = 0, x * ij = 1 for some i, j, 2) For any x * which differs by x in a single entry, where x ij = 1, x * ij = 0 for some i, j, 3) Similarly for any other x * , differing from x in two or more entries, The probability of the sequence being unchanged is simply 1 minus the sum of the above probabilities.

2.1.
A brief synopsis of previous papers [6] and [7]. In [7] we studied graph theoretic aspects. We give a brief outline here in order to inform the current work. We introduced the notions of the deviation of a graph from a given target and the score of a sequence.
, and the deviation of the above graph X t is defined as the sum of the vertex deviations, Υ t = i=1,n i,t .
Definition 2.2. There is clearly a minimum value of the deviation for any given collection of the ranges [m i , M i ], and this is termed the score.
For unique targets with m i = M i = t i , if the score is 0 the sequence is called graphic and there is a lot of work on such sequences, see e.g. [22], [18], [23], [32], [41].
There will be a set of sequences, and a corresponding set of graphs, which achieve the score. In [7] these were termed J(min) and K(min) respectively, and it was proved that there is always a path of allowable moves enabling the process to reach a member of the minimal set, K(min). Since the process could never increase the deviation of the graph, once J(min)/K(min) is reached, that set cannot be left. It was proved that for non-graphic sequences K(min) was connected, so that the process always converged to a unique closed set of states. Note that this is not true for graphic sequences, where |J(min)| = 1 but K(min) will often have more than one element (e.g. for (1, 1, 1, 1) we have |K(min) = 3|), since once the mimimal set is reached there will be no transitions, so no pair of elements of K(min) are connected. Finally in [7] we demonstrated how to find the score of any sequence using a modified Havel-Hakami algorithm [18], [23], and how to find all members of K(min) (and hence J(min)) using the methodology of Ruch-Gutman [41].
In [6] we considered the Markov chain itself. We considered the Markov chain over K(min), since all states not in this set will be transient following the above. We showed that the process was reversible and so in detailed balance, which thus yielded a unique stationary distribution over K(min). We then demonstrated a method to find this stationary distribution.
We considered some specific classes of sequence, in particular arithmetic sequences and all or nothing sequences, and in particular gave a form for the stationary distribution of the latter for an arbitrary number of vertices. We revisit the former in the current paper.
2.2. Current and future work. The current paper considers detailed aspects of the structure of K(min). In [7] we proved certain restrictions to exist on the its elements, e.g. that for any such graph we know that all Joiners (that is vertices have degree less than their target) must be joined. Here we extend such analysis to consider the possible sequences of Joiners, Breakers (vertices with degrees greater than their target), and Neutrals (the remaining vertices) through time. Vertices fall into four classes; those which are always neutral, those which are never Joiners, those never Breakers, and those which can be either Joiners or Breakers. We specify rules regarding the possible sequences of class memberships of the vertices as we move through the monotone decreasing of targets.
We then consider a model, which in contrast to those of [6] and [7] considers the possibility that the individual at a vertex may choose between the available possibilities according to some aspect of the future costs at that vertex. We have chosen to consider the case where an individual is capable of calculating the stationary distribution which will result from various changes. We discuss various criteria for switching including some where an increase in costs is possible. For systems where the calculation of the stationary distribution is not reasonable we introduce two threshold models, basing their decision on a recent sequence of states.
In Section 5 we consider a specific example (target {4,3,2,1,0}) calculating for each of the 64 strategy combinations over the minimal set, the payoff for each possible switch. We consider cases where a switch can only occur if there is a cost lower than the current one, cases where switches are possible when the cost is no bigger, and cases where there is a cost incurred in switching. We identify multiple pure Nash equilibria.
We consider two models which do not require the evaluation of the stationary distribution, rather being based on estimates from recently visited states, and thresholds for switching. These show different behaviour to the full model.
A number of questions have been left open here. We have discussed in detail but not resolved the question of what can occur under "non-strict" moves, i.e. those which allow individuals not to make their deviation as small as possible at every opportunity.
In this paper we have thus focused on two main issues; the structure of the minimal set K(min), which we expect most processes to converge to, and the introduction of strategic movement. The latter in particular is complicated and hard to deal with in generality, and so we have restricted ourselves to considering one example in detail, and demonstrating the important concepts to consider in any more extensive analysis. From this paper we see this complexity, but also that strategic choices lead to clearly different results than the simply random process from [6].
We shall discuss in a subsequent paper [8] certain classes of targets, especially those with score 1. These are the closest sequences to graphical sequences, and yield certain simplifications that will make them more amenabe to analysis. This will also involve the consideration of cases with mixed Nash equilibria.
3. A strategic model. In the model from Section 1.3 each individual has a target number of links, specified by the vector t. The graph updates through a two stage process, where a random individual is selected to update its links, and if it is below (above) its target number of links, it picks a random link to form (break). There is no strategic element to this process, which evolves as a Markov chain.
However, it may be that it is advantageous to form/break some links rather than others. For example it would be better to form a link that is less likely to be immediately re-broken, either because the change made is for mutual benefit or because the individual linked to would be likely to break another link when given the choice (through its own preference, or if it has many links that it can break).

3.1.
Strategies. The population state is denoted by the edge set X as before. In each state any individual can be selected to change one of their edges. They have n distinct (pure) choices, to change their edge to any of the other n − 1 individuals, or to make no change. We shall denote the probability that individual i chooses to change edge x ij , conditional on i being selected to make the change, by u ij , with u ii denoting the probability that no change is made. We have the following pure strategies: individual i chooses to change edge x ij is denoted by u ij = 1, and i making no change is denoted by u ii = 1. Thus we can write all selected changes in the form of a matrix U, with U having row sums equal to 1.
The strategy matrix U depends upon the state X, and so the full set of strategies of the population is denoted by U X (similarly its elements by u ij(X) ), where the strategy of individual i is represented by the set of ith rows of this collection of matrices. For any x * which differs from x in a single entry, where x ij = 0, x * ij = 1 GAMES ON DYNAMICALLY EVOLVING NETWORKS I 291 or x ij = 1, x * ij = 0 for a given i, j, In [6] the only changes made were to reduce the deviation of the individual where possible, and to not make any change if this was not possible (even if some changes allowed the deviation to stay the same). We shall call any move which decreases this deviation an improving move, any one that increases it as a worsening move and a move which does not change the individual's deviation is termed a neutral move. In the current paper we assume a unique target t and so every actual change either increases or decreases the deviation of the selected individual. Thus the only neutral move is making no change. We shall denote as the strict system the case where an individual must make an improving move if at least one exists and cannot make a worsening move, i.e. if only neutral and worsening moves exist, it must make a neutral move.
Individuals which are rational but could only see the immediate consequences of any changes would follow the strict system, and it seems logical that real systems would often follows these rules. One consequence of following the strict system is that the score cannot increase. Thus for the strict system, u ij = 0 whenever x ij = 0, e i ≥ t i or x ij = 1, e i ≤ t i , as this change would involve a worsening move. The simple "strategies" used in [6] are consistent with the strict system, for example If we allow non-strict moves, our analysis can be significantly complicated, as we see later in Section 3.

3.2.
Payoffs. Individuals want to minimise their deviation, and we shall denote their payoff as the negative of their expected long term deviation. In particular, if a process with individuals following strategies U X has a unique stationary distribution over X, denoted by π(X), then the payoff to individual i is where i (X) is the deviation of i in state X.

3.3.
Stability and strategy switches. Individuals can try to improve their payoffs by changing their strategy. We consider two types of strategic changes; Local changes -individual i changes the ith row of U X for a single state X only; Global changes -individual i changes the ith row of U X for any number of states simultaneously.
Making such global changes might be advantageous, since any individual change would potentially affect a number of the probabilities of occupying particular states/ taking particular paths, which then may alter the best choices elsewhere. This would depend upon a significant ability to calculate, and so it may be reasonable to assume that it is not possible for individuals to make such global changes. An individual with limited cognitive powers might, for example, only use strict moves and local changes.
Only changes by a single individual at a time are allowed. We shall say that an individual i plays optimally if under all allowable changes U X → U i X (including no change) it chooses a strategy which achieves max i R i (U i X ). A strategy set is a Nash equilibrium under local or global changes if, under all allowable changes by i :

MARK BROOM AND CHRIS CANNINGS
In this section we briefly consider the process not restricted to the mimimal set K(min), strict moves or neither.
3.4. The strict system on the non-minimal set. For non-strict moves, and allowing individuals to play sub-optimally (where optimal play is as described above), clearly the process does not possess the nice properties from [6] and [7]. The process does not necessarily converge to the minimum set, or have a unique stationary distribution. If individuals are allowed to remain on the same deviation when there is an opportunity to improve, then the strategy of all individuals making no changes in any circumstances will clearly not lead to the minimum set, for example.
Clearly for any non-graphic sequence the target can never be achieved, and so there is always at least one individual that is not achieving their target. Given that the targets are all between 0 and n − 1 (inclusive), any Joiner (Breaker) must have at least one available edge to form (break). For strict moves then the population can never settle in a single state and eventually either a reduction in score or a continuing sequence of moves on a given score will be reached. In [6] all allowable improving moves happened with some non-zero probability, some of which reduced the score, so that the population always eventually reached the minimum set. This is not the case here. The following example shows that even with strict (but not necessarily optimal) play the minimal set may not be reached.

Example 1.
Consider the case where vertices A and C have target 1, B and D have target 0. Suppose currently neither of A and B is linked to either of C and D. When A has no links, the strict system means that it must form a link when selected. Suppose that it always links to B. Similarly, assume that C always links to D when its score is 0. B and D will simply break the link that they have (unless they have more than one which can never occur following the above) and so the system will simply consist of two pairs A and B, C and D, repeatedly breaking and forming links, with a score of 2. The minimal set consists of the single graph with the single link A to C, which will never be reached.
However, we are principally interested in what happens when optimal play is employed by individuals, and so we shall restrict ourselves to finding Nash equilibria of our system. Note that we have not shown whether it is possible to have Nash equilibria that are not in the minimal set. This is a difficult open problem; we conjecture that it is possible to have such equilibria, but that it requires a sufficiently large number of individuals with a sequence of high score.
3.5. The non-strict system. Considering Example 1 above, we shall now show that it is possible for non-strict moves to be optimal under certain circumstances.
Example 1 Cont. Consider again the case where vertices A and C have target 1, B and D have target 0. Assume that B,C,D always play strictly (i.e. reduce their deviation when they can, and do nothing when they are on target) and that B will always split from A as its first choice move, D will always split from C as its first choice, and C will always link to D as its first choice, similarly to before. Then if A always links to B as its first choice, we obtain the situation where all follow legal strict moves but the unique minimal graph is never reached and the payoff for each is 1/2.
Suppose that A is faced with the situation where it is connected to B, but C and D are split. What if it chooses to link to C? This is a non-strict move as it is currently achieving target. However, if it does this, if C and D are picked next, they will do nothing (we assumed they always behave strictly, and they are now on target). Either A (assuming it plays as previously described except in the original case) or B will split A-B, which will lead straight to achieving the target for all (in an expected time of 2 moves). Thus here is an example where non-strict play is optimal.
Note an alternative in the initial situation would be to wait until B breaks from A and then link to C as the next opportunity, but this would mean a sub-optimal decision was made at the original decision; also it would take at least twice as long to reach the minimal graph (A would have to be picked with mean time 4, and only then would the final target have been achieved if C has not linked to D in the meantime).
4. The strict system on the mimimal set. In the above we have defined the minimal set K(min) which achieves the score of the sequence, and hence a minimal mean payoff for individuals in our population, and the strict process, where individuals try to reduce their own short term deviation as much as possible. It seems reasonable that (despite the discussion in Sections 3.4 and 3.5) in many, likely most, circumstances, the long-term behaviour will reduce to strict moves, restricted to K(min).
We now consider processes restricted to the strict system acting on K(min). We define the matrix A = (a kl ) as the matrix of transtions on the elements of K(min), such that a kl is the probability of moving from the minimum graph state X k to X l , i.e.
using equation (1). Thus given a choice of strategies U X that lead to K(min) there is a unique matrix A. We can (and will) consider alternative strategy combinations U X , leading to different matrices A. This will, in general, greatly reduce the number of transitions that need to be made. For example, if n = 5, X has 2 10 elements, each yielding a potentially different 5 × 5 matrix, giving 25600 transition parameters to consider. In Example 3 K(min) contains 8 elements and so at most 64 potential transitions, all but 28 of which have zero probability.
We know from Theorem 5 from [7] that K(min) contains at least one element with no Joiners, and at least one element with no Breakers. Thus no vertex is always a Breaker, and no vertex is always a Joiner. K(min) is connected (unless the target sequence is graphic, when all vertices are neutral), and so no vertex is sometimes a Joiner and sometimes a Breaker but never Neutral. Thus every vertex is Neutral for some elements of K(min), and we have the following: For the set K(min) associated with any target sequence, we can divide the individuals (vertices) into four classes: a) Those which are sometimes a Joiner and sometimes a Breaker (and also necessarily a Neutral) for some elements of K(min); these individuals constitute the set S A , b) those which are Joiners (or Neutrals) for some elements but never Breakers; the set S J , c) those which are Breakers (or Neutrals) for some elements but never Joiners; the set S B , d) those which are always Neutral; the set S N .

MARK BROOM AND CHRIS CANNINGS
Clearly, since any graphical sequence has deviation zero, it will only have individuals of type d). For non-graphical sequences, since K(min) contains elements with Joiners and elements with Breakers, it must contain either individuals of type a) (and possibly some of type b) and/or c) as well), or if there are no type a)'s there must be individuals of types b) and c). Proof. Suppose that we have an individual of type d). It is always Neutral, so its set of links do not change. It cannot be joined to any Breaker, as breaking that link would leave the graph in K(min), but change the vertex to a Joiner. Similarly it cannot be split from any Joiner, as forming that link would leave the graph in K(min), but change the vertex to a Breaker. Any vertex of type a) will be a Joiner at some times and so joined to the original vertex, but a Breaker at other times, and so split from it. Thus there can be no such vertex.
w i } will have precisely the same transition graph as t = {t 1 , t 2 , . . . , t n }, in the sense that if two states are joined in the original graph then the augmented states will be joined in the transition graph of the augmented target, where the new vertex will be in class d).
Proof. (i) Consider a graph with the original target t. Add a new vertex and link it to all elements in S J , split it from all elements in S B , and link it to any given element i of S N if and only if w i = 1. The new vertex will be a Neutral, and every individual will have the same deviation (and in the same direction) as the original graph. For the given choices of the w i 's, there is a 1-1 correspondence between the set of original graphs and the set of transformed graphs; the only difference is the presence of the new vertex. But this vertex is split from all vertices that can ever be Breakers and joined to all that can be Joiners, and so its deviation will never change, i.e. it is an element of S N . (ii) Similarly, let us remove an element of S N . As stated in the proof of Lemma 4.1, this vertex must be split from all vertices that can be Breakers, i.e. S B , and joined to all elements that can be Joiners, i.e. S J . This means that removing this vertex will take one from the links of all elements of S J and none from all elements of S B . It will take a link away from an element of S N if there exists one, and so this will determine the values of w i for i ∈ S N which keeps the deviations the same (note that an element of S N has the same links to all other elements of S N for all elements of K(min), so we have the same set of w i s for all graphs). Thus again we have a one-to-one correspondence between the two minimal sets.
Thus from Lemma 4.2, if we have individuals of types b), c) and d) then we can effectively add or remove as many elements of type d) as possible without changing the minimal set at all. This leads us to the following result.
. . , d n+m + l} has precisely the same transition matrix as t.
Thus for any target t of the same type as t i.e. only elements in S J and S B , there is a set of targets whose transition graphs are isomorphic to the transition graph of t.
. Further for some m, suppose we have a graph G with graphic sequence{d 1 , d 2 , . . . , d m }. Now fix the links between pairs of elements, one from the original sequence and one from G, so that they satisfy the following: every element of G is connected to every element of S J , every element of G is not connected to any element of S B . Elements of G and S N can be connected or not in any combination, where vertex i in S N has w i edges into G and ver- . . , d m + x m + u} will have precisely the same transition graph as t = {t 1 , t 2 , . . . , t n }, with the additional m vertices from G being in class d).
Proof. Similarly to part (i) of Lemma 4.2, all vertices in G are split from all vertices that can ever be Breakers and joined to all that can be Joiners, and so their deviations will never change, so they become elements of S N for the new target. All original vertices have the same deviation, so there is again a 1-1 correspondece between graphs from the original and new sequences, for the given set of links between the elements of the original sequence and G. But these links can never change within K(min), and thus we have the same minimal set.
In Theorem 4.4 we compare two sequences, where any directed edge (or its absence) in the transition graph for the original case is present or absent in the new one, and in the new case a wider set of (non-strict) strategies are available to other states not existing in the first. If an individual is restricted to moving among the states that existed in the first case, with the links to the new vertices as described, the new states have not altered the game, so that any strategies that satisfies any given equilibrium/ stability conditions in the first case would also satisfy them in the second. Further any move which involves the new individuals will increase the score; thus disallowing all such moves, there is no difference between the two cases. Thus as long as the population starts in this irreducible set and only strict moves are allowed, the solutions are equivalent.
In [7] the conjugate vector v = (v i ) of t was defined by v i = #{j : t j ≥ i} (where # means "the number of"). We write f j = r≤j (t r +1−v r ) and e i = max j≤i [0, f j ], so that the terms e i are the non-negative record values of the sequence f = (f i ), and then d i = e i − e i−1 (with e 0 = 0).
Definition 4.5. The deficit of the sequence t was defined as the summation From Definition 4.5, we can see that there is a close connection between the deficit and the sequence f = (f j ) for j = 1, . . . , λ, which we shall refer to as the deficit profile.
Definition 4.6. We say that the deficit profile has its peak at µ if f µ > 0, f µ = max 1≤j≤λ f j and if µ < λ then f µ > f µ+1 . If there is no such value µ, then we say that the deficit profile has no peak.
Clearly, the deficit is 0 if and only if there is no peak in the profile. Otherwise the deficit takes value f µ . We show below that the deficit for a target is either equal to the score defined earlier, and thus to the HH-score obtained using the modified Havel-Hakami algorithm, or to that score minus 1. The following example makes it clear that all combinations of odd or even deficit and odd or even score (with the score the deficit or the deficit plus one) are possible. See [7] for a discussion of some issues regarding odd and even deficits.
Proof. Here the deficit is µ r=1 t r + µ − µ r=1 v r , so that the score takes this value or this value plus 1. Consider a vertex x < µ. (i) Suppose that x is a Breaker (linked to y > t x individuals) in the first µ vertices. Let us remove this vertex and its links, and consider the remaining graph. We note that the order of the vertices up to the original µ + 1 is unchanged, due to the definition of the peak (t µ > v µ − 1 ≥ v µ+1 − 1 ≥ t µ+1 ) if µ < λ; or if µ = λ since then t λ > t λ+1 . Thus this will have the new deficit where z is the number of links removed from vertices µ+1, . . . , n. Note we must also add the value y − t x > 0 to this, which is the penalty for vertex x missing its target. We note that the amount that the graph misses the target will be equal to this sum GAMES ON DYNAMICALLY EVOLVING NETWORKS I 297 or one more. Adding this we obtain µ r =x t r + (y − t x ) + µ − µ r=1 v r + µ + z ≥ Def icit + 2(y − t x ), since z ≥ y − µ, which is at least 2 greater than the deficit for the original sequence, and so greater than the score for the original sequence. This is a contradiction, since we see that such a graph cannot achieve the score. Hence no vertex 1, . . . , µ can be a Breaker, so it cannot be in sets a) or c).
(ii) To be in set b), a vertex must be a Joiner in some graphs that achieve the score. Firstly we note that if in any graph achieving the score µ can be a Joiner, it is simple to find a graph where x < µ can be a Joiner, given that its target is at least as challenging.
Now consider vertex µ that is a Joiner for some graph, where it is connected to all of the other top µ individuals and z of the others (this will yield a deviation at least as low as any alternative); note that we need none of these z edges to have target 0. Removing this vertex and its links as before yields the new deficit µ−1 r=1 ((t r −1)+1−(v r −1)) to which we can add t µ −µ+1−z which is the penalty for µ missing its target. Thus the deficit of this graph is We note that the number of removed edges is µ − 1 + z. In the original sequence there were µ vertices with target at least µ (and so at least 1) including vertex µ that we removed, thus we can certainly find the z non-zero targets we require.
We have shown that through this process we obtain the original deficit, but this might not be the same as the score.
Applying the modified HH-algorithm, removing a single vertex out of order does not affect the resulting score obtained. There is always a minimal graph which has any given vertex as a neutral, and the score will be achieved by linking it to the vertices with the biggest target; doing this is equivalent to removing the vertex out of order following the modified HH process. Thus the added term t µ − µ + 1 − z above, which is the difference between the two deficits, can also be seen to be the difference between the two corresponding scores. Hence we have our result. Now consider the sequence s, with conjugate w, defined by s i = n − 1 − t n+1−i . We shall refer to s as the dual sequence of t. This corresponds to the target number of breaks (as opposed to links) of the vertices in reverse order, i.e. in the order of increasing target of links and so decreasing target of breaks. It is easy to see that the score of s is the same as the score of t, and that for any given graph for t, considering the graph with the complementary set of links has precisely the reverse Breaker-Joiner structure for s. We will also define the reverse peak when counting the number of target breaks in the reverse direction.
Corollary to Theorem 4.8. Suppose that we have a target t, with conjugate v, so that s, with conjugate w, defines the reverse sequence of breaks. If the reverse peak is at ψ where s ψ + 1 − w ψ > 0, then vertices n + 1 − ψ, . . . , n are in class c).
Theorem 4.9. Suppose that we have a target t, with conjugate v and Durfee number λ, with the following properties: Then vertices 1, . . . , λ are in class b) and vertices λ + 1, . . . , n are in class c), and thus there are no vertices of class a) or d).
Proof. From (i) and (iii) the peak is at the Durfee number λ, and so with the addition of (ii) and (v) (these imply that t λ > t λ+1 ) the conditions of Theorem 4.8 are satisfied, and all vertices 1, . . . , λ are in class b).
We have that w r = n − v n−r , and so s r From the above (iv) and (v) are the equivalent to (i) and (ii) in the reverse direction, and then imply that the reverse peak is at vertex n − λ (counting from the back), and so vertices λ + 1, . . . , n are in class c).  Proof. There are two cases to consider: (i) t λ ≥ λ, t λ+1 < λ ⇒ w n−λ = n − λ , i.e. the Durfee number is λ and the Durfee number from the back is ψ = n − λ so there is no gap. (ii) t λ ≥ λ, t λ+1 = λ ⇒ w n−λ < n − λ, so the Durfee number from the back is less that n − λ. t λ+1 = λ ⇒ s n−λ = n − λ − 1 ⇒ w n−λ−1 ≥ n − λ so that the Durfee number "from the back" is greater than or equal to n − λ − 1, so it must be n − λ − 1. Thus there is a gap of 1.
Theorem 4.11. For any target sequence, consider a pair if vertices i and j. Then if t i ≥ t j then either: i and j are in the same set; i ∈ S J and j ∈ S A ; i ∈ S A and j ∈ S B ; i ∈ S J and j ∈ S N ; i ∈ S N and j ∈ S B ; i ∈ S J and j ∈ S B .
Proof. For any pair of vertices, there are 16 orderings. There are nine orderings above, so we need to exclude the other seven.
Firstly from Lemma 4.1, we know we cannot have classes a) and d) for the same sequence, so that removes two orderings.
Consider such a pair of elements. Suppose that for a graph in K(min) that i is a Breaker and j is a Neutral. Clearly i has (at least) t i − t j + 1 more links than j. Pick a vertex linked to i and not to j, and link it to i and break it from j. This maintains the total deviation. Repeat the process, until i is a Neutral. Clearly then j is a Breaker. Thus for a graph in the minimal set with order BN, there is also one with order NB.
By analogous argument if we have ordering NJ, there will be an alternative graph with JN, and if we have BJ, there will be an alternative graph JB. This argument leads to the following cases.
Suppose a vertex is in class a). Then it can be B, N or J, so that any vertex with higher target is sometimes a J, i.e. is in class a) or b).
Thus class c) followed by class a) is not possible. Similarly any vertex with a lower target is sometimes a B, i.e. is in class a) or c). Thus class a) followed by class b) is not possible. Suppose a vertex is in class d). It is thus always N, so from the above cannot have a B above, or a J below. Thus class c) followed by class d), and class d) followed by class b) is not possible.
Suppose a vertex is in class b). It is sometimes a J, so any vertex higher is also sometimes a J, and so is not in class c). Thus class c) followed by class b) is not possible.
Theorem 4.11 implies restrictions on the possible sequences of set membership as we move through the target sequences from higher to lower value. Figure 1A shows the sequences which are not ruled out by Theorem 4.11. We can start at any vertex and proceed along the arrows. The number of possible paths of length n is (n + 1) 2 , yielding an upper bound for the number of distinct set sequences. The actual number of such sequences realised for n = 1 − 5 (obtained by computing the possibilities) is {1, 2, 5, 10, 17}. Certain sequences are easy to exclude, for example we cannot have only b's and d's, or c's and d's. For n = 4 we find additionally that sequences bbba, bbaa, bbac, baac, bacc, aacc, accc do not occur. Finally, we note that the number of ordered targets is 2n−1 C n . Table 1 gives the set membership of the vertices for the cases n = 3 and n = 4; the results for dual targets (as previously defined, if target t = (t 1 , t 2 , . . . , t n ) then its dual is t * where t * i = ((n − 1) − t i )) can be deduced straightforwardly. The number of states for K(min) is also given.  Table 1. The set membership of vertices for targets with n = 3 and n = 4, and number of graphs in the minimal set. The omitted sequences are all duals of those included.
As stated before, a sequence with a score of 0 is graphic, and yields an end state where all individuals have payoff of zero. The next simplest case is a score of 1, where at every state of the minimum set there is a unique individual which has not achieved its target, and so has an incentive to change. This is one of the situations that we investigate in a subsequent paper. We note here that we have not considered mixed equilibria in the current paper at all, as this would require consideration of a new example and significantly lengthen the paper. That such equilibria exist will be demonstrated in the subsequent paper [8].
5. An illustrative game: The arithmetic case. In [6] we considered the following illustrative example, which is a member of the class of games that we termed "arithmetic" sequences, where t i = i − 1 i = 1, . . . , n. Example 5. the target {4, 3, 2, 1, 0}. The score for this target sequence is 2, yielding K(min) with eight elements. These are graphs, but we write them below in the form of their corresponding sequences, distinguishing between the two graphs that have the same sequence. We label them G 1 to G 8 in the following order 43221, 33220, 33211(1), 23210, 43212, 33211(2), 42211 and 32210 the state 33211(1) being the graph where the vertex with target 4 is joined to the vertex with target 0 (and thus the 3 is joined to the 1) while 33211(2) has the 4 joined to the 1 and the 3 joined to the 0. For the rest of this section, for simplicity we will simply denote the vertex with target i as t i (we note that the order from previous sections is thus effectively reversed here).
Note that using Lemma 4.2, we can see that the target 3210 has the same set of solutions, by removing t 2 and its edges from our original sequence. The score is, of course, again 2. If we order the eight possible minimal graphs 3221, 2220, 2211(1), 1210, 3212, 2211(2), 3111, 2110, then the solutions are precisely the same as for the 43210 case. The transitions for target 43210 are shown in Figure 1B.
More generally for the arithmetic case with target sequence t 2m , t 2m−1 , . . . , t m+1 , t m , t m−1 , . . . , t 2 , t 1 , t 0 we have that the subgraph of the m + 1 vertices of greatest degree is complete, and the subgraph of the m + 1 vertices of lowest degree is empty. Thus we can replace the original target by two sequences with targets m, m − 1, . . . , 2, 1, 0 and m − 1, m − 2, . . . 2, 1, 0 respectively, and restrict the acceptable graphs to bipartite graphs with the two sets corresponding to the m of greatest degree and the m of lowest degree in the original sequence. The central node, that is the original t m with subsequent target 0, can be ignored. The score is m. Now at any point in time the system will be in some minimal graph. A vertex is picked at random with equal probabilities (i.e. 1/5), and that vertex might initiate a switch to another minimal graph, as per Figure 1B. For example suppose the current minimal graph is G 1 . Then if vertex t 4 ,t 3 or t 2 is picked no change can occur since they are currently at their target value, on the other hand if vertex t 0 is picked it must break its link with t 4 and so the minimal graph changes to G 2 , while if vertex t 1 is picked it has a choice of breaking with t 4 , and thus the graph becomes G 3 , or breaking with t 3 when the graph becomes G 7 . An individual's strategy thus needs to specify which of these to choose. We specify the transition probabilities for G 1 as 3/5 to remain as 1, 1/5 to switch to 2 and r 1 /5 the probability of choosing t 4 , and s 1 /5 the probability of choosing t 3 where r 1 + s 1 = 1.
There are six graphs where there are two different moves which stay within K(min). For G i we take probabilities r i and s i when a choice is possible, that is when i = 1, 2, 4, 5, 7, 8. We have chosen the numbering of the minimal graphs so that G i is obtained from G 9−i by reversing the latter and subtracting element by element from 44444. This imposes a structure on the transition matrix A, for example it is sometimes block triangular. The transition matrix A under the choice model is such that Note that had we chosen to work with the target 3210 the above matrix would have been modified by subtracting 1 from each diagonal element, and changing the scaling factor from 5 to 4, this affecting only the speed of convergence of the system, which would be quicker.
We consider only the case where each r i is either 0 or 1, i.e. pure strategies. Thus we have 64 possible transition matrices over the elements of K(min). In order to code the matrices we take the following function f from the state number to a power of two (note that there are no choices to be made in states 3 and 6); We label the matrices A 0 , A 1 , . . . , A 63 with connections as above. We suppose that when the current set of strategies is specified by A i then the system will have converged to its stationary distribution, which requires that strategic changes that lead to a change of matrix are infrequent in comparison to moves between graphs. The eigenvectors of the 64 matrices are given in Table 2 (see the Appendix). Matrices A 0 , A 4 , A 8 , A 12 have two unit eigenvalues, and thus have a two dimensional space for the dominant eigenvectors. We denote the extreme eigenvectors of A 0 by V L 0 = (0 L , 0, 0, 0, 0) where 0 L = (2, 3, 1, 4) (see Table 2) and V R 0 = (0, 0, 0, 0, 0 R ) where 0 R = (4, 1, 3, 2). Similarly we have 8 L = 0 L , 4 R = 0 R , 8 R = 12 R = (4, 3, 1, 2) and 4 L = 12 L = (2, 1, 3, 4), and so 5.1. Payoffs and strategy switching. Given a stationary distribution π = (π 1 , π 2 , . . . , π 8 ), we calculate the cost c i (which is minus the payoff) for each vertex t i ; c 2 = 0 since vertex t 2 is always neutral, and otherwise we have c 0 = π 1 +π 3 +2π 5 +π 6 +π 7 , c 1 = π 1 +π 2 , c 3 = π 7 +π 8 , c 4 = π 2 +π 3 +2π 4 +π 6 +π 8 . These costs are shown in Table 4 (see the Appendix). Note that, with the exception of matrices 51, 55, 59 and 63, where all the costs are 0.5, the cost for one of vertices t 0 and t 4 is always greater than 0.5, while the cost for vertices t 1 and t 3 is always less than 0.5. The total cost is equal to 2, the score.
We consider the optimal strategies for the vertices. Suppose that the matrix has some current index from 0 − 63, the system currently has some particular graph from G 1 − G 8 and a vertex from t 0 − t 4 is chosen at random. We consider its current payoff vis-a-vis those it would obtain by making an allowable switch to a different strategy. Suppose t 0 is chosen then in graphs G 2 , G 4 , G 8 the vertex is at its target, while in graphs G 1 , G 3 , G 6 , G 7 the vertex has a single link which it must break. Accordingly no change will occur in the value of M . Vertex t 0 only has a choice when graph G 5 occurs, so can switch from s 5 = 1 to s 5 = 0 or vice versa, i.e. decrease the index of the current matrix by 8, or increase by 8, and then compare the current cost with that in the new matrix (at its new stationary distribution). In a similar manner vertex t 4 will only have a choice if the current graph is G 4 when it can alter s 4 so change state index by 4. For vertex 1 the situation is somewhat different. There are two graphs where t 1 has a choice, G 1 and G 2 , which can cause changes in the matrix index by 1 or 2 respectively. We may restrict the possible change to a single one, i.e. 1 or 2 (corresponding to local changes as defined in Section 3. 3) or we may permit double changes in the index (corresponding to global changes). Similarly vertex t 3 may change the index by 16 or 32, or both. Thus for example if in matrix 5, for t 0 we can move to matrix 13, for vertex t 1 we can move to indices 4, 7 or if a double move permitted to 6, for vertex t 3 to 21, 37 or with a double move to 53 and for vertex t 4 to index 1. The possible choices and outcomes for matrix 5 are shown in Table 3 (see the Appendix).
We need to note that the possible invasions of states whose stationary distribution does not cover the whole space is restricted in some cases. For example suppose that the current matrix is 5. Then the possible neighbours (i.e. potential invaders) have indices 4, 7, 1, 13, 21, 37 (the binary neighbours of 5 under a single change). The first three of these have the same stationary distribution as 5, the fourth has a different stationary distribution but with the same support, while the final two have stationary distributions with support the whole set. With a two-dimensional space for the dominant eigenvector we need to consider both extremes and then deduce the result for the whole space. For example if the state is 0 and the system is at the extreme 0 L then switching to matrix indices 8, 16 or 32 will not change the stationary distribution. Switching to matrix 1 or 2 will cause a switch to a stationary distribution over the whole space. A switch to matrix 4 will cause a change but with the same restricted space, i.e. possibly to 4 L . In general for V L there are 3 potential switches which leave the stationary distribution unchanged, one which changes the distribution but not the coverage, and two which would change to a distribution over the whole space.
Whether a switch is made depends on the payoff to the vertex involved under the current matrix and that under the matrix resulting from a switch. We examine four scenarios, comprising either (1) switch only if the expected payoff increases or (2) switch if the expected payoff does not decrease, in combination with either (a) do not allow double switches or (b) allow double switches. The case where only an improvement allows a change might reflect that a cost is involved in the switch, albeit a small one. We refer to the four possibilities as 1a, 1b, 2a and 2b. We obtain radically different results, though we concentrate primarily on 1a. Later we consider the case where there is cost involved in switching.
As an example we suppose the current matrix index is 5. Table 3 gives the payoffs for the various vertices for matrix 5 and its various neighbours. The payoffs underlined are the ones that need to be compared, and those flanked with * 's are those payoffs which initiate a change under the various rules.

5.2.
Results. Table 5 gives the results of calculations and specify which states are invaded by which alternatives under the four possible rules 1a, 1b, 2a, 2b. Care needs to be used in interpreting the results for matrices with indices 0, 4, 8, 12. In general when we are at matrix 0 the stationary distribution can be any µ0 L + (1 − µ)0 R where µ ∈ [0, 1]. The distribution 0 L is invaded by 1 or 2, while 0 R is not as there is equality of the payoffs so in general for any µ = 0 there will be invasion by both 1 and 2. A similar argument implies that 3 will invade provided µ = 0, while 16, 32 and 48 will invade provided µ = 1. These arguments apply in all four cases, and in a similar way for the indices 4, 8 and 12. The remaining possible invasions by 4 and 8 are somewhat different. For case 1a there will be no invasion whereas in case 2a µ0 L +(1−µ)0 R will be replaced by µ4 L +(1−µ)4 R and µ8 L +(1−µ)8 R , respectively. Exceptional cases where for some state the eigenvector is at the extreme of its two dimensional space are singular, i.e. only when µ is 0 or 1. In general we will not reach these odd states and so they can be omitted.
For both 1a and 1b it turns out that none of the 4 extreme matrices associated with indices 0, 4, 8 and 12 can invade any of the matrices outside of that set, and that each is invadable by other matrices. In each case there are 16 matrices which are absorbing (i.e. are pure Nash equilibria), The first two pairs and 51 have no predecessors, i.e. there are no matrices from which our process will arrive at them. Additionally states 0, (4; 8), 12, (55; 59), 63 have no predecessors. For 1b we find essentially the same results, the set of eight above cannot invade outside their own set, and furthermore for the stable matrices the double change never allows an invasion.
For cases 1a and 1b the transition matrix has no asymptotic cycles. The indexing we have introduced would allow for the representation of the matrices on the vertices of a 6-cube, the edges representing possible transitions. As an example of the possible flows we have illustrated the possible transitions starting with matrices 4 and 8. None of the transitions when starting from matrix 4/8 involve a switch by 8/4. This allows the flow to be represented on a 5-cube, shown in Figure 2A.
In examining the possible changes which can be made for a given vertex we introduced the notion of a dual matrix. If the index of some matrix M is, in reversed binary notation, (i 1 i 2 i 3 i 4 i 5 i 6 ) then we refer to M * with binary index (i 6 i 5 i 4 i 3 i 2 i 1 ) as its dual; for example A 27 with binary index (110110) has dual with binary index (011011) so matrix A * 27 = A 54 , and we write A * 27 = A 54 . Further if we have a set of matrices T then we denote by T * the set whose matrices are the duals of those in T . Now we have seen earlier that in 1a vertex t 0 only has a choice to exercise when the current graph is G 5 , when a change alters the index by 8. We see from Table 4 for which matrices there is an improving change, and denote this set by T 0,5 (a list of the matrices in the set is given below). For vertex t 4 when in graph G 4 a change alters the index by 4, with an improving change occurring for the set T 4,4 = T * 0,5 . For vertices t 1 and t 3 the situation is somewhat more complex. For t 1 there are two graphs where a choice is available, G 1 and G 2 . If the graph is G 1 then vertex t 1 will have the option to change the index by 1, while if G 2 then a change by 2 is possible. Thus if t 1 is chosen when the graph is G 1 then an improving change of the index by 1 will be made for the set of matrices denoted by T 1,1 , 24 matrices in all. There are 16 pure Nash equilibria (PNEs) where obviously no change would be made, and there are 24 other matrices where no change should be made. If vertex t 1 is chosen in G 2 then again there are 24 matrices where an improving change, by index 2, should be made to the matrices in set T 1,2 . Thus for vertex t 1 improvement will be made in matrices within the set T 1 = T 1,1 T 1,2 irrespective of the graph.
Finally if vertex t 3 is chosen when the graph is G 7 then an improving change by index 16 should be made for matrices in the set T 3,7 = T * 1,2 , and for vertex t 3 in G 8 improvements can be made, changing the index by 32 in matrices T 3,8 = T * 1,1 .

A switching fee.
Suppose that there is a fee associated with switching. Thus if an individual has current cost x and cost y if it were to make a particular switch, then supposing there is a fee for switching of z the switch is only made if y + z < x. Thus cases 1a and 1b have z = 0 + , while for 2a and 2b have z = 0. Table 7 (see the Appendix) gives the thresholds at which the set of PNEs increase. It is assumed that if a switch is made the new cost will apply for some large number of steps before a further switch is made. Thus a fee of z should be regarded as applying per time step. Figure 2B shows the possible flows, analogously to Figure 2A, when a cost of 0.1 is imposed.

Towards simpler rules.
We have derived the changes which would lead to long term improvement in the payoffs if the complex computation of the resulting stationary distribution were possible. In practice an individual at some vertex might only know its own links which for example would mean that individual t 4 would not be able to differentiate graphs G 1 , G 5 and G 7 , nor G 2 , G 6 and G 8 . It might know the links of all the vertices which would allow it to differentiate all the graph states but again it would not know which matrix state applied. We discuss only the first of these cases. What information might be available to the individual? We might reasonably assume that it could keep track of the recent costs incurred, this providing an approximation to the cost incurred for this matrix. It might be able to keep track of the graphs through which it had most recently moved which would provide a proxy for the matrix. For example suppose that the system is currently in G 1 and the most recent switches between graphs were then under the assumption that during this period there were no changes of the s i , we can deduce that s 8 = 1, r 2 = 1 and r 4 = 1 so that the current matrix has 32 included in its binary index expression, but neither 4 nor 2, i.e. is one of 32,33,40,41,48,49,56,57. As we increase the length of memory we will reduce the number of possible matrices though of course a sequence made up of, for example, only 1's and 2's would give limited information. Suppose then that an individual keeps track of the most recent changes from the six graphs where two different choices can occur, for example does G 1 switch to G 3 or G 7 , i.e. is s 1 equal to 0 or 1, does G 2 switch to G 4 or G 8 etc. When an observation on the switch of each of graphs G 1 , G 2 , G 4 , G 5 , G 7 , G 8 has been made the matrix state can be determined. For example if the observed switches are respectively to G 7 , G 4 , G 1 , G 6 , G 1 , G 6 then one can infer that the matrix is 25. In a similar manner provided that over the period during which the information is collected there is no change of matrix, that matrix can be inferred exactly. If there is a switch of matrix then that might be immediately detected by an inconsistency in the recorded switches, and the collection of switches could be restarted. On the other hand it is possible that a switch of matrix might occur without causing an inconsistency, but since none of the earlier switches between graphs would be invalidated the individual can just update the appropriate data. In fact the individual just needs to keep track of the system until all of the six (potential) changes have been recorded.
Having obtained the matrix exactly then the correct switch, if any, would be determined from a check list for that vertex which might have evolved through time, though would require having a list of the 64 appropriate switches. Some simplification might be used. For example suppose we consider vertex t 0 . Then we see from the 4th and 13th columns of Table 5 (see the Appendix) that a switch should be made only for the 6 matrices 25,26,27,29,30,31. These indices require 16 and 8, not 32, and at least one of 1 and 2. In a similar way for vertex t 4 there are only six matrices where a switch should be made 22,30,38,46,54, 62 the analogues of those for vertex t 0 , and the matrices require 2 and 4, not 1, and at least one of 16 and 32. The situation for vertices t 1 and t 3 is inevitably more complex.

5.5.
A threshold model. We consider now a simple threshold model for decisionmaking by the individuals. Although it is unreasonable to expect the individuals to be able to compute the cost implications of making a specific change to their plays they will have a reasonable knowledge of their recent costs. Given that changes are likely to be infrequent, taking the average cost over the last few steps should approximate the true reward fairly well. Suppose then that given this good estimate any individual with a choice changes if this value is above some threshold. For simplicity suppose that individuals t 0 and t 4 use the same threshold h 04 while individuals t 1 and t 3 use h 13 , a reasonable simplification since t 0 and t 4 have the same distribution of costs, as do t 1 and t 3 . Then a matrix will be stable if and only if max(c 0 , c 4 ) < h 04 and max(c 1 , c 3 ) < h 13 . For example suppose we take thresholds h 04 = 0.76 and t 13 = 0.31, then {25, 33, 38} are the possible stable matrices reached under this rule. Table 8 (see the Appendix) should be interpreted in the following way; for given thresholds everything to the left and below that threshold point is stable. We note that the entries which have no others below or to the left in the table are precisely those where c 0 = c 4 and c 1 = c 3 . The other pairs of matrices have reversed costs e.g. c 0 for 22 is equal to c 4 for 26. If the thresholds lie below the line joining (1.0, 0.0) and (0.5, 0.5) then none of the matrices are stable. There is a switch at every matrix, and the system is irreducible.
The above presupposes that the thresholds are set at some point and are never changed. However this seems an unreasonable assumption. For example suppose that the thresholds were h 04 = 0.8 and h 13 = 0.2 then the system will ultimately only be fixed at matrix 45. A change of threshold h 04 to something less than 0.8 will make state 45 unstable, so that now all matrices are unstable.  Table 8 correspond to stable sets of matrices. Moreover since c 0 +c 1 +c 3 +c 4 = 2 then the h/2's corresponding to these critical matrices lie in the 3-simplex. Now consider the space R 4 and define for x ≥ (0, 0, 0, 0) the set S(x + ) = {z ∈ R 4 |z i ≥ x i , i = 1, 4}. Now there are 31 distinct cost sets all lying on the 3-simplex; we refer to these as critical points. Now for x and y define z = x ∼ y by z i = max(x i , y i ) i = 0, 1, 3, 4. Then we have the set of critical points defined by "if x and y are critical points then so is x ∼ y." If S and T are the sets of critical points corresponding to x and y then S T is the set of critical points corresponding to x ∼ y, and the region in which these matrices are stable is S(x ∼ y) + . For the cases 2a and 2b the system is much more complex. Table 6 (see the Appendix) gives the possible moves. Note that if in the 1a/1b case, we have from Table 5 that i is not invaded by some neighbour j, and j is not invaded by i, then the appropriate costs must be equal. Thus in Table 6 we will have that i and j can invade each other. There are no stable matrices (PNEs) and it is easy to see that under 2a (and thus under 2b) the system is connected.
6. Discussion. We consider an ordered set of vertices each with a target number of links, comprising the target sequence. At any stage a specific vertex will have an excess of links and so be a Breaker (i.e. wishes to break one of its existing links), a Joiner with a deficit of links (wishes to add a link), or else a Neutral (satisfied). In [7] we began analysis of the set of graphs whose vertex degrees are minimally distant from the target sequence. This set, which we termed the minimal set, was characterised in terms of the Breaker-Joiner structure of its graphs. We introduced a random process describing an updating procedure where vertices are sequentially selected to update their links, and when selected a vertex tried to reduce the difference between its number of links and its target, which resulted in the system finally settling into this minimal set, and proved that the minimal set was connected under this process. In [6] we investigated this random process, and found the (unique) stationary distribution for a particular class of target sequences.
Here we have focused in more detail on the vertices during this process and in particular once the minimal set has been reached. As the system moves around the minimal set the vertices will (unless always Neutral) change their state. We addressed questions relating to when a vertex is, for example, at some time a Breaker, at some time a Joiner and at some time a Neutral; in our terminology belongs to S A . Looking at the set of vertices we considered what combination of vertices can jointly occur, and what types of target sequences lead to what combinations. The focus was the proof of general results.
In our previous work there was no strategic element to link updating, as it simply followed a random process. Here we have introduced strategies where individuals choose which vertices to link to or break with, and indeed whether to make such a link or break at all. We have seen that in such circumstances sometimes the minimal set is never reached, and sometimes it is sensible to make moves which in the short term do not reduce the difference between an individual's target and number of links as much as possible.
We have considered our strategic process, assuming that we always do try to minimise the above difference (denoted as the strict system) and that we reach the minimal set. Here strategies are chosen to reach the stationary distribution of the Markov updating process, and we see that there may be many stable distributions. We have considered a particular example target 43210 in detail to demonstrate the issues.
The analysis of the optimal behaviour for target sequence 43210 involves considerable complexity. It is not our intention to assert that such calculations are in practice available to the individuals, since they involve calculating the stationary distributions for the current and potential states, but that some simpler process might yield essentially equivalent results. We have introduced two such possible models based on the collection by the individuals of some recent data. We intend to investigate such possible simpler rules in subsequent work. For both models 1a and 1b we have multiple pure Nash equilibria (PNEs), sixteen in all. As pointed out earlier we require the population to settle close to its stationary distribution and to stay there for some time if individuals are to benefit from their switch, so this suggest such a situation can only arise in a species which has a reasonable length of life and interaction.
In Section 4.2 we have specified which combination of population graph and individual make possible a change. For example in graph 5 the system might change if vertex 0 is picked. Computations then find in which transition matrices a change occurs, in this case 25,26,27,29,30,31. Similar analysis is carried out for each vertex.
Studies of networks in wild populations have a long history. For example [38] investigated the formation of groups amongst wild chimpanzees, and the manner these were affected by age, gender and current reproductive status. Certain classes, e.g. estrous females, avoided each other, while some classes, e.g. anestrous females, sought each other's company. Thus within a group there would be a preference for certain interactions, and the avoidance of others. If those preferences were symmetric, that is, two individuals both wish to associate with each other, or to avoid each other, then the implied set of targets will give a simple graph (i.e. with no deviation).
In contrast to the above models with reciprocity our model has no such symmetry nor such specific pairwise preference. The individuals have preferences which are intrinsic to themselves and they do not differentiate between who they wish to join to, except through the costs implied. In human groups it appears that some individuals seek many contacts and other seek few. Dunbar [14] initiated the study of the number of relationships which primates could maintain. In humans it has been suggested ( [14], [24]) that an individual will have around 5 close associates, and around 12 − 15 at a secondary level of attachment, then around 35 in a third layer and perhaps a total of 150 in all. What seem not to have been investigated is any intrinsic wish for links amongst humans, though this would seem a priori, and from common experience, to be evident. It is this kind of model which we have developed here. Our particular example is in some ways extreme in that we have one individual who wishes to link to everyone, and one who wished to avoid all links. The other problem is that the group size is very small, though the complexity involved is considerable. In a later paper we intend to look at some larger, less extreme groupings.
(C) Whenever we find an eigenvector with zeroes in some positions other than 3 and/or 6 we can, by symmetry, find other matrices with the same eigenvector.