A note on differential games with Pareto--optimal Nash equilibria: Deterministic and stochastic models

Pareto optimality and Nash equilibrium are two standard solution concepts for cooperative and non-cooperative games, respectively. At the outset, these concepts are incompatible-see, for instance, [ 7 ] or [ 10 ]. But, on the other hand, there are particular games in which Nash equilibria turn out to be Pareto-optimal [ 1 ], [ 4 ], [ 6 ], [ 18 ], [ 20 ]. A class of these games has been identified in the context of discrete-time potential games [ 13 ]. In this paper we introduce several classes of deterministic and stochastic potential differential games [ 12 ] in which open-loop Nash equilibria are also Pareto optimal.

It is important to identify games with Pareto-optimal Nash equilibria because these strategies give the best benefit to the society and, at the same time, the players receive fair payments. Players do not have incentives to change the accorded strategies to play the game, obtaining a reduction of risks and costs caused by an state of anarchy among the players [2].
In this paper we consider potential differential games (PDGs) [12], for both deterministic and stochastic models. Our main objective is to identify some classes of PDGs which have Nash equilibria that are also a Pareto solution. To this end, we present two approaches, the direct case and the potential case. The former approach is based on particular features of the game's primitive data, namely, the payoff functions and the game dynamics. In contrast, in the potential case a key assumption is that we are dealing with potential games. This is a basic fact to determine when a multiestrategy is a Pareto-optimal Nash equilibrium.
We will restrict ourselves to open-loop differential games. It is important to keep in mind that there are games whose open-loop Pareto-optimal Nash equilibria might be no longer Pareto optimal when closed-loop multistrategies are considered; see [6], [9]. For an exception to the latter case, see [18].
The structure of this note is as follows. Section 2 contains some definitions and known results about cooperative and non-cooperative (deterministic) differential games. We also include the definition of a PDG. Our main observations are presented in Section 3, where we introduce the direct case and the potential case mentioned above. In Section 4, we extend to stochastic differential games some of the facts in Section 3. We conclude with some comments in Section 5.

Differential games.
Consider the following open-loop (deterministic) differential game. (For further details see [11], [19], for instance.) LetN := {1, . . . , N } be the set of players. The state equation iṡ such that (1) has a unique solution and, in addition, the payoff function converges absolutely for u(·) = (u 1 (·), . . . , u N (·)) ∈ U where U := U 1 × · · · × U N is the space of open-loop multistrategies. The parameter ρ > 0 in (2) is a given discount factor. We first consider the noncooperative case. Assuming that the players want to maximize their payoff functions, a Nash equilibrium for the game (1)-(2) is defined as a multistrategy u * such that where we use the standard notation . In order to introduce the special models we are concerned with we will use the following concept from [12].
A noncooperative differential game is said to be a potential differential game (PDG) if there exists an optimal control problem (OCP) whose open-loop solutions are open-loop Nash equilibria for the original game. (See [12], [14].) Remark 1. Given a function P (t, x, u) we can define an objective function which together with the system equation (1) describes an OCP.
From [12], if (1)-(2) is a PDG with an associated OCP as in Remark 1, then P is called a potential function for the game.
A trivial example of a PDG is a team game. See Example 1, below. Now, consider the cooperative case. For u = (u 1 , . . . , u N ) and The corresponding reward vector r(u * ) is said to be a Pareto point. Moreover, if instead of (5), r(u) r(u * ) holds, then u * is called weakly Pareto optimal. We recall the following known facts for cooperative games; see, for instance, [11], [19], [21].
then u * is Pareto optimal.
Lemma 2.2. A multistrategy u * ∈ U is Pareto optimal if and only if, for every i ∈N , u * maximizes J i on the set In view of Lemmas 2.1 and 2.2, finding a Pareto solution is essentially the same as solving an OCP. Hence, for the existence of such solutions it suffices to give conditions for the existence of optimal controls. See [3], [11], [16], [19], [21], [23], for instance.
3.1. The direct case. The simplest example of a PDG with Pareto-optimal Nash equilibria is a team game, defined as follows.
Example 1. (See Example 1 in [12].) Team games. The game (1)-(2) is said to be a team game if the functions g i in (2) are all the same. In other words, there is a function P (t, x, u) such that With this P in (4), it is trivially seen that a team game is a PDG. That is, if u * ∈ U optimizes (4) subject to (1), then u * is a Nash equilibrium for (1)-(2); see [12]. Moreover, u * is Pareto optimal for (1)- (2). Therefore, a team game lies in the class of games we are interested in. (As noted in [12], a team game can have a Nash equilibrium that is not an optimal solution for the associated OCP.) A class of stochastic team games is studied in [5] from the viewpoint of decentralized stochastic optimal control. Theorem 3.1. Consider a differential game as in (1) Suppose that there are functionsĝ i ,f i , such that one of the following conditions holds for every i ∈N : Then the differential game (1)-(2) is a PDG. The associated OCP has an objective function J as in (4) with potential function Hence, if u * = (u 1 , . . . , u N ) ∈ U maximizes J, then u * is an open-loop Nash equilibrium. In addition, if U is convex and J i is concave on U i for every i ∈N , then u * is also Pareto optimal.
Proof. This result follows from Corollary 1 in [12] and Lemma 2.1 above.
Theorem 3.1 is illustrated by the following example.
3.2. The potential case. In the remainder of this section we impose the following hypothesis.
The following restatement of Pareto optimality is useful in some applications. Recall the notation in Lemma 2.2, above.
Theorem 3.2. Assume H. If u * is a multistrategy such that, for every i ∈N , u * maximizes J i on U i and, in addition, u * maximizes (4), then u * is a Pareto-optimal Nash equilibrium.
Proof. The theorem follows directly from Lemma 2.2 and the definition of a PDG.
The following Corollaries 1 and 2 follow from Theorem 3.2 and Lemma 2.2, respectively. Corollary 1. Assume H. If a multistrategy u * is the unique maximizer of J k for some k ∈N , and if u * is also a maximizer for (4); that is, there is an index k ∈N such that, for every u ∈ U, then u * is a Pareto-optimal Nash equilibrium. Furthermore, if u * is not the unique multistrategy satisfying (9), then u * is a Nash equilibrium and it is also weakly Pareto optimal.
Corollary 2. Consider the game (1)-(2). If a multistrategy u * is such that, for each i ∈ N, then u * is a Nash equilibrium that is also Pareto optimal.
with α, ρ > 0, and for player 2 is which are subject to the system equatioṅ This model is a PDG with potential function The potential function P and (12) define the associated OCP to this game. To obtain the optimal solution for the associated OCP, we have the following Hamiltonian system: Solving this Hamiltonian system, we have that the optimal solution u * = (u * 1 , u * 2 ) is .
The corresponding state variable is given by In addition, (9)-(10) hold. Therefore, the multistrategy u * is a Nash equilibrium and Pareto optimal for the game.
3.3. Some remarks. The following comments relate our results in this Section 3 and some known facts in the literature.
1. Team games whose Nash equilibria maximize (4), also satisfy (11) and so they are Pareto-optimal Nash equilibria. 2. All the cases considered in Theorem 3.1 satisfy (11). 3. A game as in Corollary 2 is not necessarily a PDG. 4. Remark 1 in [20] presents a particular case of Corollary 2 but requiring differentiability and convexity conditions for (1)-(2). 5. Example 3 has payoff functions depending on the state variable of the game and, in addition, player 1's payoff function also depends on the player 2's strategy variable. This is in contrast to Theorem 3.1, where, for each i, the (instantaneous) payoff functionĝ i depends only on the strategies of player i. Thus, Theorem 3.1 and Theorem 3.2 give different ways to identify differential games with Pareto-optimal Nash equilibria. 6. A result similar to Theorem 3.1(a) was obtained in [9] for a dynamic resource management game that considers overtaking multistrategies. 7. Another result similar to Theorem 3.1(a) appears in [1]. 8. Using PDGs to obtain Pareto-optimal Nash equilibria, it is possible to relax concavity and differentiability conditions on the game's primitive data. See [12] and [20]. 4. Stochastic differential games. To define a stochastic differential game we replace (1) and (2) with DIFFERENTIAL GAMES WITH PARETO-OPTIMAL NASH EQUILIBRIA 201 where x ∈ R N , W (·) is a d-dimensional Brownian motion, and respectively. In (13) and (14), u(·) is an open-loop multistrategy in U for which (13) and (14) are well-defined; see [23], for instance. For each i ∈N , let σ i := (σ i1 , . . . , σ id ) be row i of the N × d matrix σ in (13). The definitions and remarks in Sections 2 and 3 are similar for the stochastic case, with the obvious changes. For instance, in (4), we replace the right-hand side by its expected value. Similarly, the simplest example of a stochastic PDG is again a so-called team game, that is, a game in which all the players have the same payoff function.
On the other hand, if we consider a stochastic differential game as in (13)- (14), Theorem 3.1 becomes as follows.
Theorem 4.1. Consider a stochastic differential game as in (13)-(14), with f = (f 1 , . . . , f N ). Suppose that there are functionsĝ i ,f i andσ i , such that one of the following conditions holds for every i ∈N : Then the game (13)- (14) is a stochastic PDG where the associated OCP has objective function J as in (4) and potential function P =ĝ 1 + · · · +ĝ N .
Hence, if u * = (u 1 , . . . , u N ) ∈ U maximizes J, then u * is an open-loop Nash equilibrium. If, in addition, U is convex and J i is concave on U i for every i ∈N , then the reward vector r(u * ) is a Pareto point.
Proof. We prove part (a) only. The proof for (b) or (c) is similar. Let u * = (u * 1 , . . . , u * N ) be an open-loop solution of the corresponding OCP defined by the potential function (15) and the dynamics (13). Fix an arbitrary i ∈N , and let u i = u * i be an open-loop strategy for player i. As u * is optimal for the OCP, then Adding to both sides of this inequality the constant Hence, since i ∈N was arbitrary we conclude that u * is an open-loop Nash equilibrium for the game (13)- (14) when the condition (a) is satisfied. On the other hand, by Lemma 2.1, u * is also Pareto optimal.
The following example illustrates Theorem 4.1.
where F and σ are given functions [15]. This game is as in Theorem 4.1(a). Hence it is a stochastic PDG with potential function Assuming that the instantaneous utility functions L i are strictly concave, the optimal solutions of the associated OCP are both Nash equilibria and Pareto optimal.
5. Concluding remarks. In this note we have identified several classes of openloop differential games, for both deterministic and stochastic models, which have open-loop Pareto-optimal Nash equilibria. Our work hinges on the concept of potential differential games (PDGs). Briefly put, a PDG is a differential game for which we can associate an optimal control problem (OCP) whose solutions are Nash equilibria for the original game.
We present two approaches to identify Pareto-optimal Nash equilibria. The first one is based on particular features of the game's primitive data, namely, the payoff functions and the game dynamics. The second approach assumes at the outset that we are dealing with a potential differential game. Then, under an additional hypothesis, the desired result follows. We illustrate our results with some examples.