STRICT DISSIPATIVITY FOR DISCRETE TIME DISCOUNTED OPTIMAL CONTROL PROBLEMS

. The paradigm of discounting future costs is a common feature of economic applications of optimal control. In this paper, we provide several results for such discounted optimal control aimed at replicating the now well-known results in the standard, undiscounted, setting whereby (strict) dissipa- tivity, turnpike properties, and near-optimality of closed-loop systems using model predictive control are essentially equivalent. To that end, we introduce a notion of discounted strict dissipativity and show that this implies various properties including the existence of available storage functions, required sup- ply functions, and robustness of optimal equilibria. Additionally, for discount factors suﬃciently close to one we demonstrate that strict dissipativity implies discounted strict dissipativity and that optimally controlled systems, derived from a discounted cost function, yield practically asymptotically stable equi- libria. Several examples are provided throughout.


1.
Introduction. Since its introduction in [43], dissipativity has become one of the most widely used concepts in mathematical systems theory, with deep connections to optimality, stability, and robustness. Recent research has established close connections between a particular form of dissipativity-namely strict dissipativity-and both the stability and near-optimality of closed-loop solutions of model predictive control schemes, see [2,23,21].
An important class of problems not covered by these recent results involve optimal control problems with discounted stage cost, wherein the performance objective incorporates, at each time k ∈ N 0 , a multiplicative term β k , where 0 < β < 1 is called the discount factor. Such problems arise in economics, where discounting is pervasive. In the Ramsey-Cass-Koopmans (RCK) model of neoclassical economic growth, for example, policies are chosen so as to maximize a social welfare function consisting of a discounted sum of aggregate economic utility [36,10,27,7,38,1]. In this framework, the discount factor reflects the weighting attached to the economic utility enjoyed by different generations [1].
One specific application of the RCK framework prominent in the economics of climate change is the DICE (Dynamic Integrated model of Climate and the Economy) integrated assessment model (IAM) of [32,33] (see also [26]). In DICE, trajectories of anthropogenic carbon dioxide (CO 2 ) emissions reflect an optimal tradeoff between reduced economic consumption today and economically harmful climate change in the future. In this context, the choice of discount factor plays a central role in determining the conclusions of the IAM-based optimal abatement analyses, e.g. [33] and [39]. The policy-relevance of DICE (see, e.g., [25]) therefore provides strong motivation for an optimal control framework which incorporates discounting. Moreover, model predictive control appears to be ideally suited for analyzing the behavior of this model under uncertainty, see [26,42], which motivates extending the study of near-optimality of model predictive control schemes to the discounted setting. We expect that the discounted version of strict dissipativity presented in this paper will provide an important building block for this study.
In addition to the above conceptual motivation for discounting in economics, discounted stage costs have been used in other contexts for essentially mathematical reasons, namely to ensure the integrability of a wide range of cost functions over an infinite horizon [3]. To the best of our knowledge, the connections between dissipativity and optimal control with discounted stage costs have not yet been considered in the literature, either in the discrete time setting treated in this paper or in continuous time.
In this paper, after providing the necessary background in Section 2, we introduce two notions of discounted strict dissipativity that appropriately incorporate the discount factor into the well-known dissipation inequality (Section 3). We also show that an important class of problems, namely those that employ a convex cost for an affine linear system with an equilibrium satisfying the necessary optimality conditions, are discounted strictly dissipative (Section 4). We then show that discounted strict dissipativity implies several desirable properties for discounted optimal control problems including the existence of (discounted) available storage and required supply functions, robustness of optimal equilibria (Sections 5 and 6), and that optimal solutions starting near an equilibrium stay near that equilibrium for a certain number of time steps (Section 7).
While the above-mentioned results all apply for any discount factor satisfying 0 < β < 1, one might reasonably expect that moving from an undiscounted problem, considered as a discount factor of β = 1, to a discount factor very close to one, would not destroy dissipativity. Indeed, in Section 8, we provide conditions under which strict dissipativity implies the existence of a discount factor (sufficiently close to one) such that the system is discounted strictly dissipative. Naturally, it is critically important that the optimal equilibria are in fact (practically) asymptotically stable for optimal controls arising from discounted optimal control problems. Indeed, Example 7.2 shows this need not be the case. Hence, in Section 9 we show that, again for discount factors sufficiently close to one, optimally controlled discounted strictly dissipative systems result in a (practically) asymptotically stable equilibrium. Finally, in Section 10 we provide some concluding remarks.
Preliminary versions of some of the results in this work were presented in [18] and [31]. Here, we combine and unify these results and extend them to the stronger property of strict (x, u)-dissipativity. Sections 6 and 9 contain entirely novel results. In particular, Theorem 6.2 extends the required supply concept for constructing storage functions to the discounted setting, and Theorem 9.3 shows the stabilizing property of discounted optimal control under assumptions on the undiscounted problem.
2.1. System class and notation. We consider discrete time nonlinear systems of the form for a map f : X × U → X, where X and U are normed spaces. We also write (1) briefly as x + = f (x, u). We impose the constraints (x, u) ∈ Y ⊆ X × U on the state x and the input u and define X := {x ∈ X | ∃u ∈ U : (x, u) ∈ Y} and U := {u ∈ U | ∃x ∈ X : (x, u) ∈ Y}. A control sequence u ∈ U N is called admissible for x 0 ∈ X if (x(k), u(k)) ∈ Y for k = 0, . . . , N − 1 and x(N ) ∈ X. In this case, the corresponding trajectory x(k) is also called admissible. The set of admissible control sequences is denoted by U N (x 0 ). Likewise, we define U ∞ (x 0 ) as the set of all control sequences u ∈ U ∞ with (x(k), u(k)) ∈ Y for all k ∈ N 0 . In order to keep the presentation technically simple, we assume that X is controlled invariant, i.e., that U ∞ (x 0 ) = ∅ for all x 0 ∈ X. We expect that our results remain true if one restricts the initial values under consideration to the viability kernel However, the technical details of this extension are beyond the scope of this paper. The trajectories of (1) are denoted by x u (k, x 0 ) or simply by x(k) if there is no ambiguity about x 0 and u. We will make use of the function classes K and K ∞ . Recall that α : R ≥0 → R ≥0 satisfies α ∈ K if it is continuous, zero at zero, and strictly increasing. Additionally, if α ∈ K is unbounded, then α ∈ K ∞ .

2.2.
A brief summary of undiscounted strict dissipativity. Our goal in this paper is to derive a notion of strict dissipativity with discounting and explore its connections to optimal control problems with discounted stage costs. To this end, we first recall the classical notion of strict dissipativity introduced by Willems in [43] in continuous time and by Byrnes and Lin in [9] in the discrete time setting of this paper. Recently, two different variants of this notion have become popular, a weaker one which only requires strictness (meaning a positive definite lower bound on a dissipation inequality) with respect to the state and a stronger one which requires strictness with respect to the state and the input. Most of the results in this paper will apply to both variants but for some we will need the stronger version. For the following definition, we recall that (x e , u e ) ∈ Y is an equilibrium of (1) if f (x e , u e ) = x e . Definition 2.1. Let (x e , u e ) be an equilibrium.
(a) The system (1) is called strictly x-dissipative at the equilibrium (x e , u e ) with supply rate s : Y → R if there exists a storage function λ : X → R bounded from below and a function α ∈ K ∞ such that (1) is called strictly (x, u)-dissipative at the equilibrium (x e , u e ) with supply rate s : Y → R if there exists a storage function λ : X → R bounded from below and a function α ∈ K ∞ such that Note that compared to Willems' original definition we replace the requirement λ ≥ 0 by boundedness of λ from below. This does not make a decisive difference in Definition 2.1 because one can always add a constant to λ without affecting the storage function property. However, for discounted problems, a different way of scaling the storage function than requiring λ ≥ 0 turns out to be advantageous, cf. Remark 3.3, below.
One of the most useful theorems in dissipativity theory states that strict dissipativity holds for a given supply rate s if and only if holds for each x 0 ∈ X, see [43] in continuous time and [9] in discrete time 1 , with γ(x, u) = α( x−x e ) for strict x-dissipativity and γ(x, u) = α( (x−x e , u−u e ) ) for strict (x, u)-dissipativity. The function λ defined in (4) is then a storage function, called the available storage. One of the goals of our discounted generalization of strict dissipativity will be to allow for a similar notion of available storage. The notion of dissipativity has a long history in systems and control theory, dating back to the work of Willems [43]. Dissipativity theory now underpins a wide range of application domains, including distributed model predictive control, plant-wide control of chemical processes, control of cyberphysical systems, power electronics and mechanical systems, and for establishing input-output stability of adaptive control systems, switched systems, and nonlinear H ∞ control systems; see for example [41,8,29] and the references therein.
By comparison, applications of strict dissipativity have appeared less frequently in the literature. Recent research, however, has established connections between strict dissipativity and the behavior of optimal trajectories via the so-called turnpike property. It is this connection that provides the motivation for this paper. Consider the optimal control problem min u∈U N (x0) with stage cost : Y → R and subject to (1). It is known that if the system is strictly dissipative with supply rate s(x, u) = (x, u)− (x e , u e ) and bounded storage function, and if an appropriate reachability condition on x e is satisfied, then most of the time the optimal trajectories stay in a neighborhood of the equilibrium x e . This property, known as the turnpike property, is due to the fact that the optimal trajectories of (5) exhibit a similar qualitative behaviour if is replaced bỹ Strict dissipativity then implies that˜ is a positive definite stage cost 2 with respect to x e at (x e , u e ), which means that it penalizes the deviation of x from x e and thus forces the optimal trajectory to stay near x e most of the time. For details we refer to [17,Theorem 5.6]; further results on the relation between strict dissipativity and turnpike properties can be found in [14,20,40]. The turnpike property, in turn, allows for making rigorous statements about the near optimality of closed loop solutions of model predictive control schemes [23]. The aforementioned connection between the turnpike property and behavior of closed-loop solutions of model predictive control schemes has recently been extended to discounted optimal control problems, i.e., to problems of the type see [24]. Herein, the number β ∈ (0, 1) is called the discount factor. With we denote the optimal value function of (7). We remark that in the discounted case it is often possible to directly consider the infinite horizon problem because discounting ensures the convergence of the infinite sum in (7) under much milder conditions than for the undiscounted problem (5). Working directly with the infinite horizon problem simplifies some of the considerations in this paper and using the results from [19] we can easily switch between these two formulations.
Since discounted optimal control problems play an important role particularly in economic applications, it is of great interest to adapt the results outlined above to the discounted case. From the results in [15] (see also [16,35] for related results), it follows that asymptotic stability (for the infinite horizon problem (7)) or the turnpike property (for the finite horizon counterpart of (7)), respectively, can under reasonable conditions be expected, provided the stage cost is positive definite (see also the results discussed in Section 9). Therefore, our "guideline" for deriving a discounted version of strict dissipativity will be that it should allow for a definition of a modified stage cost˜ analogous to (6), which is equivalent in the sense that the infinite horizon discounted optimal trajectories corresponding to and to˜ are identical.
3. Discounted strict dissipativity. Following the motivation just discussed, we propose the following definition of discounted strict dissipativity. The subsequent proposition shows that for the particular supply rate s(x, u) = (x, u) − (x e , u e ) it indeed yields an equivalent positive definite stage cost. Definition 3.1. Let (x e , u e ) be an equilibrium and β ∈ (0, 1) a discount factor.
(a) We say that the system (1) is discounted strictly x-dissipative at the equilibrium (x e , u e ) with supply rate s : Y → R if there exists a storage function λ : X → R bounded from below with λ(x e ) = 0 and a class K ∞ -function α such that the inequality (b) We say that the system (1) is discounted strictly (x, u)-dissipative at the equilibrium (x e , u e ) with supply rate s : Y → R if the same holds with the inequality We note that it is immediate that strict (x, u)-dissipativity implies strict xdissipativity, both in the discounted and in the non-discounted setting.
Proposition 3.2. Consider the discounted optimal control problem (7) with discount factor β ∈ (0, 1) and assume the system (1) is discounted strictly x-dissipative or discounted strictly (x, u)-dissipative with supply rate s(x, u) = (x, u) − (x e , u e ) and bounded storage function λ. Then the optimal trajectories of (7) coincide with those of the problem with stage cost˜ Proof. A straightforward calculation shows that Since λ is bounded and β ∈ (0, 1), the last limit exists and is equal to 0. Hence, the objectives differ only by expressions which are independent of u, from which the identity of the optimal trajectories immediately follows. The positive definiteness of follows from (8) or (9), respectively, and the fact that λ(x e ) = 0 implies˜ (x e , u e ) = 0.
Remark 3.3. The requirement that˜ (x e , u e ) = 0 is the reason for imposing λ(x e ) = 0 as a condition in Definition 3.1. Note that in the undiscounted case λ(x e ) = 0 can be assumed without loss of generality, since if λ is a storage function then λ + c is a storage function for all c ∈ R. In the discounted case, this invariance with respect to addition of constants no longer holds.
Remark 3.4. Boundedness of λ is typically a rather mild condition if the state constraint set X is bounded, but it may be restrictive if X is unbounded; for instance if λ is an affinely linear function as in the setting discussed in Theorem 4.2, below. In this case, other conditions ensuring lim k→∞ β k λ(x u (k)) = 0 could be imposed in Proposition 3.2. For instance, if λ is bounded on bounded sets, then one could assume boundedness of near optimal trajectories for both (7) and (10). Indeed, in the proof of Proposition 3.2 boundedness of λ is used in order to conclude that lim k→∞ β k λ(x u (k)) equals 0. However, if x u (k) is bounded and λ is bounded on bounded sets, then the same conclusion can be made. Boundedness of near optimal trajectories, in turn, can be achieved by penalizing large states in the cost functional, rendering unbounded trajectories very expensive. 4. The affine linear and convex case. In the non-discounted setting it is known that strict dissipativity holds for finite-dimensional affine dynamics f (x, u) = Ax + Bu + c with x ∈ R n , u ∈ R m ; i.e., A ∈ R n×n , B ∈ R m×n , and c ∈ R n , and strictly convex stage cost ; see [12] or [11,Proposition 4.3]. The proof of this fact relies on the necessary optimality conditions for an optimal equilibrium, where optimality of an equilibrium (x e , u e ) is defined as follows.
Definition 4.1. Consider the optimal control problem (7) with 0 < β < 1. Then The term "optimal equilibrium" is chosen because the definition implies that when starting at the equilibrium x e it is optimal to stay at x e for all future times, using the constant control u ≡ u e . This is true since, as easily verified, this constant trajectory yields the objective value (x e , u e )/(1 − β). The necessary optimality conditions for such an optimal equilibrium in the discounted case read cf. [4] or [6], where the n-dimensional row vector p e denotes the co-state (or Lagrange multiplier) at the optimal equilibrium. We note that these conditions differ from the necessary optimality conditions for an optimal equilibrium for non-discounted problems, which can be found, e.g., in [11] or [12]. As the case of a constant cost function ≡ const immediately shows, the existence of an optimal equilibrium in which (12)-(14) hold does not exclude the possibility that there exist non-stationary trajectories with the same value, unless further conditions on are imposed. The following theorem shows that for affine dynamics and strictly convex cost the necessary optimality conditions imply strict dissipativity also in the discounted case.
Theorem 4.2. Consider the optimal control problem (7) with β ∈ (0, 1), X ⊆ R n bounded, U ⊆ R m , affine dynamics f , and strictly convex stage cost . Assume there is an equilibrium (x e , u e ) ∈ Y and (p e ) T ∈ R n satisfying (12)- (14). Then the system is discounted strictly (x, u)-dissipative with supply rate s(x, u) = (x, u) − (x e , u e ) and storage function λ(x) = p e (x − x e ).
Proof. By definition and boundedness of X, λ satisfies λ(x e ) = 0 and is bounded from below. Strict convexity of and affine linearity of f together with the linearity of λ imply that˜ and (13), (14) it follows that the Jacobian D˜ (x e , u e ) equals 0, which by strict convexity of˜ implies that (x e , u e ) is the unique strict minimum of this function. This implies that˜ (x, u) >˜ (x e , u e ) = 0 for all (x, u) = (x e , u e ), which by exploiting strict convexity of˜ and boundedness of X implies the existence of α ∈ K ∞ with (9).
The following example illustrates that this theorem indeed provides a constructive way to check discounted strict dissipativity.
We consider a basic growth model in discrete time which goes back to [7]. The cost function and dynamics are given by Herein, Ax α is a production function with constants A > 0, 0 < α < 1, capital stock x and control variable u. The difference between output (given by the production function) and the capital stock at the next period (given by u) is consumption. The exact solution to this problem is known (see [37]) and is given by From this it is straightforward to check that the unique optimal equilibrium for this example is given by x e = 1/ α−1 √ βαA. Since f is linear and is strictly convex, Theorem 4.2 can be applied. In order to verify discounted strict (x, u)-dissipativity and to compute the storage function λ (and in order to show how to verify optimality of x e without using the knowledge of the exact solution), we solve equations (12)- (14). Here, the corresponding equations read Inserting (17) and u e = x e from (15) into (16) yields again x e = 1/ α−1 √ βαA. From this we obtain α − β as a storage function which is bounded on every bounded interval X ⊆ R >0 containing x e . 5. Available storage and robust optimality. Incorporating the discount factor in the available storage formula (4) is reasonably straightforward and using a dynamic programming argument it is relatively easy to see that the resulting function-if it assumes finite values-satisfies the discounted strict dissipativity inequalities (8) or (9); the details are provided in the proof of Theorem 5.4, below. However, in order to adapt the concept of the available storage to the discounted setting, we have to make sure that the appropriate modification of (4) leads to a storage function satisfying λ(x e ) = 0. In order to accomplish this, it is beneficial to replace the sup K in the non-discounted available storage formula (4) by an infinite sum. That is, we consider the discounted available storage defined by As we will see in the statement and proof of Theorem 5.4, the equality λ(x e ) = 0 is closely linked to the optimality of the equilibrium (x e , u e ). To clarify this relation we need to strengthen Definition 4.1 of an optimal equilibrium to the following notions of robust optimality.
(i) An equilibrium (x e , u e ) ∈ Y is called robustly optimal w.r.t. perturbations of in x, if there is σ ∈ K ∞ such that (x e , u e ) is optimal for the optimal control problem (7) with stage costˆ (x, u) : (ii) An equilibrium (x e , u e ) ∈ Y is called robustly optimal w.r.t. perturbations of in x and u, if there is σ ∈ K ∞ such that (x e , u e ) is optimal for the optimal control problem (7) with stage costˆ (x, u) : It is immediate that robust optimality of an equilibrium implies optimality of this equilibrium. Moreover, it is easy to see that an equilibrium is optimal if and only if the corresponding (constant) trajectory is an optimal trajectory. Note that, in contrast to the undiscounted case, an optimal equilibrium need not be the one which has the lowest cost (x e , u e ) of all feasible equilibria. In particular, it may be cheaper to transfer to an equilibrium with a higher cost and then stay there (see, e.g., the example in Section 8.4). The next two lemmas clarify certain relations of these optimality concepts to positive definiteness of and to strict dissipativity.
Lemma 5.2. If the stage cost of the optimal control problem (7) is positive definite 3 w.r.t. an equilibrium x e at (x e , u e ), then this equilibrium is optimal.
u e ) and bounded storage function λ implies that the equilibrium (x e , u e ) is robustly optimal w.r.t. perturbations of in x (respectively, w.r.t. perturbations in x and u).
Proof. We show the proof for strict x-dissipativity, the proof for strict (x, u)-dissipativity is similar. Let α be the K ∞ function from discounted strict x-dissipativity (8) and define σ ∈ K ∞ by σ := α/2. Then the cost functionˆ (x, u) : Hence, the optimal control problem with stage costˆ is discounted strictly xdissipative (with K ∞ function σ) and thus the equivalent problem (10) has a stage cost which is positive definite w.r.t. x e at (x e , u e ). Hence, by Lemma 5.2 (x e , u e ) is an optimal equilibrium. Since the optimal trajectories of (10) coincide with that of the original problem (i.e., of that with stage costˆ ) (x e , u e ) is also an optimal equilibrium for stage costˆ and thus a robustly optimal equilibrium for the stage cost w.r.t. perturbations of in x.
The following main theorem of this section now shows that-under appropriate boundedness assumptions-the discounted available storage (18) is a storage function in the sense of Definition 3.1 if and only if x e is robustly optimal.
Theorem 5.4. Let X be bounded and be bounded on Y. Let (x e , u e ) ∈ Y be an equilibrium of (1) and consider the discounted optimal control problem (7) with β ∈ (0, 1). Then discounted strict x-dissipativity (respectively, strict (x, u)dissipativity) with s(x, u) = (x, u) − (x e , u e ) and bounded storage function holds if and only if (x e , u e ) is robustly optimal w.r.t. perturbations of in x (respectively, w.r.t. perturbations in x and u). In this case, the available storage λ a from (18) is a bounded storage function.
"⇐" Again, we only prove the case of strict x-dissipativity; the proof for strict (x, u)-dissipativity is identical. Assume robust optimality w.r.t. perturbations of in x and let α = σ from Definition 5.1(i). From boundedness of X and it follows that λ = λ a from (18), i.e., is a bounded function in x 0 . We claim that λ is a discounted storage function for the system. From robust optimality of (x e , u e ) it follows that u(k) ≡ u e is optimal for x(0) = x e , implying λ(x e ) = 0. In order to prove the dissipation inequality (8), (19) is attained up to ε, i.e., such that Then for the control sequenceû = (u, u ε (0), u ε (1), . . .) we obtain xû(k, x) = x uε (k − 1, x + ) for all k ≥ 1 and This shows the desired strict dissipation inequality (8) for supply rate s(x, u) = (x, u) − (x e , u e ) since ε > 0 was arbitrary.
6. Required supply. In the case where every x ∈ X is reachable from the equilibrium x e , another way to construct undiscounted storage functions is via the required supply, cf., e.g., [43,Theorem 2] in continuous time and [28,Theorem 3.2] in discrete time. In this section we show how to adapt this construction to the discounted setting. We start with the following definition.
The following theorem shows that strict dissipativity holds if and only if λ r from (20) satisfies appropriate bounds, in which case it is a storage function. Theorem 6.2. Let (x e , u e ) ∈ Y be an equilibrium of (1) and consider the discounted optimal control problem (7) with β ∈ (0, 1). Assume that N ∈N U N x (x e ) = ∅ for all x ∈ X, i.e., that every x ∈ X is reachable from x e . Then discounted strict xdissipativity (respectively, strict (x, u)-dissipativity) with s(x, u) = (x, u)− (x e , u e ) holds if and only if the required supply λ r from (20) with the respective γ is bounded from below and satisfies λ r (x e ) = 0. In this case, the required supply λ r is a storage function.
Proof. As for the previous results, we give the proof for strict x-dissipativity noting that the proof for strict (x, u)-dissipativity proceeds completely analogously. Hence, in what follows we use γ(x, u) = α( x − x e ) in (20).
"⇒" Assume discounted strict x-dissipativity and let λ be the corresponding storage function. Then from the dissipativity inequality (8) we obtain s(x u (k, x e ), u(k)) − α( x u (k, x e ) − x e ) ≥ βλ(x u (k + 1, x e )) − λ(x u (k, x e )). (21) This implies because λ(x e ) = 0. Since λ is bounded from below, it follows that λ r is bounded from below and λ r (x e ) ≥ 0. As, moreover, u ≡ u e lies in U N x e (x e ) for arbitrary N ∈ N and yields x u (k, x e ) = x e , we obtain Together this implies λ r (x e ) = 0.
"⇐" We show the assertion by proving that λ r is a storage function. To this end, the inequality (8) needs to be shown. In order prove (8), consider x + = f (x, u) with x ∈ X and (x, u) ∈ Y. Then, for N ≥ 2 and any control sequenceû ∈ U N −1 x (x e ) the control sequenceû + defined bŷ where in the third and fourth step we used the definition ofû + and in the second last step we have made the substitution N := N − 1. Multiplying this equation by β we obtain the desired inequality (8).
Remark 6.3. From (22) one immediately sees that λ r is greater than or equal to any other storage function λ. Similarly, inserting the inequality (21) into the definition of the available storage (18) and using the boundedness from below of λ, one obtains that λ a is less than or equal than any other storage function λ. Moreover, one easily checks that any convex combination µλ 1 + (1 − µ)λ 2 , µ ∈ [0, 1] of two storage functions λ 1 and λ 2 is again a storage function. Hence, as in the undiscounted case (cf. [43,Theorem 3]), any convex combination µλ a + (1 − µ)λ r of the available storage and the required supply is a storage function.
7. Continuity of optimal trajectories near the equilibrium. It was shown in [17,Lemma 6.3] that in the non-discounted setting, strict dissipativity (along with other assumptions) implies that optimal trajectories starting near x e stay near x e for a certain number of time steps. In this section we show that the same is true for our proposed discounted notion of strict dissipativity.
Theorem 7.1. Consider the discounted optimal control problem (7) with β ∈ (0, 1) and assume system (1) is discounted strictly x-dissipative with s(x, u) = (x, u) − (x e , u e ) and bounded storage function λ. Assume, moreover, that V ∞ and λ are continuous at the equilibrium x e . Then for each K ∈ N there exists η K ∈ K ∞ such that the optimal trajectories x * satisfy for all k = 0, . . . , K, where x 0 = x * (0). In case strict (x, u)-dissipativity holds, in addition the corresponding optimal control sequences u * satisfy for all k = 0, . . . , K.
Proof. We start by showing (23). It is sufficient to show the property for the equivalent optimal control problem (10). Since V ∞ and λ are continuous at x e , it follows from (11) that V ∞ is also continuous at x e . Since positive definiteness of˜ implies V ∞ (x e ) = 0, by continuity there is ρ ∈ K ∞ with Given K ∈ N, we claim that the assertion holds for η K (r) : Thus, since˜ ≥ 0 we obtain contradicting (25).
Proceeding as above, this leads to a contradiction of inequality (25).
The following example shows that the statement of Theorem 7.1 may not hold for K = ∞, i.e., that discounted strict dissipativity does not necessarily imply stability of the optimal equilibrium x e . Example 7.2. Example 1 in [34] shows that the discounted linear quadratic optimal control problem with x, u ∈ R does not yield an optimal stabilizing feedback controller for discount factors β ≤ 1/3. Indeed, the discounted optimal control can be obtained by solving the discrete time algebraic Riccati equation with √ βA and √ βB in place of A and B and, for β = 0.3, the resulting closed-loop system is x + ≈ 1.0799x.
Since is bounded from below by α( (x − x e , u − u e ) ) with α(r) = r 2 and x e = u e = 0, it is straightforward to see that the system is (discounted) strictly (x, u)-dissipative at (x e , u e ) = (0, 0) for all β ∈ (0, 1] with supply rate s(x, u) = (x, u) − (x e , u e ) and bounded storage function λ ≡ 0. Consequently, Theorem 7.1 states that for every K ∈ N we can find an appropriate η K ∈ K ∞ to satisfy (23). However, since the origin is clearly unstable for β = 0.3, we see that Theorem 7.1 cannot hold for K = ∞. We note that the instability of the closed loop is consistent with the result in [15], which only ensures asymptotic stability for β sufficiently close to 1. We address (practical) asymptotic stability of optimally controlled strictly dissipative systems in Section 9 below. Remark 7.3. In the linear quadratic and unconstrained setting of Example 7.2, the assertion of Theorem 7.1 could also be concluded from the Lipschitz continuity of the right-hand side of the optimally controlled closed loop system. However, in general -and in particular in the presence of nonlinearities and constraints -optimal controls and the resulting optimal trajectories do not need to depend continuously on the initial value, which makes the assertion of Theorem 7.1 nontrivial. 8. Dissipativity and discounted dissipativity. In this section, we show under what conditions strict dissipativity implies discounted strict dissipativity for discount factors β sufficiently close to one. Contrary to the results in the previous sections, the results in this section require strict dissipativity with respect to x and u, i.e., strict (x, u)-dissipativity. Since β ∈ (0, 1] is a varying number in this and in the following section, rather than a fixed parameter as before, from now on we explicitly reflect the dependence of all quantities on β in our notation, with β = 1 denoting the undiscounted case. For instance, we write x e (β) instead of x e , λ(x, β) instead of λ(x), and˜ (x, u, β) instead of˜ (x, u), In order to simplify the notation, for β = 1 we write x e instead of x e (1). 8.1. Nonlinear programming. We first briefly recall some results from nonlinear programming. Namely, consider a constrained optimization problem of the form min y ϕ(y) s.t. h(y) = 0 and g(y) ≤ 0, where y ∈ R ny and the functions ϕ : R ny → R, h : R ny → R n h and g : R ny → R ng are twice continuously differentiable. Denote the set of active inequality constraints at a feasible point y by A(y) := {1 ≤ j ≤ n g : g j (y) = 0}.
A feasible point y is regular if, for 1 ≤ i ≤ n h and j ∈ A(y), ∇ y h i (y) and ∇ y g j (y) are linearly independent. If a point y * is regular and a local minimizer of the above optimization problem, then there exist (unique) Lagrange multiplier vectors ν ∈ R n h and µ ∈ R ng ≥0 such that ∇ y ϕ(y * ) + ν T ∇ y h(y * ) + µ T ∇ y g(y * ) = 0 with µ j = 0 for all j / ∈ A(y * ), see, e.g., [5,Proposition 3.3.1]. Furthermore, in the following we will make use of the second order sufficiency conditions [5, Proposition 3.3.2], i.e., (i) w T ∇ 2 y (ϕ(y * ) + ν T h(y * ) + µ T g(y * ))w > 0 for all w = 0 with ∇ y h(y * )w = 0 and ∇ y g j (y * )w = 0 for all j ∈ A(y * ), and 4 (ii) µ j > 0 for all j ∈ A(y * ).

8.2.
Optimal control related supply rates. In this section we consider the optimal control problem (7) and a supply rate induced by the running cost via s(x, u) = (x, u) − (x e (β), u e (β)). We assume that the state and input constraint set Y is defined in terms of inequality constraints, i.e., Clearly, if system (1) is strictly (x, u)-dissipative with supply rate s(x, u) = (x, u)− (x e , u e ) for some equilibrium (x e , u e ), then this equilibrium is the unique minimizer of problem (27). Now consider the undiscounted modified cost functioñ (x, u, 1) := (x, u) − (x e , u e ) + λ(x) − λ(f (x, u)). (28) If the system is strictly (x, u)-dissipative with respect to the supply rate s(x, u) = (x, u) − (x e , u e ), from (3) it follows that˜ (x, u, 1) ≥ α( (x − x e , u − u e ) ) for all (x, u) ∈ Y. This means that 0 =˜ (x e , u e ) = min i.e., (x e , u e ) is the unique minimizer of˜ on the set Y. We now impose the following assumption.
Assumption 8.1. The following hold: (i) The functions f , , and g, are twice continuously differentiable and Y is bounded. (ii) The point (x e , u e ) is a regular point of problem (27) and satisfies the second order sufficiency conditions. (iii) The undiscounted problem is strictly (x, u)-dissipative with respect to the supply rate s(x, u) = (x, u) − (x e , u e ). Furthermore, the storage function λ is twice continuously differentiable and (x e , u e ) satisfies the second order sufficiency conditions for problem (29).
We are now in a position to prove the following result. In its proof, we construct storage functions λ(·, β) for the discounted problems from the undiscounted storage function λ from Assumption 8.1, using necessary optimality conditions of a suitable optimization problem and the implicit function theorem. Theorem 8.2. Let Assumption 8.1 be satisfied. Then there existsβ < 1 such that for all β ∈ (β, 1), there exists an equilibrium (x e (β), u e (β)) such that the system is discounted strictly (x, u)-dissipative with respect to the supply rate s(x, u) = (x, u)− (x e (β), u e (β)), i.e., there exist a storage function λ(x, β) with λ(x e (β), β) = 0 and σ ∈ K ∞ such that the functioñ Proof. The proof of Theorem 8.2 exploits the fact that for the specific supply rate considered here, (discounted) strict (x, u)-dissipativity can be reformulated as the equilibrium (x e , u e ) being the unique minimizer to some optimization problem (compare the discussion around (29)). In particular, we first determine a suitable equilibrium (x e (β), u e (β)) and a storage function candidate λ(x, β) (see (34) below), and then show that for β sufficiently close to one, (x e (β), u e (β)) is indeed the unique minimizer to a suitably defined optimization problem (see (35) below), resulting in discounted strict (x, u)-dissipativity.
Hence for β sufficiently close to one, (x e (β), u e (β)) is a strict local minimizer of˜ (see, e.g. [5,Proposition 3.3.2]). But then, since (x e , u e ) was a global minimizer of˜ on the compact set Y, by continuity β can be chosen close enough to one such that also (x e (β), u e (β)) is a global minimizer of˜ on Y, i.e., there exists σ ∈ K ∞ such that˜ (x, u, β) ≥ σ( (x − x e (β), u − u e (β)) ) for all (x, u) ∈ Y. Together with the fact that λ(x e (β), β) = 0 as established above, this implies that the system is discounted strictly (x, u)-dissipative with respect to the supply rate s(x, u) = (x, u) − (x e (β), u e (β)), which concludes the proof of Theorem 8.2.

Remark 8.3.
In [30,Theorem 5], robustness of (undiscounted) dissipativity with respect to parameter variations in the constraint set Y was studied. Both the above proof of Theorem 8.2 and the proof of [30,Theorem 5] use ideas from the context of nonlinear programming. However, in [30,Theorem 5] one could directly apply sensitivity results, i.e., results on the parametric dependence of the solution of the KKT conditions for optimization problems. In the above proof, this was not the case, since for β = 1, the set of equations (31) do not correspond to the KKT conditions of some associated optimization problem, but only for β = 1. Still, the proof idea is similar in the sense that both approaches eventually rely on the application of a form of the implicit function theorem.

Remark 8.4.
It remains an open question whether or not Theorem 8.2 holds under the assumption of strict x-dissipativity. However, showing this is likely to require a different proof technique since it is strict (x, u)-dissipativity that is used to guarantee that J of (32), and in particular H of (33), is nonsingular. 8.3. Extension to general supply rates. We now briefly discuss how the preceding results can be extended to general supply rates. Namely, given an equilibrium (x e , u e ), suppose that system (1) is strictly (x, u)-dissipative with respect to some supply rate s : Y → R. We can now distinguish two cases. First, if the minimum of the problem min x,u is (strictly) positive and f , s, and λ are continuous, then also min g(x,u)≤0 s(x, u) + λ(x) − βλ(f (x, u)) > 0 for β close enough to one (due to compactness of Y). Hence system (1) is also discounted strictly (x, u)-dissipative in this case. Second, if the minimum of the problem (36) is zero, by strict (x, u)-dissipativity it follows that the minimizer of problem (36) is the point (x e , u e ), which is also the minimizer to the problem min x,u s(x, u) s.t. x = f (x, u) and g(x, u) ≤ 0.
In this case, discounted strict (x, u)-dissipativity can be established analogously to the proof of Theorem 8.2, using the following modified assumption.
Assumption 8.5. The following hold: 1. The functions f , s, g, and λ are twice continuously differentiable. We then arrive at the following corollary.
Corollary 8.6. Suppose that system (1) is strictly (x, u)-dissipative with respect to the supply rate s and that either (i) the minimum of the problem (36) is positive and f , s, and λ are continuous, or (ii) Assumption 8.5 holds. Then there existŝ β < 1 such that for all β ∈ (β, 1), the system is discounted strictly (x, u)-dissipative with respect to the supply rate s.

8.4.
Example. We illustrate the preceding results with a simple example. Consider the system with stage cost (x, u) = (x + 1) 2 + (u − 1) 2 and state and input constraint set Y given by (26) where The optimal equilibrium is (x e , u e ) = (0, 0) with associated stage cost (x e , u e ) = 2. One can show that the system (38) is strictly (x, u)-dissipative with respect to the supply rate s(x, u) = (x, u)− (x e , u e ) and storage function λ(x) = −2x. The point (x e , u e ) = (0, 0) is a regular point of problem (27) and satisfies the second order sufficiency conditions for problems (27) and (29) (note that both and˜ are quadratic and none of the constraints specified by g are active at (x e , u e ) = (0, 0)). Hence Assumption 8.1 is satisfied and we can apply Theorem 8.2 to conclude that there exists an equilibrium (x e (β), u e (β)) such that the system (38) is also discounted strictly (x, u)-dissipative with respect to the supply rate s(x, u) = (x, u) − (x e (β), u e (β)) for discount factors β close enough to one. Indeed, as shown in the proof of Theorem 8.2, the optimal equilibrium (x e (β), u e (β)) varies continuously in β and is given by The corresponding storage function λ is given by which is in accordance with (34). Both the optimal equilibria and the storage function can be computed with the approach presented in Section 4 since the system is linear with strictly convex stage cost . Additionally, appealing to Theorem 4.2, the system is discounted strictly (x, u)-dissipative for all β ∈ (0, 1). In order to illustrate the comment following Definition 5.1, we compute J ∞ (x 0 , u) from (7) for β = 1/2 and two initial conditions. In this case, the optimal equilibrium (39) is (1/3, 1/3). First, consider the costs associated with starting and remaining at either the equilibrium (0, 0) or (1/3, 1/3); i.e., In other words, starting at x 0 = 0 and staying there is cheaper than starting at x 0 = 1/3 and staying there. However, consider starting at x 0 = 0 and immediately moving to x e (1/2) = 1/3; i.e., take u(k) = 1/3 for all k ∈ N 0 . Then J ∞ (0, 1/3) = 33 9 ≈ 3.6 and, hence, from x 0 = 0, it is cheaper to move to x e (1/2) and remain there than it is to stay at the equilibrium x = 0 even though the running cost satisfies (0, 0) = 2 < 20 9 = (1/3, 1/3). Finally, consider starting at x 0 = 1/3 and moving immediately to x = 0, in which case J ∞ (1/3, 0) = 43 9 ≈ 4.7. Returning to the general example (i.e., without fixing β), consider the additional constraint g ad (x, u) = x + u ≤ 0, i.e., the state and input constraint set Y of (26) is determined by Since (x e , u e ) = (0, 0) is still a feasible point (however, now on the boundary of the set Y), clearly the system (38) is still strictly (x, u)-dissipative with respect to the supply rate s(x, u) = (x, u) − (x e , u e ) and storage function λ(x) = −2x as above. On the other hand, for any β ∈ (0, 1), the equilibrium (x e (β), u e (β)) given by (39) is not feasible. Indeed, for any β ∈ (0, 1), the system is not discounted strictly (x, u)-dissipative. According to Theorem 5.4, this can be proven by showing that none of the feasible equilibria is optimal, i.e., for all feasible equilibria (x e , u e ) we have V ∞ (x e ) < (x e , u e )/(1 + β). Namely, given any feasible equilibrium −2 ≤ x e ≤ 0 with u e = x e , consider the input and corresponding state sequences u = (δ, −δ, δ, . . . ) and x = (x e , δ, −δ, δ, . . . ) for some δ > 0. Straightforward (but cumbersome) computations show that J ∞ (x e , u ) < (x e , u e )/(1 + β) if δ < 2(1 − β) 2 /(1 + β) 2 . Hence for each 0 < β < 1, there exists some δ > 0 such that V ∞ (x e ) ≤ J ∞ (x e , u ) < (x e , u e )/(1 + β). This contradicts optimality of each equilibrium (x e , u e ) for each β ∈ (0, 1) and thus implies that the system is not discounted strictly (x, u)-dissipative, since strict (x, u)-disipativity would require the existence of an optimal equilibrium. The reason why Theorem 8.2 fails is that the second order sufficiency conditions for problems (27) and (29) are not satisfied (while the rest of Assumption 8.1 holds). Namely, the strict complementarity condition is not satisfied since the constraint x + u ≤ 0 is active at (x e , u e ) = (0, 0), but the corresponding Lagrange multiplier is zero because the equilibrium remains optimal when removing this constraint. 9. Practical asymptotic stability of discounted optimal trajectories. Under suitable conditions, discounted optimal trajectories are asymptotically stable at an optimal equilibrium, see [15,16,35]. More precisely, if we write the optimally controlled system in feedback form with optimal feedback law 5 µ : X → U, then the closed loop system (41) has an optimal equilibrium with certain stability properties. Theorem 9.2, below, provides conditions for such a result. For its formulation and the subsequent considerations we need the following practical asymptotic stability definition. Here KL denotes the set of all continuous functions µ : [0, ∞) 2 → [0, ∞) such that r → µ(t, r) is in K for all t ≥ 0 and t → µ(t, r) is strictly decreasing to 0 for all r > 0.
Definition 9.1. For two numbers ∆ > δ > 0, an equilibrium (x e , u e ) is called (δ, ∆)-practically asymptotically stable, if there exists a function η ∈ KL such that all closed-loop trajectories x(k) with x(0) − x e ≤ ∆ satisfy the inequality for all k ∈ N 0 .
The following is [15,Corollary 4.3], which will be used in the proof of Theorem 9.3 below.
Next we formulate the main result of this section, which states that undiscounted strict dissipativity implies semiglobal practical asymptotic stability for the discounted optimal closed loop system with β close to 1, provided the optimal equilibrium does not lie at the boundary of the constraint set. While at first glance the next theorem appears similar to Theorem 4.4 in [15], there is a decisive difference: whereas in [15] assumptions on the discounted optimal control problem are made, here we make assumptions on the undiscounted problem, but still derive stability properties for the discounted optimal solutions. In the theorem and its proof, β is again a varying parameter, hence we explicitly denote β as a function argument. As in the last section, to simplify notation, we write x e and u e instead of x e (1) and u e (1), respectively. Theorem 9.3. Consider an optimal control problem satisfying Assumption 8.1 with optimal equilibrium (x e , u e ) ∈ int Y. Assume that there existsα ∈ K ∞ with V (x, 1) ≤α( x−x e ) for all x ∈ X. Then for all ∆ > δ > 0 there isβ < 1 such that for each β ∈ (β, 1) the optimal equilibrium x e (β) is (δ, ∆)-practically asymptotically stable for the discounted optimal closed-loop system.
Proof. Under the assumptions of this theorem, it follows that x e is a globally asymptotically stable equilibrium of the undiscounted optimal closed-loop system. This follows, e.g., by applying [22,Theorem 4.8] to the problem with modified stage cost˜ from (28), observing that V (x, 1) ≥ α( x − x e ) holds for α from the strict dissipativity assumption. This theorem implies that there exists µ ∈ KL such that the undiscounted optimal trajectory x * satisfies Moreover, the strict (x, u)-dissipativity and non-negativity of˜ imply that for all k ∈ N. Hence, for any Θ > 0 all undiscounted optimal trajectories with V (x * (0), 1) ≤ Θ are uniformly bounded. By fixing an arbitrary Θ > 0 and considering only those initial conditions satisfying V (x * (0), 1) ≤ Θ, for the following considerations, we may thus without loss of generality assume that Y is bounded. Theorem 8.2 now implies the existence ofβ < 1 such that the discounted problem is strictly (x, u)-dissipative for all β ∈ (β, 1). We claim that from this it follows (for all β sufficiently close to 1) that the assumptions of Theorem 9.2 hold with α 2 = (C 1 + 1)α for C 1 > 0 specified below. Since Y × [β, 1] is bounded and˜ is continuous, for any Θ > 0 we obtain a bound M Θ with˜ (x * (k), u * (k), β) ≤ M Θ for all k ∈ N.
For the subsequent estimates we use the fact that the definitions of the rotated cost functions imply the existence of constants C 1 > 0, C 2 > 0 and β 1 < 1 such that the inequality˜ holds for all β ∈ [β 1 , 1] and all x, u ∈ Y.
We subsequently demonstrated that discounted strict dissipativity is equivalent to a form of robust optimality (Theorem 5.4) and that discounted strict dissipativity implies a certain continuity of trajectories near an optimal equilibrium (Theorem 7.1). These results are a prerequisite to demonstrating an equivalence between discounted strict dissipativity, turnpike properties, and near optimality of closed loop solutions of model predictive control schemes based on optimal control problems with discounted stage costs.
Under certain regularity conditions commonly used in the context of nonlinear programming, we demonstrated that strict dissipativity implies discounted strict dissipativity for discount factors close enough to one. Hence, statements in economic model predictive control about steady-state optimality, turnpike properties, and closed-loop performance and convergence, are preserved under sufficiently mild discounting. We additionally showed that, under standard assumptions solely on the undiscounted problem, optimal controls computed from a discounted stage cost yield a (practically) asymptotically stable equilibrium in closed-loop, again for sufficiently mild discounting. Importantly, our motivating applications in economics usually have a discount factor of 0.95 or higher [13,26,33,42].