# American Institute of Mathematical Sciences

October  2019, 6(4): 337-366. doi: 10.3934/jdg.2019022

## Markovian strategies for piecewise deterministic differential games with continuous and impulse controls

 Department of Mathematics, Statistics and Computer Science, Purdue University Northwest, Hammond, IN 46323-2094, USA

Received  September 2019 Revised  September 2019 Published  October 2019

This paper is concerned with the Markovian feedback strategies of piecewise deterministic differential games and their applications to business and management decision-making problems that involve multiple agents and continuous and impulse controls. For a class of piecewise deterministic differential games in finite or infinite horizons we formulate conditions for the value functions in the form of quasi-variational inequalities, prove a verification theorem, and derive a criterion for the Markovian regime change in certain case. These results are applied to a technology adoption problem that involves multiple companies engaged in extraction of an exhaustible resource with different technologies. Using the model proposed by Long et al in [16], we show the existence of a pure Markovian strategy and develop an algorithm for computing the solutions.

Citation: Weihua Ruan. Markovian strategies for piecewise deterministic differential games with continuous and impulse controls. Journal of Dynamics & Games, 2019, 6 (4) : 337-366. doi: 10.3934/jdg.2019022
##### References:
 [1] R. Amit, Petroleum reservoir exploitation: Switching from primary to secondary recovery, Operations Research, 34 (1986), 534-549.  doi: 10.1287/opre.34.4.534.  Google Scholar [2] T. Başar and A. Haurie, Feedback equilibria in differential games with structural and modal uncertainties, Advances in Large Scale Systems, New York, 1 (1984), 163–201.  Google Scholar [3] R. Boucekkine, A. Pommeret and F. Prieur, Optimal regime switching and threshold effects, ournal of Economic Dynamics and Control, 37 (2013), 2979-2997.  doi: 10.1016/j.jedc.2013.08.008.  Google Scholar [4] R. Boucekkine, F. Prieur and K. Puzon, On the timing of political regime changes in resource-dependent economies, European Economic Review, 85 (2016), 188-207.  doi: 10.1016/j.euroecorev.2016.02.016.  Google Scholar [5] R. Boucekkine, C. Saglam and T. Vallée., Technology adoption under embodiment: A two-stage optimal control approach, Macroeconomic Dynamics, 8 (2004), 250-271.  doi: 10.1017/S1365100503030062.  Google Scholar [6] M. Breton and A. Haurie, Two-layer piecewise deterministic games, In Proceedings of the 28th IEEE Conference on Decision and Control, New York, 1989,198–200. doi: 10.1109/CDC.1989.70102.  Google Scholar [7] E. J. Dockner, S. J$\phi$rgensen, N. V. Long and G. Sorger, Differential Games in Economics and Management Science, Cambridge, Cambridge University Press, 2000. doi: 10.1017/CBO9780511805127.  Google Scholar [8] T. L. Friesz, Dynamic Optimization and Differential Games, Springer, New York Dordrecht Heidelberg London, 2010. Google Scholar [9] D. Fudenberg and J. Tirole, Preemption and rent equalization in the adoption of new technology, Review of Economic Studies, 52 (1985), 383-401.  doi: 10.2307/2297660.  Google Scholar [10] M. Ghosh and S. Marcus, Stochastic differential games with multiple modes, Stochastic Analysis and Applications, 16 (1998), 91-105.  doi: 10.1080/07362999808809519.  Google Scholar [11] J. Haunschmied, V. M. Veliov and S. Wrzaczek, editors, Dynamic Games in Economics, volume 16 of Dynamic Modeling and Econometrics in Economics and Finance, Springer-Verlag, Berlin Heidelberg, 2014. doi: 10.1007/978-3-642-54248-0.  Google Scholar [12] A. Haurie, Piecewise deterministic and piecewise diffusion differential games with modal uncertainties, Decision Processes in Economics, Springer, Berlin, 353 (1991), 107–120. doi: 10.1007/978-3-642-45686-2_10.  Google Scholar [13] A. Haurie, From repeated to differential games: How time and uncertainty pervade the theory of games, Frontiers of Game Theory, MIT Press, Cambridge, MA, 1993,165–194.  Google Scholar [14] A. Haurie, S. Muto, L. A. Petrosjan and T. E. S. Raghavan, editors, Advances in Dynamic Games–Applications to Economics, Management Science, Engineering, and Environmental Management, volume 8 of Annals of the International Society of Dynamic Games, Birkhäuser, Boston Basel Berlin, 2006. doi: 10.1007/0-8176-4501-2.  Google Scholar [15] S. J${\phi}$rgensen and G. Zaccour, Developments in differential game theory and numerical methods: Economic and management applications, Computational Management Science, 4 (2007), 159-181.  doi: 10.1007/s10287-006-0032-x.  Google Scholar [16] N. V. Long, F. Prieur, M. Tidball and K. Puzon, Piecewise closed-loop equilibria in differential games with regime switching strategies, J. Econom. Dynam. Control, 76 (2017), 264-284.  doi: 10.1016/j.jedc.2017.01.008.  Google Scholar [17] M. Makris, Necessary conditions for infinite-horizon discounted two-stage optimal control problems, J. Econom. Dynam. Control, 25 (2001), 1935-1950.  doi: 10.1016/S0165-1889(00)00009-9.  Google Scholar [18] F. E. Ouardighi and K. Kogan, editors, Models and Methods in Economics and Management Science–Essays in Honor of Charles S. Tapiero, volume 198 of International Series in Operations Research and Management Science, Springer, Cham Heidelberg New York Dordrecht London, 2014. doi: 10.1007/978-3-319-00669-7.  Google Scholar [19] J. F. Reinganum, On the diffusion of new technology: A game theoretic approach, Review of Economic Studies, 48 (1981), 395-405.  doi: 10.2307/2297153.  Google Scholar [20] C. Saglam, Optimal pattern of technology adoptions under embodiment: A multi-stage optimal control approach, Optim. Control Appl. Meth., 32 (2011), 574-586.  doi: 10.1002/oca.960.  Google Scholar [21] K. Tomiyama, Two-stage optimal control problems and optimality conditions, ournal of Economic Dynamics and Control, 9 (1985), 317-337.  doi: 10.1016/0165-1889(85)90010-7.  Google Scholar [22] K. Tomiyama and R. Rossana, Two-stage optimal control problems with an explicit switch point dependence–optimality criteria and an example of delivery lags and investment, J. Econom. Dynam. Control, 13 (1989), 319-337.  doi: 10.1016/0165-1889(89)90027-4.  Google Scholar [23] S. Valente, Endogenous growth, backstop technology adoption, and optimal jumps, Macroeconomic Dynamics, 15 (2011), 293-325.   Google Scholar

show all references

##### References:
 [1] R. Amit, Petroleum reservoir exploitation: Switching from primary to secondary recovery, Operations Research, 34 (1986), 534-549.  doi: 10.1287/opre.34.4.534.  Google Scholar [2] T. Başar and A. Haurie, Feedback equilibria in differential games with structural and modal uncertainties, Advances in Large Scale Systems, New York, 1 (1984), 163–201.  Google Scholar [3] R. Boucekkine, A. Pommeret and F. Prieur, Optimal regime switching and threshold effects, ournal of Economic Dynamics and Control, 37 (2013), 2979-2997.  doi: 10.1016/j.jedc.2013.08.008.  Google Scholar [4] R. Boucekkine, F. Prieur and K. Puzon, On the timing of political regime changes in resource-dependent economies, European Economic Review, 85 (2016), 188-207.  doi: 10.1016/j.euroecorev.2016.02.016.  Google Scholar [5] R. Boucekkine, C. Saglam and T. Vallée., Technology adoption under embodiment: A two-stage optimal control approach, Macroeconomic Dynamics, 8 (2004), 250-271.  doi: 10.1017/S1365100503030062.  Google Scholar [6] M. Breton and A. Haurie, Two-layer piecewise deterministic games, In Proceedings of the 28th IEEE Conference on Decision and Control, New York, 1989,198–200. doi: 10.1109/CDC.1989.70102.  Google Scholar [7] E. J. Dockner, S. J$\phi$rgensen, N. V. Long and G. Sorger, Differential Games in Economics and Management Science, Cambridge, Cambridge University Press, 2000. doi: 10.1017/CBO9780511805127.  Google Scholar [8] T. L. Friesz, Dynamic Optimization and Differential Games, Springer, New York Dordrecht Heidelberg London, 2010. Google Scholar [9] D. Fudenberg and J. Tirole, Preemption and rent equalization in the adoption of new technology, Review of Economic Studies, 52 (1985), 383-401.  doi: 10.2307/2297660.  Google Scholar [10] M. Ghosh and S. Marcus, Stochastic differential games with multiple modes, Stochastic Analysis and Applications, 16 (1998), 91-105.  doi: 10.1080/07362999808809519.  Google Scholar [11] J. Haunschmied, V. M. Veliov and S. Wrzaczek, editors, Dynamic Games in Economics, volume 16 of Dynamic Modeling and Econometrics in Economics and Finance, Springer-Verlag, Berlin Heidelberg, 2014. doi: 10.1007/978-3-642-54248-0.  Google Scholar [12] A. Haurie, Piecewise deterministic and piecewise diffusion differential games with modal uncertainties, Decision Processes in Economics, Springer, Berlin, 353 (1991), 107–120. doi: 10.1007/978-3-642-45686-2_10.  Google Scholar [13] A. Haurie, From repeated to differential games: How time and uncertainty pervade the theory of games, Frontiers of Game Theory, MIT Press, Cambridge, MA, 1993,165–194.  Google Scholar [14] A. Haurie, S. Muto, L. A. Petrosjan and T. E. S. Raghavan, editors, Advances in Dynamic Games–Applications to Economics, Management Science, Engineering, and Environmental Management, volume 8 of Annals of the International Society of Dynamic Games, Birkhäuser, Boston Basel Berlin, 2006. doi: 10.1007/0-8176-4501-2.  Google Scholar [15] S. J${\phi}$rgensen and G. Zaccour, Developments in differential game theory and numerical methods: Economic and management applications, Computational Management Science, 4 (2007), 159-181.  doi: 10.1007/s10287-006-0032-x.  Google Scholar [16] N. V. Long, F. Prieur, M. Tidball and K. Puzon, Piecewise closed-loop equilibria in differential games with regime switching strategies, J. Econom. Dynam. Control, 76 (2017), 264-284.  doi: 10.1016/j.jedc.2017.01.008.  Google Scholar [17] M. Makris, Necessary conditions for infinite-horizon discounted two-stage optimal control problems, J. Econom. Dynam. Control, 25 (2001), 1935-1950.  doi: 10.1016/S0165-1889(00)00009-9.  Google Scholar [18] F. E. Ouardighi and K. Kogan, editors, Models and Methods in Economics and Management Science–Essays in Honor of Charles S. Tapiero, volume 198 of International Series in Operations Research and Management Science, Springer, Cham Heidelberg New York Dordrecht London, 2014. doi: 10.1007/978-3-319-00669-7.  Google Scholar [19] J. F. Reinganum, On the diffusion of new technology: A game theoretic approach, Review of Economic Studies, 48 (1981), 395-405.  doi: 10.2307/2297153.  Google Scholar [20] C. Saglam, Optimal pattern of technology adoptions under embodiment: A multi-stage optimal control approach, Optim. Control Appl. Meth., 32 (2011), 574-586.  doi: 10.1002/oca.960.  Google Scholar [21] K. Tomiyama, Two-stage optimal control problems and optimality conditions, ournal of Economic Dynamics and Control, 9 (1985), 317-337.  doi: 10.1016/0165-1889(85)90010-7.  Google Scholar [22] K. Tomiyama and R. Rossana, Two-stage optimal control problems with an explicit switch point dependence–optimality criteria and an example of delivery lags and investment, J. Econom. Dynam. Control, 13 (1989), 319-337.  doi: 10.1016/0165-1889(89)90027-4.  Google Scholar [23] S. Valente, Endogenous growth, backstop technology adoption, and optimal jumps, Macroeconomic Dynamics, 15 (2011), 293-325.   Google Scholar
Possible strategies with parameter values given by (3.15)
Extraction and consumption rates using parameters in (3.15)
Possible strategies with $\gamma^2 = 1.6$ and other parameter values given by (3.15)
Extraction and consumption rates when $\gamma ^{2}$ is changed to 1.6
Extraction and consumption rates. The blue and red curves on the right are the consumption rates of Companies 1 and 2, respectively
Extraction and consumption rates when $\gamma _{2}^{2}$ is changed to 1.2. The blue and red curves on the right are the consumption rates of Companies 1 and 2, respectively
 [1] Siyang Cai, Yongmei Cai, Xuerong Mao. A stochastic differential equation SIS epidemic model with regime switching. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020317 [2] Ténan Yeo. Stochastic and deterministic SIS patch model. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2021012 [3] Philipp Harms. Strong convergence rates for markovian representations of fractional processes. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020367 [4] Laura Aquilanti, Simone Cacace, Fabio Camilli, Raul De Maio. A Mean Field Games model for finite mixtures of Bernoulli and categorical distributions. Journal of Dynamics & Games, 2020  doi: 10.3934/jdg.2020033 [5] Simone Fagioli, Emanuela Radici. Opinion formation systems via deterministic particles approximation. Kinetic & Related Models, 2021, 14 (1) : 45-76. doi: 10.3934/krm.2020048 [6] Juan Pablo Pinasco, Mauro Rodriguez Cartabia, Nicolas Saintier. Evolutionary game theory in mixed strategies: From microscopic interactions to kinetic equations. Kinetic & Related Models, 2021, 14 (1) : 115-148. doi: 10.3934/krm.2020051 [7] Wei Chen, Yongkai Ma, Weihao Hu. Electricity supply chain coordination with carbon abatement technology investment under the benchmarking mechanism. Journal of Industrial & Management Optimization, 2020  doi: 10.3934/jimo.2020175 [8] Mark F. Demers. Uniqueness and exponential mixing for the measure of maximal entropy for piecewise hyperbolic maps. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 217-256. doi: 10.3934/dcds.2020217 [9] Djamel Aaid, Amel Noui, Özen Özer. Piecewise quadratic bounding functions for finding real roots of polynomials. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 63-73. doi: 10.3934/naco.2020015 [10] Fanni M. Sélley. A self-consistent dynamical system with multiple absolutely continuous invariant measures. Journal of Computational Dynamics, 2021, 8 (1) : 9-32. doi: 10.3934/jcd.2021002 [11] Meilan Cai, Maoan Han. Limit cycle bifurcations in a class of piecewise smooth cubic systems with multiple parameters. Communications on Pure & Applied Analysis, 2021, 20 (1) : 55-75. doi: 10.3934/cpaa.2020257 [12] Hui Lv, Xing'an Wang. Dissipative control for uncertain singular markovian jump systems via hybrid impulsive control. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 127-142. doi: 10.3934/naco.2020020 [13] Youming Guo, Tingting Li. Optimal control strategies for an online game addiction model with low and high risk exposure. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020347 [14] Yuyuan Ouyang, Trevor Squires. Some worst-case datasets of deterministic first-order methods for solving binary logistic regression. Inverse Problems & Imaging, 2021, 15 (1) : 63-77. doi: 10.3934/ipi.2020047 [15] Yuri Fedorov, Božidar Jovanović. Continuous and discrete Neumann systems on Stiefel varieties as matrix generalizations of the Jacobi–Mumford systems. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020375 [16] Mingjun Zhou, Jingxue Yin. Continuous subsonic-sonic flows in a two-dimensional semi-infinitely long nozzle. Electronic Research Archive, , () : -. doi: 10.3934/era.2020122 [17] Huanhuan Tian, Maoan Han. Limit cycle bifurcations of piecewise smooth near-Hamiltonian systems with a switching curve. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020368 [18] Lars Grüne, Roberto Guglielmi. On the relation between turnpike properties and dissipativity for continuous time linear quadratic optimal control problems. Mathematical Control & Related Fields, 2021, 11 (1) : 169-188. doi: 10.3934/mcrf.2020032 [19] Lorenzo Zambotti. A brief and personal history of stochastic partial differential equations. Discrete & Continuous Dynamical Systems - A, 2021, 41 (1) : 471-487. doi: 10.3934/dcds.2020264 [20] Aisling McGlinchey, Oliver Mason. Observations on the bias of nonnegative mechanisms for differential privacy. Foundations of Data Science, 2020, 2 (4) : 429-442. doi: 10.3934/fods.2020020

Impact Factor: