# American Institute of Mathematical Sciences

October  2019, 6(4): 337-366. doi: 10.3934/jdg.2019022

## Markovian strategies for piecewise deterministic differential games with continuous and impulse controls

 Department of Mathematics, Statistics and Computer Science, Purdue University Northwest, Hammond, IN 46323-2094, USA

Received  September 2019 Revised  September 2019 Published  October 2019

This paper is concerned with the Markovian feedback strategies of piecewise deterministic differential games and their applications to business and management decision-making problems that involve multiple agents and continuous and impulse controls. For a class of piecewise deterministic differential games in finite or infinite horizons we formulate conditions for the value functions in the form of quasi-variational inequalities, prove a verification theorem, and derive a criterion for the Markovian regime change in certain case. These results are applied to a technology adoption problem that involves multiple companies engaged in extraction of an exhaustible resource with different technologies. Using the model proposed by Long et al in [16], we show the existence of a pure Markovian strategy and develop an algorithm for computing the solutions.

Citation: Weihua Ruan. Markovian strategies for piecewise deterministic differential games with continuous and impulse controls. Journal of Dynamics & Games, 2019, 6 (4) : 337-366. doi: 10.3934/jdg.2019022
##### References:
 [1] R. Amit, Petroleum reservoir exploitation: Switching from primary to secondary recovery, Operations Research, 34 (1986), 534-549.  doi: 10.1287/opre.34.4.534.  Google Scholar [2] T. Başar and A. Haurie, Feedback equilibria in differential games with structural and modal uncertainties, Advances in Large Scale Systems, New York, 1 (1984), 163–201.  Google Scholar [3] R. Boucekkine, A. Pommeret and F. Prieur, Optimal regime switching and threshold effects, ournal of Economic Dynamics and Control, 37 (2013), 2979-2997.  doi: 10.1016/j.jedc.2013.08.008.  Google Scholar [4] R. Boucekkine, F. Prieur and K. Puzon, On the timing of political regime changes in resource-dependent economies, European Economic Review, 85 (2016), 188-207.  doi: 10.1016/j.euroecorev.2016.02.016.  Google Scholar [5] R. Boucekkine, C. Saglam and T. Vallée., Technology adoption under embodiment: A two-stage optimal control approach, Macroeconomic Dynamics, 8 (2004), 250-271.  doi: 10.1017/S1365100503030062.  Google Scholar [6] M. Breton and A. Haurie, Two-layer piecewise deterministic games, In Proceedings of the 28th IEEE Conference on Decision and Control, New York, 1989,198–200. doi: 10.1109/CDC.1989.70102.  Google Scholar [7] E. J. Dockner, S. J$\phi$rgensen, N. V. Long and G. Sorger, Differential Games in Economics and Management Science, Cambridge, Cambridge University Press, 2000. doi: 10.1017/CBO9780511805127.  Google Scholar [8] T. L. Friesz, Dynamic Optimization and Differential Games, Springer, New York Dordrecht Heidelberg London, 2010. Google Scholar [9] D. Fudenberg and J. Tirole, Preemption and rent equalization in the adoption of new technology, Review of Economic Studies, 52 (1985), 383-401.  doi: 10.2307/2297660.  Google Scholar [10] M. Ghosh and S. Marcus, Stochastic differential games with multiple modes, Stochastic Analysis and Applications, 16 (1998), 91-105.  doi: 10.1080/07362999808809519.  Google Scholar [11] J. Haunschmied, V. M. Veliov and S. Wrzaczek, editors, Dynamic Games in Economics, volume 16 of Dynamic Modeling and Econometrics in Economics and Finance, Springer-Verlag, Berlin Heidelberg, 2014. doi: 10.1007/978-3-642-54248-0.  Google Scholar [12] A. Haurie, Piecewise deterministic and piecewise diffusion differential games with modal uncertainties, Decision Processes in Economics, Springer, Berlin, 353 (1991), 107–120. doi: 10.1007/978-3-642-45686-2_10.  Google Scholar [13] A. Haurie, From repeated to differential games: How time and uncertainty pervade the theory of games, Frontiers of Game Theory, MIT Press, Cambridge, MA, 1993,165–194.  Google Scholar [14] A. Haurie, S. Muto, L. A. Petrosjan and T. E. S. Raghavan, editors, Advances in Dynamic Games–Applications to Economics, Management Science, Engineering, and Environmental Management, volume 8 of Annals of the International Society of Dynamic Games, Birkhäuser, Boston Basel Berlin, 2006. doi: 10.1007/0-8176-4501-2.  Google Scholar [15] S. J${\phi}$rgensen and G. Zaccour, Developments in differential game theory and numerical methods: Economic and management applications, Computational Management Science, 4 (2007), 159-181.  doi: 10.1007/s10287-006-0032-x.  Google Scholar [16] N. V. Long, F. Prieur, M. Tidball and K. Puzon, Piecewise closed-loop equilibria in differential games with regime switching strategies, J. Econom. Dynam. Control, 76 (2017), 264-284.  doi: 10.1016/j.jedc.2017.01.008.  Google Scholar [17] M. Makris, Necessary conditions for infinite-horizon discounted two-stage optimal control problems, J. Econom. Dynam. Control, 25 (2001), 1935-1950.  doi: 10.1016/S0165-1889(00)00009-9.  Google Scholar [18] F. E. Ouardighi and K. Kogan, editors, Models and Methods in Economics and Management Science–Essays in Honor of Charles S. Tapiero, volume 198 of International Series in Operations Research and Management Science, Springer, Cham Heidelberg New York Dordrecht London, 2014. doi: 10.1007/978-3-319-00669-7.  Google Scholar [19] J. F. Reinganum, On the diffusion of new technology: A game theoretic approach, Review of Economic Studies, 48 (1981), 395-405.  doi: 10.2307/2297153.  Google Scholar [20] C. Saglam, Optimal pattern of technology adoptions under embodiment: A multi-stage optimal control approach, Optim. Control Appl. Meth., 32 (2011), 574-586.  doi: 10.1002/oca.960.  Google Scholar [21] K. Tomiyama, Two-stage optimal control problems and optimality conditions, ournal of Economic Dynamics and Control, 9 (1985), 317-337.  doi: 10.1016/0165-1889(85)90010-7.  Google Scholar [22] K. Tomiyama and R. Rossana, Two-stage optimal control problems with an explicit switch point dependence–optimality criteria and an example of delivery lags and investment, J. Econom. Dynam. Control, 13 (1989), 319-337.  doi: 10.1016/0165-1889(89)90027-4.  Google Scholar [23] S. Valente, Endogenous growth, backstop technology adoption, and optimal jumps, Macroeconomic Dynamics, 15 (2011), 293-325.   Google Scholar

show all references

##### References:
 [1] R. Amit, Petroleum reservoir exploitation: Switching from primary to secondary recovery, Operations Research, 34 (1986), 534-549.  doi: 10.1287/opre.34.4.534.  Google Scholar [2] T. Başar and A. Haurie, Feedback equilibria in differential games with structural and modal uncertainties, Advances in Large Scale Systems, New York, 1 (1984), 163–201.  Google Scholar [3] R. Boucekkine, A. Pommeret and F. Prieur, Optimal regime switching and threshold effects, ournal of Economic Dynamics and Control, 37 (2013), 2979-2997.  doi: 10.1016/j.jedc.2013.08.008.  Google Scholar [4] R. Boucekkine, F. Prieur and K. Puzon, On the timing of political regime changes in resource-dependent economies, European Economic Review, 85 (2016), 188-207.  doi: 10.1016/j.euroecorev.2016.02.016.  Google Scholar [5] R. Boucekkine, C. Saglam and T. Vallée., Technology adoption under embodiment: A two-stage optimal control approach, Macroeconomic Dynamics, 8 (2004), 250-271.  doi: 10.1017/S1365100503030062.  Google Scholar [6] M. Breton and A. Haurie, Two-layer piecewise deterministic games, In Proceedings of the 28th IEEE Conference on Decision and Control, New York, 1989,198–200. doi: 10.1109/CDC.1989.70102.  Google Scholar [7] E. J. Dockner, S. J$\phi$rgensen, N. V. Long and G. Sorger, Differential Games in Economics and Management Science, Cambridge, Cambridge University Press, 2000. doi: 10.1017/CBO9780511805127.  Google Scholar [8] T. L. Friesz, Dynamic Optimization and Differential Games, Springer, New York Dordrecht Heidelberg London, 2010. Google Scholar [9] D. Fudenberg and J. Tirole, Preemption and rent equalization in the adoption of new technology, Review of Economic Studies, 52 (1985), 383-401.  doi: 10.2307/2297660.  Google Scholar [10] M. Ghosh and S. Marcus, Stochastic differential games with multiple modes, Stochastic Analysis and Applications, 16 (1998), 91-105.  doi: 10.1080/07362999808809519.  Google Scholar [11] J. Haunschmied, V. M. Veliov and S. Wrzaczek, editors, Dynamic Games in Economics, volume 16 of Dynamic Modeling and Econometrics in Economics and Finance, Springer-Verlag, Berlin Heidelberg, 2014. doi: 10.1007/978-3-642-54248-0.  Google Scholar [12] A. Haurie, Piecewise deterministic and piecewise diffusion differential games with modal uncertainties, Decision Processes in Economics, Springer, Berlin, 353 (1991), 107–120. doi: 10.1007/978-3-642-45686-2_10.  Google Scholar [13] A. Haurie, From repeated to differential games: How time and uncertainty pervade the theory of games, Frontiers of Game Theory, MIT Press, Cambridge, MA, 1993,165–194.  Google Scholar [14] A. Haurie, S. Muto, L. A. Petrosjan and T. E. S. Raghavan, editors, Advances in Dynamic Games–Applications to Economics, Management Science, Engineering, and Environmental Management, volume 8 of Annals of the International Society of Dynamic Games, Birkhäuser, Boston Basel Berlin, 2006. doi: 10.1007/0-8176-4501-2.  Google Scholar [15] S. J${\phi}$rgensen and G. Zaccour, Developments in differential game theory and numerical methods: Economic and management applications, Computational Management Science, 4 (2007), 159-181.  doi: 10.1007/s10287-006-0032-x.  Google Scholar [16] N. V. Long, F. Prieur, M. Tidball and K. Puzon, Piecewise closed-loop equilibria in differential games with regime switching strategies, J. Econom. Dynam. Control, 76 (2017), 264-284.  doi: 10.1016/j.jedc.2017.01.008.  Google Scholar [17] M. Makris, Necessary conditions for infinite-horizon discounted two-stage optimal control problems, J. Econom. Dynam. Control, 25 (2001), 1935-1950.  doi: 10.1016/S0165-1889(00)00009-9.  Google Scholar [18] F. E. Ouardighi and K. Kogan, editors, Models and Methods in Economics and Management Science–Essays in Honor of Charles S. Tapiero, volume 198 of International Series in Operations Research and Management Science, Springer, Cham Heidelberg New York Dordrecht London, 2014. doi: 10.1007/978-3-319-00669-7.  Google Scholar [19] J. F. Reinganum, On the diffusion of new technology: A game theoretic approach, Review of Economic Studies, 48 (1981), 395-405.  doi: 10.2307/2297153.  Google Scholar [20] C. Saglam, Optimal pattern of technology adoptions under embodiment: A multi-stage optimal control approach, Optim. Control Appl. Meth., 32 (2011), 574-586.  doi: 10.1002/oca.960.  Google Scholar [21] K. Tomiyama, Two-stage optimal control problems and optimality conditions, ournal of Economic Dynamics and Control, 9 (1985), 317-337.  doi: 10.1016/0165-1889(85)90010-7.  Google Scholar [22] K. Tomiyama and R. Rossana, Two-stage optimal control problems with an explicit switch point dependence–optimality criteria and an example of delivery lags and investment, J. Econom. Dynam. Control, 13 (1989), 319-337.  doi: 10.1016/0165-1889(89)90027-4.  Google Scholar [23] S. Valente, Endogenous growth, backstop technology adoption, and optimal jumps, Macroeconomic Dynamics, 15 (2011), 293-325.   Google Scholar
Possible strategies with parameter values given by (3.15)
Extraction and consumption rates using parameters in (3.15)
Possible strategies with $\gamma^2 = 1.6$ and other parameter values given by (3.15)
Extraction and consumption rates when $\gamma ^{2}$ is changed to 1.6
Extraction and consumption rates. The blue and red curves on the right are the consumption rates of Companies 1 and 2, respectively
Extraction and consumption rates when $\gamma _{2}^{2}$ is changed to 1.2. The blue and red curves on the right are the consumption rates of Companies 1 and 2, respectively
 [1] Dejian Chang, Zhen Wu. Stochastic maximum principle for non-zero sum differential games of FBSDEs with impulse controls and its application to finance. Journal of Industrial & Management Optimization, 2015, 11 (1) : 27-40. doi: 10.3934/jimo.2015.11.27 [2] Alejandra Fonseca-Morales, Onésimo Hernández-Lerma. A note on differential games with Pareto-optimal NASH equilibria: Deterministic and stochastic models†. Journal of Dynamics & Games, 2017, 4 (3) : 195-203. doi: 10.3934/jdg.2017012 [3] Zeyang Wang, Ovanes Petrosian. On class of non-transferable utility cooperative differential games with continuous updating. Journal of Dynamics & Games, 2020, 7 (4) : 291-302. doi: 10.3934/jdg.2020020 [4] Yulan Lu, Minghui Song, Mingzhu Liu. Convergence rate and stability of the split-step theta method for stochastic differential equations with piecewise continuous arguments. Discrete & Continuous Dynamical Systems - B, 2019, 24 (2) : 695-717. doi: 10.3934/dcdsb.2018203 [5] Michael Basin, Mark A. Pinsky. Control of Kalman-like filters using impulse and continuous feedback design. Discrete & Continuous Dynamical Systems - B, 2002, 2 (2) : 169-184. doi: 10.3934/dcdsb.2002.2.169 [6] Kashi Behrstock, Michel Benaïm, Morris W. Hirsch. Smale strategies for network prisoner's dilemma games. Journal of Dynamics & Games, 2015, 2 (2) : 141-155. doi: 10.3934/jdg.2015.2.141 [7] Zhichuan Zhu, Bo Yu, Li Yang. Globally convergent homotopy method for designing piecewise linear deterministic contractual function. Journal of Industrial & Management Optimization, 2014, 10 (3) : 717-741. doi: 10.3934/jimo.2014.10.717 [8] Jeremias Epperlein, Stefan Siegmund, Petr Stehlík, Vladimír  Švígler. Coexistence equilibria of evolutionary games on graphs under deterministic imitation dynamics. Discrete & Continuous Dynamical Systems - B, 2016, 21 (3) : 803-813. doi: 10.3934/dcdsb.2016.21.803 [9] Astridh Boccabella, Roberto Natalini, Lorenzo Pareschi. On a continuous mixed strategies model for evolutionary game theory. Kinetic & Related Models, 2011, 4 (1) : 187-213. doi: 10.3934/krm.2011.4.187 [10] Yaozhong Hu, David Nualart, Xiaobin Sun, Yingchao Xie. Smoothness of density for stochastic differential equations with Markovian switching. Discrete & Continuous Dynamical Systems - B, 2019, 24 (8) : 3615-3631. doi: 10.3934/dcdsb.2018307 [11] Lin Xu, Rongming Wang, Dingjun Yao. Optimal stochastic investment games under Markov regime switching market. Journal of Industrial & Management Optimization, 2014, 10 (3) : 795-815. doi: 10.3934/jimo.2014.10.795 [12] Yves Cornulier. Realizations of groups of piecewise continuous transformations of the circle. Journal of Modern Dynamics, 2020, 16: 59-80. doi: 10.3934/jmd.2020003 [13] Yves Achdou, Victor Perez. Iterative strategies for solving linearized discrete mean field games systems. Networks & Heterogeneous Media, 2012, 7 (2) : 197-217. doi: 10.3934/nhm.2012.7.197 [14] Miquel Oliu-Barton. Asymptotically optimal strategies in repeated games with incomplete information and vanishing weights. Journal of Dynamics & Games, 2019, 6 (4) : 259-275. doi: 10.3934/jdg.2019018 [15] John A. Morgan. Interception in differential pursuit/evasion games. Journal of Dynamics & Games, 2016, 3 (4) : 335-354. doi: 10.3934/jdg.2016018 [16] Guangliang Zhao, Fuke Wu, George Yin. Feedback controls to ensure global solutions and asymptotic stability of Markovian switching diffusion systems. Mathematical Control & Related Fields, 2015, 5 (2) : 359-376. doi: 10.3934/mcrf.2015.5.359 [17] Yong He. Switching controls for linear stochastic differential systems. Mathematical Control & Related Fields, 2020, 10 (2) : 443-454. doi: 10.3934/mcrf.2020005 [18] Vincent Renault, Michèle Thieullen, Emmanuel Trélat. Optimal control of infinite-dimensional piecewise deterministic Markov processes and application to the control of neuronal dynamics via Optogenetics. Networks & Heterogeneous Media, 2017, 12 (3) : 417-459. doi: 10.3934/nhm.2017019 [19] Pierre Cardaliaguet, Chloé Jimenez, Marc Quincampoix. Pure and Random strategies in differential game with incomplete informations. Journal of Dynamics & Games, 2014, 1 (3) : 363-375. doi: 10.3934/jdg.2014.1.363 [20] Carmen Núñez, Rafael Obaya. A non-autonomous bifurcation theory for deterministic scalar differential equations. Discrete & Continuous Dynamical Systems - B, 2008, 9 (3&4, May) : 701-730. doi: 10.3934/dcdsb.2008.9.701

Impact Factor: