October  2019, 6(4): 337-366. doi: 10.3934/jdg.2019022

Markovian strategies for piecewise deterministic differential games with continuous and impulse controls

Department of Mathematics, Statistics and Computer Science, Purdue University Northwest, Hammond, IN 46323-2094, USA

Received  September 2019 Revised  September 2019 Published  October 2019

This paper is concerned with the Markovian feedback strategies of piecewise deterministic differential games and their applications to business and management decision-making problems that involve multiple agents and continuous and impulse controls. For a class of piecewise deterministic differential games in finite or infinite horizons we formulate conditions for the value functions in the form of quasi-variational inequalities, prove a verification theorem, and derive a criterion for the Markovian regime change in certain case. These results are applied to a technology adoption problem that involves multiple companies engaged in extraction of an exhaustible resource with different technologies. Using the model proposed by Long et al in [16], we show the existence of a pure Markovian strategy and develop an algorithm for computing the solutions.

Citation: Weihua Ruan. Markovian strategies for piecewise deterministic differential games with continuous and impulse controls. Journal of Dynamics & Games, 2019, 6 (4) : 337-366. doi: 10.3934/jdg.2019022
References:
[1]

R. Amit, Petroleum reservoir exploitation: Switching from primary to secondary recovery, Operations Research, 34 (1986), 534-549.  doi: 10.1287/opre.34.4.534.  Google Scholar

[2]

T. Başar and A. Haurie, Feedback equilibria in differential games with structural and modal uncertainties, Advances in Large Scale Systems, New York, 1 (1984), 163–201.  Google Scholar

[3]

R. BoucekkineA. Pommeret and F. Prieur, Optimal regime switching and threshold effects, ournal of Economic Dynamics and Control, 37 (2013), 2979-2997.  doi: 10.1016/j.jedc.2013.08.008.  Google Scholar

[4]

R. BoucekkineF. Prieur and K. Puzon, On the timing of political regime changes in resource-dependent economies, European Economic Review, 85 (2016), 188-207.  doi: 10.1016/j.euroecorev.2016.02.016.  Google Scholar

[5]

R. BoucekkineC. Saglam and T. Vallée., Technology adoption under embodiment: A two-stage optimal control approach, Macroeconomic Dynamics, 8 (2004), 250-271.  doi: 10.1017/S1365100503030062.  Google Scholar

[6]

M. Breton and A. Haurie, Two-layer piecewise deterministic games, In Proceedings of the 28th IEEE Conference on Decision and Control, New York, 1989,198–200. doi: 10.1109/CDC.1989.70102.  Google Scholar

[7]

E. J. Dockner, S. J$\phi $rgensen, N. V. Long and G. Sorger, Differential Games in Economics and Management Science, Cambridge, Cambridge University Press, 2000. doi: 10.1017/CBO9780511805127.  Google Scholar

[8]

T. L. Friesz, Dynamic Optimization and Differential Games, Springer, New York Dordrecht Heidelberg London, 2010. Google Scholar

[9]

D. Fudenberg and J. Tirole, Preemption and rent equalization in the adoption of new technology, Review of Economic Studies, 52 (1985), 383-401.  doi: 10.2307/2297660.  Google Scholar

[10]

M. Ghosh and S. Marcus, Stochastic differential games with multiple modes, Stochastic Analysis and Applications, 16 (1998), 91-105.  doi: 10.1080/07362999808809519.  Google Scholar

[11]

J. Haunschmied, V. M. Veliov and S. Wrzaczek, editors, Dynamic Games in Economics, volume 16 of Dynamic Modeling and Econometrics in Economics and Finance, Springer-Verlag, Berlin Heidelberg, 2014. doi: 10.1007/978-3-642-54248-0.  Google Scholar

[12]

A. Haurie, Piecewise deterministic and piecewise diffusion differential games with modal uncertainties, Decision Processes in Economics, Springer, Berlin, 353 (1991), 107–120. doi: 10.1007/978-3-642-45686-2_10.  Google Scholar

[13]

A. Haurie, From repeated to differential games: How time and uncertainty pervade the theory of games, Frontiers of Game Theory, MIT Press, Cambridge, MA, 1993,165–194.  Google Scholar

[14]

A. Haurie, S. Muto, L. A. Petrosjan and T. E. S. Raghavan, editors, Advances in Dynamic Games–Applications to Economics, Management Science, Engineering, and Environmental Management, volume 8 of Annals of the International Society of Dynamic Games, Birkhäuser, Boston Basel Berlin, 2006. doi: 10.1007/0-8176-4501-2.  Google Scholar

[15]

S. J${\phi}$rgensen and G. Zaccour, Developments in differential game theory and numerical methods: Economic and management applications, Computational Management Science, 4 (2007), 159-181.  doi: 10.1007/s10287-006-0032-x.  Google Scholar

[16]

N. V. LongF. PrieurM. Tidball and K. Puzon, Piecewise closed-loop equilibria in differential games with regime switching strategies, J. Econom. Dynam. Control, 76 (2017), 264-284.  doi: 10.1016/j.jedc.2017.01.008.  Google Scholar

[17]

M. Makris, Necessary conditions for infinite-horizon discounted two-stage optimal control problems, J. Econom. Dynam. Control, 25 (2001), 1935-1950.  doi: 10.1016/S0165-1889(00)00009-9.  Google Scholar

[18]

F. E. Ouardighi and K. Kogan, editors, Models and Methods in Economics and Management Science–Essays in Honor of Charles S. Tapiero, volume 198 of International Series in Operations Research and Management Science, Springer, Cham Heidelberg New York Dordrecht London, 2014. doi: 10.1007/978-3-319-00669-7.  Google Scholar

[19]

J. F. Reinganum, On the diffusion of new technology: A game theoretic approach, Review of Economic Studies, 48 (1981), 395-405.  doi: 10.2307/2297153.  Google Scholar

[20]

C. Saglam, Optimal pattern of technology adoptions under embodiment: A multi-stage optimal control approach, Optim. Control Appl. Meth., 32 (2011), 574-586.  doi: 10.1002/oca.960.  Google Scholar

[21]

K. Tomiyama, Two-stage optimal control problems and optimality conditions, ournal of Economic Dynamics and Control, 9 (1985), 317-337.  doi: 10.1016/0165-1889(85)90010-7.  Google Scholar

[22]

K. Tomiyama and R. Rossana, Two-stage optimal control problems with an explicit switch point dependence–optimality criteria and an example of delivery lags and investment, J. Econom. Dynam. Control, 13 (1989), 319-337.  doi: 10.1016/0165-1889(89)90027-4.  Google Scholar

[23]

S. Valente, Endogenous growth, backstop technology adoption, and optimal jumps, Macroeconomic Dynamics, 15 (2011), 293-325.   Google Scholar

show all references

References:
[1]

R. Amit, Petroleum reservoir exploitation: Switching from primary to secondary recovery, Operations Research, 34 (1986), 534-549.  doi: 10.1287/opre.34.4.534.  Google Scholar

[2]

T. Başar and A. Haurie, Feedback equilibria in differential games with structural and modal uncertainties, Advances in Large Scale Systems, New York, 1 (1984), 163–201.  Google Scholar

[3]

R. BoucekkineA. Pommeret and F. Prieur, Optimal regime switching and threshold effects, ournal of Economic Dynamics and Control, 37 (2013), 2979-2997.  doi: 10.1016/j.jedc.2013.08.008.  Google Scholar

[4]

R. BoucekkineF. Prieur and K. Puzon, On the timing of political regime changes in resource-dependent economies, European Economic Review, 85 (2016), 188-207.  doi: 10.1016/j.euroecorev.2016.02.016.  Google Scholar

[5]

R. BoucekkineC. Saglam and T. Vallée., Technology adoption under embodiment: A two-stage optimal control approach, Macroeconomic Dynamics, 8 (2004), 250-271.  doi: 10.1017/S1365100503030062.  Google Scholar

[6]

M. Breton and A. Haurie, Two-layer piecewise deterministic games, In Proceedings of the 28th IEEE Conference on Decision and Control, New York, 1989,198–200. doi: 10.1109/CDC.1989.70102.  Google Scholar

[7]

E. J. Dockner, S. J$\phi $rgensen, N. V. Long and G. Sorger, Differential Games in Economics and Management Science, Cambridge, Cambridge University Press, 2000. doi: 10.1017/CBO9780511805127.  Google Scholar

[8]

T. L. Friesz, Dynamic Optimization and Differential Games, Springer, New York Dordrecht Heidelberg London, 2010. Google Scholar

[9]

D. Fudenberg and J. Tirole, Preemption and rent equalization in the adoption of new technology, Review of Economic Studies, 52 (1985), 383-401.  doi: 10.2307/2297660.  Google Scholar

[10]

M. Ghosh and S. Marcus, Stochastic differential games with multiple modes, Stochastic Analysis and Applications, 16 (1998), 91-105.  doi: 10.1080/07362999808809519.  Google Scholar

[11]

J. Haunschmied, V. M. Veliov and S. Wrzaczek, editors, Dynamic Games in Economics, volume 16 of Dynamic Modeling and Econometrics in Economics and Finance, Springer-Verlag, Berlin Heidelberg, 2014. doi: 10.1007/978-3-642-54248-0.  Google Scholar

[12]

A. Haurie, Piecewise deterministic and piecewise diffusion differential games with modal uncertainties, Decision Processes in Economics, Springer, Berlin, 353 (1991), 107–120. doi: 10.1007/978-3-642-45686-2_10.  Google Scholar

[13]

A. Haurie, From repeated to differential games: How time and uncertainty pervade the theory of games, Frontiers of Game Theory, MIT Press, Cambridge, MA, 1993,165–194.  Google Scholar

[14]

A. Haurie, S. Muto, L. A. Petrosjan and T. E. S. Raghavan, editors, Advances in Dynamic Games–Applications to Economics, Management Science, Engineering, and Environmental Management, volume 8 of Annals of the International Society of Dynamic Games, Birkhäuser, Boston Basel Berlin, 2006. doi: 10.1007/0-8176-4501-2.  Google Scholar

[15]

S. J${\phi}$rgensen and G. Zaccour, Developments in differential game theory and numerical methods: Economic and management applications, Computational Management Science, 4 (2007), 159-181.  doi: 10.1007/s10287-006-0032-x.  Google Scholar

[16]

N. V. LongF. PrieurM. Tidball and K. Puzon, Piecewise closed-loop equilibria in differential games with regime switching strategies, J. Econom. Dynam. Control, 76 (2017), 264-284.  doi: 10.1016/j.jedc.2017.01.008.  Google Scholar

[17]

M. Makris, Necessary conditions for infinite-horizon discounted two-stage optimal control problems, J. Econom. Dynam. Control, 25 (2001), 1935-1950.  doi: 10.1016/S0165-1889(00)00009-9.  Google Scholar

[18]

F. E. Ouardighi and K. Kogan, editors, Models and Methods in Economics and Management Science–Essays in Honor of Charles S. Tapiero, volume 198 of International Series in Operations Research and Management Science, Springer, Cham Heidelberg New York Dordrecht London, 2014. doi: 10.1007/978-3-319-00669-7.  Google Scholar

[19]

J. F. Reinganum, On the diffusion of new technology: A game theoretic approach, Review of Economic Studies, 48 (1981), 395-405.  doi: 10.2307/2297153.  Google Scholar

[20]

C. Saglam, Optimal pattern of technology adoptions under embodiment: A multi-stage optimal control approach, Optim. Control Appl. Meth., 32 (2011), 574-586.  doi: 10.1002/oca.960.  Google Scholar

[21]

K. Tomiyama, Two-stage optimal control problems and optimality conditions, ournal of Economic Dynamics and Control, 9 (1985), 317-337.  doi: 10.1016/0165-1889(85)90010-7.  Google Scholar

[22]

K. Tomiyama and R. Rossana, Two-stage optimal control problems with an explicit switch point dependence–optimality criteria and an example of delivery lags and investment, J. Econom. Dynam. Control, 13 (1989), 319-337.  doi: 10.1016/0165-1889(89)90027-4.  Google Scholar

[23]

S. Valente, Endogenous growth, backstop technology adoption, and optimal jumps, Macroeconomic Dynamics, 15 (2011), 293-325.   Google Scholar

Figure 3.1.  Possible strategies with parameter values given by (3.15)
Figure 3.2.  Extraction and consumption rates using parameters in (3.15)
Figure 3.3.  Possible strategies with $ \gamma^2 = 1.6 $ and other parameter values given by (3.15)
Figure 4.  Extraction and consumption rates when $ \gamma ^{2} $ is changed to 1.6
Figure 3.5.  Extraction and consumption rates. The blue and red curves on the right are the consumption rates of Companies 1 and 2, respectively
Figure 3.6.  Extraction and consumption rates when $ \gamma _{2}^{2} $ is changed to 1.2. The blue and red curves on the right are the consumption rates of Companies 1 and 2, respectively
[1]

Dejian Chang, Zhen Wu. Stochastic maximum principle for non-zero sum differential games of FBSDEs with impulse controls and its application to finance. Journal of Industrial & Management Optimization, 2015, 11 (1) : 27-40. doi: 10.3934/jimo.2015.11.27

[2]

Alejandra Fonseca-Morales, Onésimo Hernández-Lerma. A note on differential games with Pareto-optimal NASH equilibria: Deterministic and stochastic models. Journal of Dynamics & Games, 2017, 4 (3) : 195-203. doi: 10.3934/jdg.2017012

[3]

Yulan Lu, Minghui Song, Mingzhu Liu. Convergence rate and stability of the split-step theta method for stochastic differential equations with piecewise continuous arguments. Discrete & Continuous Dynamical Systems - B, 2019, 24 (2) : 695-717. doi: 10.3934/dcdsb.2018203

[4]

Michael Basin, Mark A. Pinsky. Control of Kalman-like filters using impulse and continuous feedback design. Discrete & Continuous Dynamical Systems - B, 2002, 2 (2) : 169-184. doi: 10.3934/dcdsb.2002.2.169

[5]

Kashi Behrstock, Michel Benaïm, Morris W. Hirsch. Smale strategies for network prisoner's dilemma games. Journal of Dynamics & Games, 2015, 2 (2) : 141-155. doi: 10.3934/jdg.2015.2.141

[6]

Zhichuan Zhu, Bo Yu, Li Yang. Globally convergent homotopy method for designing piecewise linear deterministic contractual function. Journal of Industrial & Management Optimization, 2014, 10 (3) : 717-741. doi: 10.3934/jimo.2014.10.717

[7]

Jeremias Epperlein, Stefan Siegmund, Petr Stehlík, Vladimír  Švígler. Coexistence equilibria of evolutionary games on graphs under deterministic imitation dynamics. Discrete & Continuous Dynamical Systems - B, 2016, 21 (3) : 803-813. doi: 10.3934/dcdsb.2016.21.803

[8]

Yaozhong Hu, David Nualart, Xiaobin Sun, Yingchao Xie. Smoothness of density for stochastic differential equations with Markovian switching. Discrete & Continuous Dynamical Systems - B, 2019, 24 (8) : 3615-3631. doi: 10.3934/dcdsb.2018307

[9]

Lin Xu, Rongming Wang, Dingjun Yao. Optimal stochastic investment games under Markov regime switching market. Journal of Industrial & Management Optimization, 2014, 10 (3) : 795-815. doi: 10.3934/jimo.2014.10.795

[10]

Astridh Boccabella, Roberto Natalini, Lorenzo Pareschi. On a continuous mixed strategies model for evolutionary game theory. Kinetic & Related Models, 2011, 4 (1) : 187-213. doi: 10.3934/krm.2011.4.187

[11]

John A. Morgan. Interception in differential pursuit/evasion games. Journal of Dynamics & Games, 2016, 3 (4) : 335-354. doi: 10.3934/jdg.2016018

[12]

Guangliang Zhao, Fuke Wu, George Yin. Feedback controls to ensure global solutions and asymptotic stability of Markovian switching diffusion systems. Mathematical Control & Related Fields, 2015, 5 (2) : 359-376. doi: 10.3934/mcrf.2015.5.359

[13]

Yves Achdou, Victor Perez. Iterative strategies for solving linearized discrete mean field games systems. Networks & Heterogeneous Media, 2012, 7 (2) : 197-217. doi: 10.3934/nhm.2012.7.197

[14]

Miquel Oliu-Barton. Asymptotically optimal strategies in repeated games with incomplete information and vanishing weights. Journal of Dynamics & Games, 2019, 6 (4) : 259-275. doi: 10.3934/jdg.2019018

[15]

Tak Kuen Siu, Yang Shen. Risk-minimizing pricing and Esscher transform in a general non-Markovian regime-switching jump-diffusion model. Discrete & Continuous Dynamical Systems - B, 2017, 22 (7) : 2595-2626. doi: 10.3934/dcdsb.2017100

[16]

Vincent Renault, Michèle Thieullen, Emmanuel Trélat. Optimal control of infinite-dimensional piecewise deterministic Markov processes and application to the control of neuronal dynamics via Optogenetics. Networks & Heterogeneous Media, 2017, 12 (3) : 417-459. doi: 10.3934/nhm.2017019

[17]

Pierre Cardaliaguet, Chloé Jimenez, Marc Quincampoix. Pure and Random strategies in differential game with incomplete informations. Journal of Dynamics & Games, 2014, 1 (3) : 363-375. doi: 10.3934/jdg.2014.1.363

[18]

Carmen Núñez, Rafael Obaya. A non-autonomous bifurcation theory for deterministic scalar differential equations. Discrete & Continuous Dynamical Systems - B, 2008, 9 (3&4, May) : 701-730. doi: 10.3934/dcdsb.2008.9.701

[19]

Sigurdur F. Hafstein, Christopher M. Kellett, Huijuan Li. Computing continuous and piecewise affine lyapunov functions for nonlinear systems. Journal of Computational Dynamics, 2015, 2 (2) : 227-246. doi: 10.3934/jcd.2015004

[20]

Hui Liang, Hermann Brunner. Collocation methods for differential equations with piecewise linear delays. Communications on Pure & Applied Analysis, 2012, 11 (5) : 1839-1857. doi: 10.3934/cpaa.2012.11.1839

 Impact Factor: 

Metrics

  • PDF downloads (19)
  • HTML views (25)
  • Cited by (0)

Other articles
by authors

[Back to Top]