American Institute of Mathematical Sciences

June  2022, 1(2): 287-319. doi: 10.3934/fmf.2021011

Convergence of deep fictitious play for stochastic differential games

 1 Center for Computational Mathematics, Flatiron Institute, 162 5th Avenue, New York, NY, USA 2 Department of Mathematics, Princeton University, Princeton, NJ, USA 3 Department of Mathematics, and Department of Statistics and Applied Probability, University of California, Santa Barbara, CA, USA 4 The Program in Applied and Computational Mathematics, Princeton University, Princeton, NJ, USA

* Corresponding author: Ruimeng Hu

Received  September 2021 Revised  March 2022 Published  June 2022 Early access  May 2022

Fund Project: R.H. was partially supported by the NSF grant DMS-1953035

Stochastic differential games have been used extensively to model agents' competitions in finance, for instance, in P2P lending platforms from the Fintech industry, the banking system for systemic risk, and insurance markets. The recently proposed machine learning algorithm, deep fictitious play, provides a novel and efficient tool for finding Markovian Nash equilibrium of large $N$-player asymmetric stochastic differential games [J. Han and R. Hu, Mathematical and Scientific Machine Learning Conference, pages 221-245, PMLR, 2020]. By incorporating the idea of fictitious play, the algorithm decouples the game into $N$ sub-optimization problems, and identifies each player's optimal strategy with the deep backward stochastic differential equation (BSDE) method parallelly and repeatedly. In this paper, we prove the convergence of deep fictitious play (DFP) to the true Nash equilibrium. We can also show that the strategy based on DFP forms an $\epsilon$-Nash equilibrium. We generalize the algorithm by proposing a new approach to decouple the games, and present numerical results of large population games showing the empirical convergence of the algorithm beyond the technical assumptions in the theorems.

Citation: Jiequn Han, Ruimeng Hu, Jihao Long. Convergence of deep fictitious play for stochastic differential games. Frontiers of Mathematical Finance, 2022, 1 (2) : 287-319. doi: 10.3934/fmf.2021011
References:

show all references

References:
A sample path for all $N = 10$ players in the inter-bank game, obtained from decoupling the problem by policy update and solving the sub-problems with the Deep BSDE method. Top: the optimal state process $X_t^i$ (solid lines) and its neural networks approximation $\hat{X}_t^i$ (circles), under the same realized path of Brownian motion. Bottom: comparisons of the strategies $\alpha_t^i$ and $\hat{\alpha}_t^i$ (dashed lines)
 [1] Rui Mu, Zhen Wu. Nash equilibrium points of recursive nonzero-sum stochastic differential games with unbounded coefficients and related multiple\\ dimensional BSDEs. Mathematical Control and Related Fields, 2017, 7 (2) : 289-304. doi: 10.3934/mcrf.2017010 [2] Yaozhong Hu, David Nualart, Xiaobin Sun, Yingchao Xie. Smoothness of density for stochastic differential equations with Markovian switching. Discrete and Continuous Dynamical Systems - B, 2019, 24 (8) : 3615-3631. doi: 10.3934/dcdsb.2018307 [3] Jasmina Djordjević, Svetlana Janković. Reflected backward stochastic differential equations with perturbations. Discrete and Continuous Dynamical Systems, 2018, 38 (4) : 1833-1848. doi: 10.3934/dcds.2018075 [4] Jan A. Van Casteren. On backward stochastic differential equations in infinite dimensions. Discrete and Continuous Dynamical Systems - S, 2013, 6 (3) : 803-824. doi: 10.3934/dcdss.2013.6.803 [5] Joscha Diehl, Jianfeng Zhang. Backward stochastic differential equations with Young drift. Probability, Uncertainty and Quantitative Risk, 2017, 2 (0) : 5-. doi: 10.1186/s41546-017-0016-5 [6] Yanqiang Chang, Huabin Chen. Stability analysis of stochastic delay differential equations with Markovian switching driven by Lévy noise. Discrete and Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021301 [7] Chuchu Chen, Jialin Hong. Mean-square convergence of numerical approximations for a class of backward stochastic differential equations. Discrete and Continuous Dynamical Systems - B, 2013, 18 (8) : 2051-2067. doi: 10.3934/dcdsb.2013.18.2051 [8] Dariusz Borkowski. Forward and backward filtering based on backward stochastic differential equations. Inverse Problems and Imaging, 2016, 10 (2) : 305-325. doi: 10.3934/ipi.2016002 [9] Ying Hu, Shanjian Tang. Switching game of backward stochastic differential equations and associated system of obliquely reflected backward stochastic differential equations. Discrete and Continuous Dynamical Systems, 2015, 35 (11) : 5447-5465. doi: 10.3934/dcds.2015.35.5447 [10] Xin Chen, Ana Bela Cruzeiro. Stochastic geodesics and forward-backward stochastic differential equations on Lie groups. Conference Publications, 2013, 2013 (special) : 115-121. doi: 10.3934/proc.2013.2013.115 [11] Alejandra Fonseca-Morales, Onésimo Hernández-Lerma. A note on differential games with Pareto-optimal NASH equilibria: Deterministic and stochastic models†. Journal of Dynamics and Games, 2017, 4 (3) : 195-203. doi: 10.3934/jdg.2017012 [12] Qi Zhang, Huaizhong Zhao. Backward doubly stochastic differential equations with polynomial growth coefficients. Discrete and Continuous Dynamical Systems, 2015, 35 (11) : 5285-5315. doi: 10.3934/dcds.2015.35.5285 [13] Yufeng Shi, Qingfeng Zhu. A Kneser-type theorem for backward doubly stochastic differential equations. Discrete and Continuous Dynamical Systems - B, 2010, 14 (4) : 1565-1579. doi: 10.3934/dcdsb.2010.14.1565 [14] Yanqing Wang. A semidiscrete Galerkin scheme for backward stochastic parabolic differential equations. Mathematical Control and Related Fields, 2016, 6 (3) : 489-515. doi: 10.3934/mcrf.2016013 [15] Weidong Zhao, Jinlei Wang, Shige Peng. Error estimates of the $\theta$-scheme for backward stochastic differential equations. Discrete and Continuous Dynamical Systems - B, 2009, 12 (4) : 905-924. doi: 10.3934/dcdsb.2009.12.905 [16] Weidong Zhao, Yang Li, Guannan Zhang. A generalized $\theta$-scheme for solving backward stochastic differential equations. Discrete and Continuous Dynamical Systems - B, 2012, 17 (5) : 1585-1603. doi: 10.3934/dcdsb.2012.17.1585 [17] Yueyang Zheng, Jingtao Shi. A stackelberg game of backward stochastic differential equations with partial information. Mathematical Control and Related Fields, 2021, 11 (4) : 797-828. doi: 10.3934/mcrf.2020047 [18] Jiongmin Yong. Forward-backward stochastic differential equations: Initiation, development and beyond. Numerical Algebra, Control and Optimization, 2022  doi: 10.3934/naco.2022011 [19] Yinggu Chen, Said HamadÈne, Tingshu Mu. Mean-field doubly reflected backward stochastic differential equations. Numerical Algebra, Control and Optimization, 2022  doi: 10.3934/naco.2022012 [20] Alain Bensoussan, Jens Frehse, Jens Vogelgesang. Systems of Bellman equations to stochastic differential games with non-compact coupling. Discrete and Continuous Dynamical Systems, 2010, 27 (4) : 1375-1389. doi: 10.3934/dcds.2010.27.1375

Impact Factor: