Deep neural networks for probability of default modelling

Kyriakos Georgiou; Athanasios N. Yannacopoulos

doi:10.3934/jimo.2024038

Article Contents

2024, Volume 20, Issue 12: 3647-3677. Doi: 10.3934/jimo.2024038

This issue Previous Article The newsvendor problem with normal, worst-case and binomial distribution of demand: Managerial implications with examples Next Article A novel reinforcement learning based routing algorithm for energy management in networks

Deep neural networks for probability of default modelling

Kyriakos Georgiou^, and
Athanasios N. Yannacopoulos

Department of Statistics and Stochastic Modeling and Applications Laboratory, Athens University of Economics and Business, Greece

^*Corresponding author: Kyriakos Georgiou
^*Corresponding author: Kyriakos Georgiou

Received: December 2023

Revised: February 2024

Early access: March 15, 2024

Published online: March 15, 2024

Abstract / Introduction Full Text(HTML) Figure(13) / Table(3) Related Papers Cited by

Abstract

In this paper we develop Deep Neural Networks for the approximation of the solution to Partial Integro-Differential Equations (PIDE) that arise in the calculation of Probability of Default functions. We consider a modelling framework in compliance with the spirit and regulations of the International Financial Reporting Standard 9 and use the resulting Deep Learning models to estimate default probabilities that can be used to solve credit risk problems. Detailed comparisons with standard numerical analysis schemes for the solutions to these PIDEs are also reported, enhancing the understanding and adding to the discussion regarding the applicability of the related Machine Learning methodologies.

Keywords:

Mathematics Subject Classification: 60H30, 68T07, 45K05, 91G40, 91G60, 91-08.

Citation:

Full Text(HTML)

Figure 1. Fully connected Neural Network with an $ n $-dimensional input layer, $ k $ dense layers

Download: Full-size image PowerPoint slide

Figure 2. Graphical illustration of the proposed model

Download: Full-size image PowerPoint slide

Figure 3. Fully connected Neural Network with a one-dimensional input layer and a one-dimensional output layer

Download: Full-size image PowerPoint slide

Figure 4. Graphs of the PD as a function of the initial position for (top left) the DNN model with $ \Psi(x, 0) $, (top right) the DNN model with $ \Psi(x, \rho, 0) $ and $ \rho = 2 $, and (bottom) the DNN model with $ \Psi(x, y, 0) $ and $ y = 0.2 $

Download: Full-size image PowerPoint slide

Figure 5. Fully connected Neural Network with a two dimensional input layer, $ k $ hidden layers and a one dimensional output layer

Download: Full-size image PowerPoint slide

Figure 6. (Left) DNN model for $ \Phi(x, u) $ (Middle) DNN model for $ \Phi(x, \rho, u) $ with $ \rho = 2 $ (Right) DNN model for $ \Phi(x, y, u) $ with $ y = 2.0 $

Download: Full-size image PowerPoint slide

Figure 7. (Left) DNN model for $ \Phi(x, \rho, u) $ with $ \rho = 2 $. (Right) DNN model for $ \Psi(x, y, u) $ with $ y = 2.0 $

Download: Full-size image PowerPoint slide

Figure 8. Fully connected Neural Network with a four-dimensional input layer, $ k $ hidden layers and a one-dimensional output layer

Download: Full-size image PowerPoint slide

Figure 9. DNN model for $ \Psi(x, \rho, y, u) $ with $ \rho = 1 $ and $ y = 2.0 $

Download: Full-size image PowerPoint slide

Figure 10. Relative error of the survival probability from the NN models under the (top left) one-dimensional, (top right) regime switching and (bottom) stochastic volatility asset value model

Download: Full-size image PowerPoint slide

Figure 11. Absolute error of the survival probability from the NN models under the one-dimensional (top left), regime switching (top right) and stochastic volatility (bottom) asset value model

Download: Full-size image PowerPoint slide

Figure 12. Fully connected Neural Network with a 4-dimensional input layer and $ k $ hidden layers

Download: Full-size image PowerPoint slide

Figure 13. PD functions for various choices of asset value process parameters

Download: Full-size image PowerPoint slide

Table 1. Quantities of interest

Quantity (type)	Notation
Probability of Default (function)	$\Psi$
Probability of Survival (function)	$\Phi$
Initial position (variable)	$x$
Initial volatility (variable)	$y$
Initial regime (variable)	$\rho$
Asset value (random variable)	$G$
Input of model (5) (parameters)	$k, \theta, \sigma, \kappa, \mu, \xi$
Starting time (variable)	$t$
Maturity time (variable)	$T$

| Show Table

DownLoad: CSV

Table 2. Comparison of computational time required for FD and NN methods

Model	FD approximation time (seconds)	DNN training time (seconds)
One dimensional	209	66, 243
Regime switching	1, 141	96, 699
Stochastic volatility	154, 542	190, 734
Generalized	Intractable	194, 208

| Show Table

DownLoad: CSV

Table 3. Neural Network architectures for the three asset value models

Asset value model	Simulated paths	Layers	Neurons per layer
One dimensional	30, 000	5	10
Regime switching	50, 000	3	10
Stochastic volatility	50, 000	3	7
Generalized	60, 000	3	7

| Show Table

DownLoad: CSV

Related Papers

Cited by

References

[1]	P. M. Addo, D. Guegan and B. Hassani, Credit risk analysis using machine and deep learning models, Risks, 6 (2018), 38.
[2]	D. Applebaum, Lévy Processes and Stochastic Calculus, Cambridge university press, Cambridge, 2009. doi: 10.1017/CBO9780511809781.
[3]	C. Aristidou, The meta-phillips curve: Modelling us inflation in the presence of regime change, Journal of Macroeconomics, 57 (2018), 367-379. doi: 10.1016/j.jmacro.2018.07.002.
[4]	L. Ballotta and G. Fusai, Counterparty credit risk in a multivariate structural model with jumps, Finance, 36 (2015), 39-74.
[5]	O. E. Barndorff-Nielsen and N. Shephard, Modelling by lévy processess for financial econometrics, Lévy Processes, Birkhäuser Boston, Inc., Boston, MA, 2001,283-318.
[6]	O. E. Barndorff-Nielsen and N. Shephard, Non-gaussian ornstein–uhlenbeck-based models and some of their uses in financial economics, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 63 (2001), 167-241. doi: 10.1111/1467-9868.00282.
[7]	D. S. Bates, Jumps and stochastic volatility: Exchange rate processes implicit in deutsche mark options, The Review of Financial Studies, 9 (1996), 69-107. doi: 10.1093/rfs/9.1.69.
[8]	C. Beck, S. Becker, P. Grohs, N. Jaafari and A. Jentzen, Solving the kolmogorov pde by means of deep learning, Journal of Scientific Computing, 88 (2021), 1-28. doi: 10.1007/s10915-021-01590-0.
[9]	D. Beerbaum, Significant increase in credit risk according to ifrs 9: Implications for financial institutions, International Journal of Economics & Management Sciences, 4 (2015), 1-3. doi: 10.2139/ssrn.2654120.
[10]	E. Benhamou, E. Gobet and M. Miri, Time dependent heston model, SIAM Journal on Financial Mathematics, 1 (2010), 289-325. doi: 10.1137/090753814.
[11]	I. Bilbao and J. Bilbao, Overfitting problem and the over-training in the era of data: Particularly for artificial neural networks, In 2017 Eighth International Conference on Intelligent Computing and Information Systems (ICICIS), IEEE, 2017,173-177. doi: 10.1109/INTELCIS.2017.8260032.
[12]	J. Blechschmidt and O. G. Ernst, Three ways to solve partial differential equations with neural networks-a review, GAMM-Mitteilungen, 44 (2021), e202100006. doi: 10.1002/gamm.202100006.
[13]	V. Brattka., From Hilbert's 13th Problem to the theory of neural networks: Constructive aspects of Kolmogorov's Superposition Theorem, Kolmogorov's Heritage in Mathematics, Springer, Berlin, 2007,253-280. doi: 10.1007/978-3-540-36351-4_13.
[14]	S. Cai, Z. Wang, S. Wang, P. Perdikaris and G. E. Karniadakis, Physics-informed neural networks for heat transfer problems, Journal of Heat Transfer, 143 (2021), 060801. doi: 10.1115/1.4050542.
[15]	R. Caruana, S. Lawrence and C. Giles, Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping, Advances in Neural Information Processing Systems, 13 (2000).
[16]	T. Chen and H. Chen, Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems, IEEE Transactions on Neural Networks, 6 (1995), 911-917.
[17]	J. M. Clements, D. Xu, N. Yousefi and D. Efimov, Sequential deep learning for credit risk monitoring with tabular financial data, arXiv preprint, arXiv: 2012.15330, 2020.
[18]	R. Cont and E. Voltchkova, A finite difference scheme for option pricing in jump diffusion and exponential lévy models, SIAM Journal on Numerical Analysis, 43 (2005), 1596-1626. doi: 10.1137/S0036142903436186.
[19]	J. C. Cox, J. E. Ingersoll Jr and S. A. Ross, A theory of the term structure of interest rates, In Theory of Valuation, World Scientific, (2005), 129-164. doi: 10.1142/9789812701022_0005.
[20]	S. Cuomo, V. Schiano Di Cola, F. Giampaolo, G. Rozza, M. Raissi and F. Piccialli, Scientific machine learning through physics–informed neural networks: Where we are and what's next, Journal of Scientific Computing, 92 (2022), Paper No. 88, 62 pp. doi: 10.1007/s10915-022-01939-z.
[21]	J.-C. Duan, I. Popova and P. Ritchken, Option pricing under regime switching, Quantitative Finance, 2 (2002), 116-132. doi: 10.1088/1469-7688/2/2/303.
[22]	K. Eggensperger, M. Feurer, F. Hutter, J. Bergstra, J. Snoek, H. Hoos, K. Leyton-Brown, et al., Towards an empirical foundation for assessing bayesian optimization of hyperparameters, In NIPS Workshop on Bayesian Optimization in Theory and Practice, 10 (2013).
[23]	A. Fadlalla and C.-H. Lin, An analysis of the applications of neural networks in finance, Interfaces, 31 (2001), 112-122. doi: 10.1287/inte.31.4.112.9662.
[24]	R. Frey and V. Köck, Deep neural network algorithms for parabolic pides and applications in insurance mathematics, In Mathematical and Statistical Methods for Actuarial Sciences and Finance: MAF 2022, Springer, (2022), 272-277. doi: 10.1007/978-3-030-99638-3_44.
[25]	K. Georgiou, G. N. Domazakis, D. Pappas and A. N. Yannacopoulos, Markov chain lumpability and applications to credit risk modelling in compliance with the international financial reporting standard 9 framework, European Journal of Operational Research, 292 (2021), 1146-1164. doi: 10.1016/j.ejor.2020.11.014.
[26]	K. Georgiou and A. N. Yannacopoulos, Probability of default modelling with l$\backslash$'evy-driven ornstein-uhlenbeck processes and applications in credit risk under the ifrs 9, arXiv preprint, arXiv: 2309.12384, 2023.
[27]	S. Goutte, Pricing and hedging in stochastic volatility regime switching models, Journal of Mathematical Finance, 3 (2013), 70-80. doi: 10.4236/jmf.2013.31006.
[28]	S. Goutte and A. Ngoupeyou, Conditional laplace formula in regime switching model: Application to defaultable bond, Preprint, 2011.
[29]	T. Guo, J. Dong, H. Li and Y. Gao, Simple convolutional neural network on image classification, In 2017 IEEE 2nd International Conference on Big Data Analysis (ICBDA), IEEE, (2017), 721-724.
[30]	D. Hainaut, Financial modeling with switching lévy processes, ESC, 2011.
[31]	D. Hainaut and O. L. Courtois, An intensity model for credit risk with switching lévy processes, Quantitative Finance, 14 (2014), 1453-1465. doi: 10.1080/14697688.2012.756583.
[32]	J. D. Hamilton, Regime switching models, In Macroeconometrics and time series analysis, Springer, (2010), 202-209.
[33]	S. Hayou, A. Doucet and J. Rousseau, On the selection of initialization and activation function for deep neural networks, arXiv preprint, arXiv: 1805.08266, 2018.
[34]	R. Hecht-Nielsen, Theory of the backpropagation neural network, In Neural Networks for Perception, Elsevier, (1992), 65-93. doi: 10.1109/IJCNN.1989.118638.
[35]	S. Hijazi, R. Kumar, C. Rowen, et al., Using convolutional neural networks for image recognition, Cadence Design Systems Inc.: San Jose, CA, USA, 9 (2015), 1.
[36]	A. Khashman, Neural networks for credit risk evaluation: Investigation of different neural models and learning schemes., Expert Systems with Applications, 37 (2010), 6233-6239. doi: 10.1016/j.eswa.2010.02.101.
[37]	K. Kumar and G. S. M. Thakur, Advanced applications of neural networks and artificial intelligence: A review, International journal of information technology and computer science, 4 (2012), 57. doi: 10.5815/ijitcs.2012.06.08.
[38]	A. E. Kyprianou, Introductory Lectures on Fluctuations of Lévy Processes with Applications, Springer Science & Business Media, 2006.
[39]	L. Liao, H. Li, W. Shang and L. Ma, An empirical study of the impact of hyperparameter tuning and model optimization on the performance properties of deep neural networks, ACM Transactions on Software Engineering and Methodology (TOSEM), 31 (2022), 1-40.
[40]	R. Lotfi, A. Gholamrezaei, M. Kadłubek, M. Afshar, S. S. Ali and K. Kheiri, A robust and resilience machine learning for forecasting agri-food production, Scientific Reports, 12 (2022), 21787. doi: 10.1038/s41598-022-26449-8.
[41]	R. Lotfi, K. Kheiri, A. Sadeghi and E. B. Tirkolaee, An extended robust mathematical model to project the course of covid-19 epidemic in iran, Annals of Operations Research, (2022), 1-25. doi: 10.1007/s10479-021-04490-6.
[42]	L. Lu, P. Jin, G. Pang, Z. Zhang and G. E. Karniadakis, Learning nonlinear operators via deeponet based on the universal approximation theorem of operators, Nature Machine Intelligence, 3 (2021), 218-229. doi: 10.1038/s42256-021-00302-5.
[43]	E. Luciano and W. Schoutens, A multivariate jump-driven financial asset model, Quantitative Finance, 6 (2006), 385-402. doi: 10.1080/14697680600806275.
[44]	L. R. Medsker and L. C. Jain, Recurrent neural networks, Design and Applications, 5 (2001), 64-67.
[45]	Y. Mishura and O. Ragulina, Ruin Probabilities: Smoothness, Bounds, Supermartingale Approach, ISTE Press, London; Elsevier, Inc., Oxford, 2016.
[46]	C. M. Møller, Stochastic differential equations for ruin probabilities, Journal of Applied Probability, 32 (1995), 74-89. doi: 10.2307/3214922.
[47]	B. K. Øksendal and A. Sulem, Applied Stocstic Control of Jump Diffusions, volume 498., Springer, Berlin, 2007. doi: 10.1007/978-3-540-69826-5.
[48]	O. Onalan, Financial modelling with ornstein-uhlenbeck processes driven by lévy process, Proceedings of the World Congress on Engineering, 2 (2009), 1-3.
[49]	G. Pang, L. Lu and G. E. Karniadakis, fpinns: Fractional physics-informed neural networks, SIAM Journal on Scientific Computing, 41 (2019), A2603-A2626. doi: 10.1137/18M1229845.
[50]	K. Papik, B. Molnar, R. Schaefer, Z. Dombovari, Z. Tulassay and J. Feher, Application of neural networks in medicine-a review, Med Sci Monit, 4 (1998), 538-546.
[51]	H. Pratiwi, A. P. Windarto, S. Susliansyah, R. R. Aria, S. Susilowati, L. K. Rahayu, Y. Fitriani, A. Merdekawati and I. R. Rahadjeng, Sigmoid activation function in selecting the best model of artificial neural networks, In Journal of Physics: Conference Series, IOP Publishing, 1471 (2020), 012010. doi: 10.1088/1742-6596/1471/1/012010.
[52]	M. Raissi, P. Perdikaris and G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, Journal of Computational Physics, 378 (2019), 686-707. doi: 10.1016/j.jcp.2018.10.045.
[53]	E. Savku and G.-W. Weber, A regime-switching model with applications to finance: Markovian and non-markovian cases, Dynamic Economic Problems with Regime Switches, 25 (2021), 287-309. doi: 10.1007/978-3-030-54576-5_13.
[54]	E. Savku and G.-W. Weber, Stochastic differential games for optimal investment problems in a markov regime-switching jump-diffusion market, Annals of Operations Research, 312 (2022), 1171-1196. doi: 10.1007/s10479-020-03768-5.
[55]	W. Schoutens and J. Cariboni, Lévy Processes in Credit Risk, volume 519., John Wiley & Sons, 2010.
[56]	F. Shen, X. Zhao, G. Kou and F. E. Alsaadi, A new deep learning ensemble credit risk evaluation model with an improved synthetic minority oversampling technique, Applied Soft Computing, 98 (2021), 106852. doi: 10.1016/j.asoc.2020.106852.
[57]	N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever and R. Salakhutdinov, Dropout: A simple way to prevent neural networks from overfitting, The Journal of Machine Learning Research, 15 (2014), 1929-1958.
[58]	I. Sutskever, J. Martens and G. E. Hinton, Generating text with recurrent neural networks, In Proceedings of the 28th International Conference on Machine Learning (ICML-11), (2011), 1017-1024.
[59]	J. Toivanen, A componentwise splitting method for pricing american options under the bates model, In Applied and Numerical Partial Differential Equations, Springer, 15 (2010), 213-227. doi: 10.1007/978-90-481-3239-3_16.
[60]	O. Vasicek, An equilibrium characterization of the term structure, Journal of Financial Economics, 5 (1977), 177-188. doi: 10.1002/9781119186229.ch6.
[61]	X. Xu, Estimating lifetime expected credit losses under ifrs 9, Available at SSRN 2758513, 2016.
[62]	C. Zhou, A jump-diffusion approach to modeling credit risk and valuing defaultable securities, Available at SSRN 39800, 1997.
[63]	C. Zhu, G. Yin and N. A. Baran, Feynman–kac formulas for regime-switching jump diffusions and their applications, Stochastics An International Journal of Probability and Stochastic Processes, 87 (2015), 1000-1032. doi: 10.1080/17442508.2015.1019884.