# American Institute of Mathematical Sciences

April  2018, 5(2): 143-163. doi: 10.3934/jdg.2018009

## A risk minimization problem for finite horizon semi-Markov decision processes with loss rates

 1 School of Mathematical Sciences, South China Normal University, Guangzhou 510631, China 2 School of Economics and Statistics, Guangzhou University, Guangzhou 510006, China

* Corresponding author. Emails: liuql@m.scnu.edu.cn; zouxiaol@gzhu.edu.cn

Received  June 2017 Revised  July 2017 Published  February 2018

Fund Project: Research supported by Natural Science Foundation of Guangdong Province (Grant No.2014A030313438), Zhujiang New Star (Grant No. 201506010056) and Guangdong Province outstanding young teacher training plan (Grant No. YQ2015050)

This paper deals with the risk probability for finite horizon semi-Markov decision processes with loss rates. The criterion to be minimized is the risk probability that the total loss incurred during a finite horizon exceed a loss level. For such an optimality problem, we first establish the optimality equation, and prove that the optimal value function is a unique solution to the optimality equation. We then show the existence of an optimal policy, and develop a value iteration algorithm for computing the value function and optimal policies. We also derive the approximation of the value function and the rules of iteration. Finally, a numerical example is given to illustrate our results.

Citation: Qiuli Liu, Xiaolong Zou. A risk minimization problem for finite horizon semi-Markov decision processes with loss rates. Journal of Dynamics & Games, 2018, 5 (2) : 143-163. doi: 10.3934/jdg.2018009
##### References:

show all references

##### References:
The function $F^*_{(n_0+1)k}(1, t, \lambda)$
The function $F^*_{(n_0+1)k}(2, t, \lambda)$
The function $H^aF^*_{(n_0+1)k-1}(i, 10, \lambda)$
The function $H^aF^*_{(n_0+1)k-1}(i, 15, \lambda)$
The function $\lambda^*(i, t)$
 [1] Mathias Staudigl. A limit theorem for Markov decision processes. Journal of Dynamics & Games, 2014, 1 (4) : 639-659. doi: 10.3934/jdg.2014.1.639 [2] A. Mittal, N. Hemachandra. Learning algorithms for finite horizon constrained Markov decision processes. Journal of Industrial & Management Optimization, 2007, 3 (3) : 429-444. doi: 10.3934/jimo.2007.3.429 [3] Gábor Horváth, Zsolt Saffer, Miklós Telek. Queue length analysis of a Markov-modulated vacation queue with dependent arrival and service processes and exhaustive service policy. Journal of Industrial & Management Optimization, 2017, 13 (3) : 1365-1381. doi: 10.3934/jimo.2016077 [4] Linyi Qian, Wei Wang, Rongming Wang. Risk-minimizing portfolio selection for insurance payment processes under a Markov-modulated model. Journal of Industrial & Management Optimization, 2013, 9 (2) : 411-429. doi: 10.3934/jimo.2013.9.411 [5] Lin Xu, Rongming Wang. Upper bounds for ruin probabilities in an autoregressive risk model with a Markov chain interest rate. Journal of Industrial & Management Optimization, 2006, 2 (2) : 165-175. doi: 10.3934/jimo.2006.2.165 [6] Ming Yan, Hongtao Yang, Lei Zhang, Shuhua Zhang. Optimal investment-reinsurance policy with regime switching and value-at-risk constraint. Journal of Industrial & Management Optimization, 2017, 13 (5) : 1-17. doi: 10.3934/jimo.2019050 [7] Jiaqin Wei, Zhuo Jin, Hailiang Yang. Optimal dividend policy with liability constraint under a hidden Markov regime-switching model. Journal of Industrial & Management Optimization, 2019, 15 (4) : 1965-1993. doi: 10.3934/jimo.2018132 [8] Vincent Renault, Michèle Thieullen, Emmanuel Trélat. Optimal control of infinite-dimensional piecewise deterministic Markov processes and application to the control of neuronal dynamics via Optogenetics. Networks & Heterogeneous Media, 2017, 12 (3) : 417-459. doi: 10.3934/nhm.2017019 [9] Zhimin Zhang. On a risk model with randomized dividend-decision times. Journal of Industrial & Management Optimization, 2014, 10 (4) : 1041-1058. doi: 10.3934/jimo.2014.10.1041 [10] Wael Bahsoun, Paweł Góra. SRB measures for certain Markov processes. Discrete & Continuous Dynamical Systems - A, 2011, 30 (1) : 17-37. doi: 10.3934/dcds.2011.30.17 [11] Artur Stephan, Holger Stephan. Memory equations as reduced Markov processes. Discrete & Continuous Dynamical Systems - A, 2019, 39 (4) : 2133-2155. doi: 10.3934/dcds.2019089 [12] Yinghui Dong, Guojing Wang. Ruin probability for renewal risk model with negative risk sums. Journal of Industrial & Management Optimization, 2006, 2 (2) : 229-236. doi: 10.3934/jimo.2006.2.229 [13] Kebing Chen, Tiaojun Xiao. Reordering policy and coordination of a supply chain with a loss-averse retailer. Journal of Industrial & Management Optimization, 2013, 9 (4) : 827-853. doi: 10.3934/jimo.2013.9.827 [14] H.Thomas Banks, Shuhua Hu. Nonlinear stochastic Markov processes and modeling uncertainty in populations. Mathematical Biosciences & Engineering, 2012, 9 (1) : 1-25. doi: 10.3934/mbe.2012.9.1 [15] Xian Chen, Zhi-Ming Ma. A transformation of Markov jump processes and applications in genetic study. Discrete & Continuous Dynamical Systems - A, 2014, 34 (12) : 5061-5084. doi: 10.3934/dcds.2014.34.5061 [16] A. M. Vershik. Polymorphisms, Markov processes, quasi-similarity. Discrete & Continuous Dynamical Systems - A, 2005, 13 (5) : 1305-1324. doi: 10.3934/dcds.2005.13.1305 [17] Emilija Bernackaitė, Jonas Šiaulys. The finite-time ruin probability for an inhomogeneous renewal risk model. Journal of Industrial & Management Optimization, 2017, 13 (1) : 207-222. doi: 10.3934/jimo.2016012 [18] M. A. Efendiev. On the compactness of the stable set for rate independent processes. Communications on Pure & Applied Analysis, 2003, 2 (4) : 495-509. doi: 10.3934/cpaa.2003.2.495 [19] T. J. Sullivan, M. Koslowski, F. Theil, Michael Ortiz. Thermalization of rate-independent processes by entropic regularization. Discrete & Continuous Dynamical Systems - S, 2013, 6 (1) : 215-233. doi: 10.3934/dcdss.2013.6.215 [20] Yang Yang, Kaiyong Wang, Jiajun Liu, Zhimin Zhang. Asymptotics for a bidimensional risk model with two geometric Lévy price processes. Journal of Industrial & Management Optimization, 2019, 15 (2) : 481-505. doi: 10.3934/jimo.2018053

Impact Factor: