\`x^2+y_1+z_12^34\`
Advanced Search
Article Contents
Article Contents

A risk minimization problem for finite horizon semi-Markov decision processes with loss rates

Research supported by Natural Science Foundation of Guangdong Province (Grant No.2014A030313438), Zhujiang New Star (Grant No. 201506010056) and Guangdong Province outstanding young teacher training plan (Grant No. YQ2015050)

Abstract Full Text(HTML) Figure(5) Related Papers Cited by
  • This paper deals with the risk probability for finite horizon semi-Markov decision processes with loss rates. The criterion to be minimized is the risk probability that the total loss incurred during a finite horizon exceed a loss level. For such an optimality problem, we first establish the optimality equation, and prove that the optimal value function is a unique solution to the optimality equation. We then show the existence of an optimal policy, and develop a value iteration algorithm for computing the value function and optimal policies. We also derive the approximation of the value function and the rules of iteration. Finally, a numerical example is given to illustrate our results.

    Mathematics Subject Classification: Primary: 90C40; Secondary: 93E20.

    Citation:

    \begin{equation} \\ \end{equation}
  • 加载中
  • Figure 1.  The function $F^*_{(n_0+1)k}(1, t, \lambda)$

    Figure 2.  The function $F^*_{(n_0+1)k}(2, t, \lambda)$

    Figure 3.  The function $H^aF^*_{(n_0+1)k-1}(i, 10, \lambda)$

    Figure 4.  The function $H^aF^*_{(n_0+1)k-1}(i, 15, \lambda)$

    Figure 5.  The function $\lambda^*(i, t)$

  •   N. Bauerle and U. Rieder, Markov Decision Processes with Application to Finance, Universitext, Springer, Heidelberg, 2011.
      M. Bouakiz  and  Y. Kebir , Target-level criterion in Markov decision processes, Journal of Optimization Theory and Applications, 86 (1995) , 1-15.  doi: 10.1007/BF02193458.
      M. K. Ghosh  and  S. Subhamay , Non-stationary semi-Markov secision processes on a finite horizon, Stochastic Analysis and Applications, 31 (2013) , 183-190.  doi: 10.1080/07362994.2013.741405.
      X. P. Guo and O. Hernández-Lerma, Continuous-Time Markov Decision Processes: Theory and Applications, Springer-Verlag, Berlin, 2009.
      X. P. Guo  and  J. Yang , A new condition and approach for zero-sum stochastic games with average payoffs, Stochastic Analysis and Applications, 26 (2008) , 537-561.  doi: 10.1080/07362990802007095.
      X. P. Guo , P. Shi  and  W. P. Zhu , Strong average optimality for controlled nonhomogeneous Markov chains, Stochastic Analysis and Applications, 19 (2001) , 115-134.  doi: 10.1081/SAP-100001186.
      O. Hernández-Lerma and J. B. Lasserre, Discrete-time Markov Control Processes, Basic optimality criteria, Springer-Verlag, New York, 1996.
      O. Hernández-Lerma and J. B. Lasserre, Further Topics on Discrete-Time Markov Control Processes, Springer-Verlag, New York, 1999.
      Y. H. Huang  and  X. P. Guo , Optimal risk probability for first passage models in semi-Markov decision processes, Journal Mathematical Analysis Applications, 359 (2009) , 404-420.  doi: 10.1016/j.jmaa.2009.05.058.
      Y. H. Huang , X. P. Guo  and  X. Y. Song , Performance analysis for controlled semi-Markov systems with application to maintenance, Journal of Optimization Theory and Applications, 150 (2011) , 395-415.  doi: 10.1007/s10957-011-9813-7.
      Y. H. Huang  and  X. P. Guo , Finite horizon semi-Markov decision processes with application to maintenance systems, European Journal Operations Research, 212 (2011) , 131-140.  doi: 10.1016/j.ejor.2011.01.027.
      Y. H. Huang , X. P. Guo  and  Z. F. Li , Minimum risk probability for finite horizon semi-Markov decision processes, Journal Mathematical Analysis Applications, 402 (2013) , 378-391.  doi: 10.1016/j.jmaa.2013.01.021.
      Y. H. Huang  and  X. P. Guo , Mean-variance problems for finite horizon semi-Markov decision processes, Applications Mathematical Optimization, 72 (2015) , 233-259.  doi: 10.1007/s00245-014-9278-9.
      N. Limnios and G. Oprisan, Semi-Markov Processes and Reliability, Birkhäuser Boston, Inc., Boston, MA, 2001.
      J. Y. Liu  and  S. M. Huang , Markov decision processes with distribution function criterion of first-passage time, Applications Mathematical Optimization, 43 (2001) , 187-201.  doi: 10.1007/s00245-001-0007-9.
      P. M. Madhani , Rebalancing fixed and variable pay in a sales organization: A business cycle perspective, Compensation Benefits Review, 42 (2010) , 179-189.  doi: 10.1177/0886368709359668.
      J. W. Mamer , Successive approximations for finite horizon semi-Markov decision processes with application to asset liquidation, Oper. Res., 34 (1986) , 638-644.  doi: 10.1287/opre.34.4.638.
      Y. Ohtsubo , Minimizing risk models in stochastic shortest path problems, Mathematical Methods of Operations Research, 57 (2003) , 79-88.  doi: 10.1007/s001860200246.
      Y. Ohtsubo , Optimal threshold probability in undiscounted Markov decision processes with a target set, Appl. Math. Comput., 149 (2004) , 519-532.  doi: 10.1016/S0096-3003(03)00158-9.
      M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, John Wiley & Sons, Inc., New York, 1994.
      C. Ruhm , Are recessions good for your health?, Quarterly Journal of Economics, 115 (2000) , 617-650.  doi: 10.3386/w5570.
      M. Sakaguchi  and  Y. Ohtsubo , Markov decision processes associated with two threshold probability criteria, Journal Control Theory Applications, 11 (2013) , 548-557.  doi: 10.1007/s11768-013-2194-8.
      Q. D. Wei  and  X. P. Guo , New average optimality conditions for semi-Markov decision processes in Borel spaces, Journal of Optimization Theory and Applications, 153 (2012) , 709-732.  doi: 10.1007/s10957-012-9986-8.
      D. J. White , Minimising a threshold probability in discounted Markov decision processes, J. Math. Anal. Appl., 173 (1993) , 634-646.  doi: 10.1006/jmaa.1993.1093.
      Y. H. Wu , Bounds for the ruin probability under a Markovian modulated risk model, Communications in statistics Stochastic Models, 15 (1999) , 125-136.  doi: 10.1080/15326349908807529.
      S. X. Yu , Y. L. Lin  and  P. F. Yan , Optimization models for the first arrival target distribution function in discrete time, J. Math. Anal. Appl., 225 (1998) , 193-223.  doi: 10.1006/jmaa.1998.6015.
  • 加载中

Figures(5)

SHARE

Article Metrics

HTML views(494) PDF downloads(228) Cited by(0)

Access History

Other Articles By Authors

  • on this site
  • on Google Scholar

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return