July  2007, 3(3): 429-444. doi: 10.3934/jimo.2007.3.429

Learning algorithms for finite horizon constrained Markov decision processes

1. 

IE and OR Interdisciplinary Programme, IIT Bombay, Mumbai, 400 076, India, India

Received  April 2006 Revised  November 2006 Published  July 2007

We propose a heuristic and two stochastic approximation based learning algorithms for finite horizon, finite state-action constrained Markov decision models. We include models and numerical examples arising from risk management in fund allocation, retailer-depot product availability in a supply chain and admission control in a simple queue, that have to satisfy performance based constraints.
Citation: A. Mittal, N. Hemachandra. Learning algorithms for finite horizon constrained Markov decision processes. Journal of Industrial and Management Optimization, 2007, 3 (3) : 429-444. doi: 10.3934/jimo.2007.3.429
[1]

Mathias Staudigl. A limit theorem for Markov decision processes. Journal of Dynamics and Games, 2014, 1 (4) : 639-659. doi: 10.3934/jdg.2014.1.639

[2]

Qiuli Liu, Xiaolong Zou. A risk minimization problem for finite horizon semi-Markov decision processes with loss rates. Journal of Dynamics and Games, 2018, 5 (2) : 143-163. doi: 10.3934/jdg.2018009

[3]

K. F. Cedric Yiu, S. Y. Wang, K. L. Mak. Optimal portfolios under a value-at-risk constraint with applications to inventory control in supply chains. Journal of Industrial and Management Optimization, 2008, 4 (1) : 81-94. doi: 10.3934/jimo.2008.4.81

[4]

K. F. C. Yiu, L. L. Xie, K. L. Mak. Analysis of bullwhip effect in supply chains with heterogeneous decision models. Journal of Industrial and Management Optimization, 2009, 5 (1) : 81-94. doi: 10.3934/jimo.2009.5.81

[5]

Yongtao Peng, Dan Xu, Eleonora Veglianti, Elisabetta Magnaghi. A product service supply chain network equilibrium considering risk management in the context of COVID-19 pandemic. Journal of Industrial and Management Optimization, 2022  doi: 10.3934/jimo.2022094

[6]

Linyi Qian, Wei Wang, Rongming Wang. Risk-minimizing portfolio selection for insurance payment processes under a Markov-modulated model. Journal of Industrial and Management Optimization, 2013, 9 (2) : 411-429. doi: 10.3934/jimo.2013.9.411

[7]

Salah Eddine Choutri, Boualem Djehiche, Hamidou Tembine. Optimal control and zero-sum games for Markov chains of mean-field type. Mathematical Control and Related Fields, 2019, 9 (3) : 571-605. doi: 10.3934/mcrf.2019026

[8]

Vincent Renault, Michèle Thieullen, Emmanuel Trélat. Optimal control of infinite-dimensional piecewise deterministic Markov processes and application to the control of neuronal dynamics via Optogenetics. Networks and Heterogeneous Media, 2017, 12 (3) : 417-459. doi: 10.3934/nhm.2017019

[9]

Ciro D'Apice, Peter I. Kogut, Rosanna Manzo. On relaxation of state constrained optimal control problem for a PDE-ODE model of supply chains. Networks and Heterogeneous Media, 2014, 9 (3) : 501-518. doi: 10.3934/nhm.2014.9.501

[10]

Ciro D'Apice, Rosanna Manzo. A fluid dynamic model for supply chains. Networks and Heterogeneous Media, 2006, 1 (3) : 379-398. doi: 10.3934/nhm.2006.1.379

[11]

Xiaoli Yan, Hui Yu. Channel structures of transnational supply chains. Journal of Industrial and Management Optimization, 2022  doi: 10.3934/jimo.2022131

[12]

Andrew P. Sage. Risk in system of systems engineering and management. Journal of Industrial and Management Optimization, 2008, 4 (3) : 477-487. doi: 10.3934/jimo.2008.4.477

[13]

Wenyuan Wang, Ran Xu. General drawdown based dividend control with fixed transaction costs for spectrally negative Lévy risk processes. Journal of Industrial and Management Optimization, 2022, 18 (2) : 795-823. doi: 10.3934/jimo.2020179

[14]

Wael Bahsoun, Paweł Góra. SRB measures for certain Markov processes. Discrete and Continuous Dynamical Systems, 2011, 30 (1) : 17-37. doi: 10.3934/dcds.2011.30.17

[15]

Artur Stephan, Holger Stephan. Memory equations as reduced Markov processes. Discrete and Continuous Dynamical Systems, 2019, 39 (4) : 2133-2155. doi: 10.3934/dcds.2019089

[16]

Felix X.-F. Ye, Yue Wang, Hong Qian. Stochastic dynamics: Markov chains and random transformations. Discrete and Continuous Dynamical Systems - B, 2016, 21 (7) : 2337-2361. doi: 10.3934/dcdsb.2016050

[17]

Yakov Pesin. On the work of Sarig on countable Markov chains and thermodynamic formalism. Journal of Modern Dynamics, 2014, 8 (1) : 1-14. doi: 10.3934/jmd.2014.8.1

[18]

Zhimin Zhang. On a risk model with randomized dividend-decision times. Journal of Industrial and Management Optimization, 2014, 10 (4) : 1041-1058. doi: 10.3934/jimo.2014.10.1041

[19]

Gabriella Bretti, Ciro D’Apice, Rosanna Manzo, Benedetto Piccoli. A continuum-discrete model for supply chains dynamics. Networks and Heterogeneous Media, 2007, 2 (4) : 661-694. doi: 10.3934/nhm.2007.2.661

[20]

Yang Peng, Xiaoming Yan, Ang Zhou, Tai Chiu Edwin Cheng, Min Ji. Entry game in supply chains with yield uncertainty. Journal of Industrial and Management Optimization, 2022  doi: 10.3934/jimo.2022158

2021 Impact Factor: 1.411

Metrics

  • PDF downloads (151)
  • HTML views (0)
  • Cited by (1)

Other articles
by authors

[Back to Top]