January  2020, 5: 4 doi: 10.1186/s41546-020-00046-x

Uncertainty and filtering of hidden Markov models in discrete time

Mathematical Institute, University of Oxford, Woodstock Road, Oxford, UK

Received  June 13, 2018 Published  June 2020

We consider the problem of filtering an unseen Markov chain from noisy observations, in the presence of uncertainty regarding the parameters of the processes involved. Using the theory of nonlinear expectations, we describe the uncertainty in terms of a penalty function, which can be propagated forward in time in the place of the filter. We also investigate a simple control problem in this context.
Citation: Samuel N. Cohen. Uncertainty and filtering of hidden Markov models in discrete time. Probability, Uncertainty and Quantitative Risk, 2020, 5 (0) : 4-. doi: 10.1186/s41546-020-00046-x
References:
[1]

Allan, A.L. and S.N. Cohen. (2019a). Parameter uncertainty in the Kalman–Bucy filter, SIAM J. Control Optim. 57, no. 3, 1646–1671.,

[2]

Allan, A.L. and S.N. Cohen. (2020). Pathwise Stochastic Control with Applications to Robust Filtering, Ann. Appl. Prob. arXiv::1902.05434.,

[3]

Artzner, P., F. Delbaen, J.-M. Eber, and D. Heath. (1999). Coherent measures of risk, Math. Finan. 9, no. 3, 203–228.,

[4]

Başar, T. and P. Bernhard. (1991). H-Optimal Control and Related Minimax Design Problems, A Dynamic Game Approach, Birkhäuser, Basel.,

[5]

Bain, A. and D. Crisan. (2009). Fundamentals of Stochastic Filtering, Springer, Berlin–Heidelberg–New York.,

[6]

Bielecki, T.R., T. Chen, and I. Cialenco. (2017). Recursive construction of confidence regions, Electron. J. Stat. 11, no. 2, 4674–4700.,

[7]

Boel, R.K., M.R. James, and I.R. Petersen. (2002). Robustness and risk-sensitive filtering, IEEE Trans. Autom. Control 47, no. 3, 451–461.,

[8]

Cohen, S.N. and R.J. Elliott. (2010). A general theory of finite state backward stochastic difference equations, Stoch. Process. Appl. 120, no. 4, 442–466.,

[9]

Cohen, S.N. and R.J. Elliott. (2011). Backward stochastic difference equations and nearly-time-consistent nonlinear expectations, SIAM J. Control Optim. 49, no. 1, 125–139.,

[10]

Cohen, S.N. and R.J. Elliott. (2015). Stochastic Calculus and Applications, 2nd ed., Birkhäuser, New York.,

[11]

Cohen, S.N. (2017). Data-driven nonlinear expectations for statistical uncertainty in decisions, Electron. J. Stat. 11, no. 1, 1858–1889.,

[12]

Delbaen, F., S. Peng, and E. Rosazza Gianin. (2010). Representation of the penalty term of dynamic concave utilities, Finan. Stochast. 14, no. 3, 449–472.,

[13]

Dey, S. and J.B. Moore. (1995). Risk-sensitive filtering and smoothing for hidden Markov models, Syst. Control Lett. 25, 361–366.,

[14]

Douc, R., E. Moulines, J. Olsson, and R. van Handel. (2011). Consistency of the maximum likelihood estimator for general hidden Markov models, Ann. Stat. 39, no. 1, 474–513.,

[15]

Duffie, D. and L.G. Epstein. (1992). Asset pricing with stochastic differential utility, Rev. Finan. Stud. 5, no. 3, 411–436.,

[16]

El Karoui, N., S. Peng, and M.C. Quenez. (1997). Backward stochastic differential equations in finance, Math. Finan. 7, no. 1, 1–71.,

[17]

Epstein, L.G. and M. Schneider. (2003). Recursive multiple-priors, J. Econ. Theory 113, 1–31.,

[18]

Fagin, R. and J. Halpern. (1990). A new approach to updating beliefs, AUAI Press, Corvallis.,

[19]

Föllmer, H. and A. Schied. (2002a). Convex measures of risk and trading constraints, Finan. Stochast. 6, 429–447.,

[20]

Föllmer, H. and A. Schied. (2002b). Stochastic Finance: An Introduction in Discrete Time. Studies in Mathematics 27, de Gruyter, Berlin-New York.,

[21]

Frittelli, M. and E. Rosazza Gianin. (2002). Putting order in risk measures, J. Bank. Financ. 26, no. 7, 1473–1486.,

[22]

Graf, S. (1980). A Radon–Nikodym theorem for capacities, J. für die reine und angewandte Mathematik 320, 192–214.,

[23]

Grimble, M.J. and A. El Sayed. (1990). Solution of the H∞ optimal linear filtering problem for discretetime systems, Trans. Acoust. Speech Sig. Process. IEEE 38, no. 7.,

[24]

Hansen, L.P. and T.J. Sargent. (2005). Robust estimation and control under commitment, J. Econ. Theory 124, 258–301.,

[25]

Hansen, L.P. and T.J. Sargent. (2007). Recursive robust estimation and control without commitment, J. Econ. Theory 136, no. 1, 1–27.,

[26]

Hansen, L.P. and T.J. Sargent. (2008). Robustness, Princeton University Press, Princeton.,

[27]

Huber, P.J. and E.M. Roncetti. (2009). Robust Statistics, 2nd edn., Wiley, Hoboken.,

[28]

James, M.R., J.S. Baras, and R.J. Elliott. (1994). Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems, Trans. Autom. Control IEEE 39, no. 4, 780–792. https://doi.org/10.1109/9.286253.,

[29]

Kalman, R.E. (1960). A new approach to linear filtering and prediction problems, J. Basic Eng. ASME 82, 33–45.,

[30]

Kalman, R.E. and R.S. Bucy. (1961). New results in linear filtering and prediction theory, J. Basic Eng. ASME 83, 95–108.,

[31]

Keynes, J.M. (1921). A Treatise on Probability, Macmillan and Co., New York. Reprint BN Publishing, 2008.,

[32]

Knight, F.H. (1921). Risk, Uncertainty and Profit, Houghton Mifflin, Boston. reprint Dover 2006.,

[33]

Kupper, M. and W. Schachermayer. (2009). Representation results for law invariant time consistent functions, Math. Financ. Econ. 2, no. 3, 189–210.,

[34]

Leroux, B.G. (1992). Maximum-likelihood estimation for hidden Markov models, Stoch. Process. Appl. 40, 127–143.,

[35]

Peng, S. (2010). Nonlinear Expectations and Stochastic Calculus under Uncertainty, arxiv::1002.4546v1.,

[36]

Riedel, F. (2004). Dynamic coherent risk measures, Stochast. Process. Appl. 112, no. 2, 185–200.,

[37]

Rockafellar, R.T., S. Uryasev, and M. Zabarankin. (2006). Generalized deviations in risk analysis, Finan. Stochast. 10, 51–74.,

[38]

Wald, A. (1945). Statistical decision functions which minimize the maximum risk, Ann. Math. 46, no. 2, 265–280.,

[39]

Walley, P. (1991). Statistical Reasoning with Imprecise Probabilities, Chapman and Hall, London.,

[40]

Wonham, W.N. (1965). Some applications of stochastic differential equations to optimal nonlinear filtering, SIAM J. Control 2, 347–369.,

[41]

Zhang, J., Y. Xia, and P. Shi. (2009). Parameter-dependent robust H∞ filtering for uncertain discrete-time systems, Automatica 45, 560–565.,

show all references

References:
[1]

Allan, A.L. and S.N. Cohen. (2019a). Parameter uncertainty in the Kalman–Bucy filter, SIAM J. Control Optim. 57, no. 3, 1646–1671.,

[2]

Allan, A.L. and S.N. Cohen. (2020). Pathwise Stochastic Control with Applications to Robust Filtering, Ann. Appl. Prob. arXiv::1902.05434.,

[3]

Artzner, P., F. Delbaen, J.-M. Eber, and D. Heath. (1999). Coherent measures of risk, Math. Finan. 9, no. 3, 203–228.,

[4]

Başar, T. and P. Bernhard. (1991). H-Optimal Control and Related Minimax Design Problems, A Dynamic Game Approach, Birkhäuser, Basel.,

[5]

Bain, A. and D. Crisan. (2009). Fundamentals of Stochastic Filtering, Springer, Berlin–Heidelberg–New York.,

[6]

Bielecki, T.R., T. Chen, and I. Cialenco. (2017). Recursive construction of confidence regions, Electron. J. Stat. 11, no. 2, 4674–4700.,

[7]

Boel, R.K., M.R. James, and I.R. Petersen. (2002). Robustness and risk-sensitive filtering, IEEE Trans. Autom. Control 47, no. 3, 451–461.,

[8]

Cohen, S.N. and R.J. Elliott. (2010). A general theory of finite state backward stochastic difference equations, Stoch. Process. Appl. 120, no. 4, 442–466.,

[9]

Cohen, S.N. and R.J. Elliott. (2011). Backward stochastic difference equations and nearly-time-consistent nonlinear expectations, SIAM J. Control Optim. 49, no. 1, 125–139.,

[10]

Cohen, S.N. and R.J. Elliott. (2015). Stochastic Calculus and Applications, 2nd ed., Birkhäuser, New York.,

[11]

Cohen, S.N. (2017). Data-driven nonlinear expectations for statistical uncertainty in decisions, Electron. J. Stat. 11, no. 1, 1858–1889.,

[12]

Delbaen, F., S. Peng, and E. Rosazza Gianin. (2010). Representation of the penalty term of dynamic concave utilities, Finan. Stochast. 14, no. 3, 449–472.,

[13]

Dey, S. and J.B. Moore. (1995). Risk-sensitive filtering and smoothing for hidden Markov models, Syst. Control Lett. 25, 361–366.,

[14]

Douc, R., E. Moulines, J. Olsson, and R. van Handel. (2011). Consistency of the maximum likelihood estimator for general hidden Markov models, Ann. Stat. 39, no. 1, 474–513.,

[15]

Duffie, D. and L.G. Epstein. (1992). Asset pricing with stochastic differential utility, Rev. Finan. Stud. 5, no. 3, 411–436.,

[16]

El Karoui, N., S. Peng, and M.C. Quenez. (1997). Backward stochastic differential equations in finance, Math. Finan. 7, no. 1, 1–71.,

[17]

Epstein, L.G. and M. Schneider. (2003). Recursive multiple-priors, J. Econ. Theory 113, 1–31.,

[18]

Fagin, R. and J. Halpern. (1990). A new approach to updating beliefs, AUAI Press, Corvallis.,

[19]

Föllmer, H. and A. Schied. (2002a). Convex measures of risk and trading constraints, Finan. Stochast. 6, 429–447.,

[20]

Föllmer, H. and A. Schied. (2002b). Stochastic Finance: An Introduction in Discrete Time. Studies in Mathematics 27, de Gruyter, Berlin-New York.,

[21]

Frittelli, M. and E. Rosazza Gianin. (2002). Putting order in risk measures, J. Bank. Financ. 26, no. 7, 1473–1486.,

[22]

Graf, S. (1980). A Radon–Nikodym theorem for capacities, J. für die reine und angewandte Mathematik 320, 192–214.,

[23]

Grimble, M.J. and A. El Sayed. (1990). Solution of the H∞ optimal linear filtering problem for discretetime systems, Trans. Acoust. Speech Sig. Process. IEEE 38, no. 7.,

[24]

Hansen, L.P. and T.J. Sargent. (2005). Robust estimation and control under commitment, J. Econ. Theory 124, 258–301.,

[25]

Hansen, L.P. and T.J. Sargent. (2007). Recursive robust estimation and control without commitment, J. Econ. Theory 136, no. 1, 1–27.,

[26]

Hansen, L.P. and T.J. Sargent. (2008). Robustness, Princeton University Press, Princeton.,

[27]

Huber, P.J. and E.M. Roncetti. (2009). Robust Statistics, 2nd edn., Wiley, Hoboken.,

[28]

James, M.R., J.S. Baras, and R.J. Elliott. (1994). Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems, Trans. Autom. Control IEEE 39, no. 4, 780–792. https://doi.org/10.1109/9.286253.,

[29]

Kalman, R.E. (1960). A new approach to linear filtering and prediction problems, J. Basic Eng. ASME 82, 33–45.,

[30]

Kalman, R.E. and R.S. Bucy. (1961). New results in linear filtering and prediction theory, J. Basic Eng. ASME 83, 95–108.,

[31]

Keynes, J.M. (1921). A Treatise on Probability, Macmillan and Co., New York. Reprint BN Publishing, 2008.,

[32]

Knight, F.H. (1921). Risk, Uncertainty and Profit, Houghton Mifflin, Boston. reprint Dover 2006.,

[33]

Kupper, M. and W. Schachermayer. (2009). Representation results for law invariant time consistent functions, Math. Financ. Econ. 2, no. 3, 189–210.,

[34]

Leroux, B.G. (1992). Maximum-likelihood estimation for hidden Markov models, Stoch. Process. Appl. 40, 127–143.,

[35]

Peng, S. (2010). Nonlinear Expectations and Stochastic Calculus under Uncertainty, arxiv::1002.4546v1.,

[36]

Riedel, F. (2004). Dynamic coherent risk measures, Stochast. Process. Appl. 112, no. 2, 185–200.,

[37]

Rockafellar, R.T., S. Uryasev, and M. Zabarankin. (2006). Generalized deviations in risk analysis, Finan. Stochast. 10, 51–74.,

[38]

Wald, A. (1945). Statistical decision functions which minimize the maximum risk, Ann. Math. 46, no. 2, 265–280.,

[39]

Walley, P. (1991). Statistical Reasoning with Imprecise Probabilities, Chapman and Hall, London.,

[40]

Wonham, W.N. (1965). Some applications of stochastic differential equations to optimal nonlinear filtering, SIAM J. Control 2, 347–369.,

[41]

Zhang, J., Y. Xia, and P. Shi. (2009). Parameter-dependent robust H∞ filtering for uncertain discrete-time systems, Automatica 45, 560–565.,

[1]

Diana Keller. Optimal control of a linear stochastic Schrödinger equation. Conference Publications, 2013, 2013 (special) : 437-446. doi: 10.3934/proc.2013.2013.437

[2]

Paula A. González-Parra, Sunmi Lee, Leticia Velázquez, Carlos Castillo-Chavez. A note on the use of optimal control on a discrete time model of influenza dynamics. Mathematical Biosciences & Engineering, 2011, 8 (1) : 183-197. doi: 10.3934/mbe.2011.8.183

[3]

Luke Finlay, Vladimir Gaitsgory, Ivan Lebedev. Linear programming solutions of periodic optimization problems: approximation of the optimal control. Journal of Industrial & Management Optimization, 2007, 3 (2) : 399-413. doi: 10.3934/jimo.2007.3.399

[4]

Xiaohong Li, Mingxin Sun, Zhaohua Gong, Enmin Feng. Multistage optimal control for microbial fed-batch fermentation process. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021040

[5]

John T. Betts, Stephen Campbell, Claire Digirolamo. Examination of solving optimal control problems with delays using GPOPS-Ⅱ. Numerical Algebra, Control & Optimization, 2021, 11 (2) : 283-305. doi: 10.3934/naco.2020026

[6]

Shanjian Tang, Fu Zhang. Path-dependent optimal stochastic control and viscosity solution of associated Bellman equations. Discrete & Continuous Dynamical Systems - A, 2015, 35 (11) : 5521-5553. doi: 10.3934/dcds.2015.35.5521

[7]

Yves Dumont, Frederic Chiroleu. Vector control for the Chikungunya disease. Mathematical Biosciences & Engineering, 2010, 7 (2) : 313-345. doi: 10.3934/mbe.2010.7.313

[8]

Y. Latushkin, B. Layton. The optimal gap condition for invariant manifolds. Discrete & Continuous Dynamical Systems - A, 1999, 5 (2) : 233-268. doi: 10.3934/dcds.1999.5.233

[9]

J. Frédéric Bonnans, Justina Gianatti, Francisco J. Silva. On the convergence of the Sakawa-Shindo algorithm in stochastic control. Mathematical Control & Related Fields, 2016, 6 (3) : 391-406. doi: 10.3934/mcrf.2016008

[10]

Martin Bohner, Sabrina Streipert. Optimal harvesting policy for the Beverton--Holt model. Mathematical Biosciences & Engineering, 2016, 13 (4) : 673-695. doi: 10.3934/mbe.2016014

[11]

Xingchun Wang, Yongjin Wang. Variance-optimal hedging for target volatility options. Journal of Industrial & Management Optimization, 2014, 10 (1) : 207-218. doi: 10.3934/jimo.2014.10.207

[12]

Alberto Bressan, Ke Han, Franco Rampazzo. On the control of non holonomic systems by active constraints. Discrete & Continuous Dynamical Systems - A, 2013, 33 (8) : 3329-3353. doi: 10.3934/dcds.2013.33.3329

[13]

Yanqin Fang, Jihui Zhang. Multiplicity of solutions for the nonlinear Schrödinger-Maxwell system. Communications on Pure & Applied Analysis, 2011, 10 (4) : 1267-1279. doi: 10.3934/cpaa.2011.10.1267

[14]

Deren Han, Zehui Jia, Yongzhong Song, David Z. W. Wang. An efficient projection method for nonlinear inverse problems with sparsity constraints. Inverse Problems & Imaging, 2016, 10 (3) : 689-709. doi: 10.3934/ipi.2016017

[15]

Olena Naboka. On synchronization of oscillations of two coupled Berger plates with nonlinear interior damping. Communications on Pure & Applied Analysis, 2009, 8 (6) : 1933-1956. doi: 10.3934/cpaa.2009.8.1933

[16]

Jiangxing Wang. Convergence analysis of an accurate and efficient method for nonlinear Maxwell's equations. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2429-2440. doi: 10.3934/dcdsb.2020185

[17]

Manoel J. Dos Santos, Baowei Feng, Dilberto S. Almeida Júnior, Mauro L. Santos. Global and exponential attractors for a nonlinear porous elastic system with delay term. Discrete & Continuous Dynamical Systems - B, 2021, 26 (5) : 2805-2828. doi: 10.3934/dcdsb.2020206

[18]

Thierry Cazenave, Ivan Naumkin. Local smooth solutions of the nonlinear Klein-gordon equation. Discrete & Continuous Dynamical Systems - S, 2021, 14 (5) : 1649-1672. doi: 10.3934/dcdss.2020448

[19]

Scipio Cuccagna, Masaya Maeda. A survey on asymptotic stability of ground states of nonlinear Schrödinger equations II. Discrete & Continuous Dynamical Systems - S, 2021, 14 (5) : 1693-1716. doi: 10.3934/dcdss.2020450

[20]

Guirong Jiang, Qishao Lu. The dynamics of a Prey-Predator model with impulsive state feedback control. Discrete & Continuous Dynamical Systems - B, 2006, 6 (6) : 1301-1320. doi: 10.3934/dcdsb.2006.6.1301

[Back to Top]