American Institute of Mathematical Sciences

Advanced
January  2020, 5: 4 doi: 10.1186/s41546-020-00046-x

Uncertainty and filtering of hidden Markov models in discrete time

Samuel N. Cohen

Mathematical Institute, University of Oxford, Woodstock Road, Oxford, UK

Received  June 13, 2018 Published  June 2020

We consider the problem of filtering an unseen Markov chain from noisy observations, in the presence of uncertainty regarding the parameters of the processes involved. Using the theory of nonlinear expectations, we describe the uncertainty in terms of a penalty function, which can be propagated forward in time in the place of the filter. We also investigate a simple control problem in this context.
Keywords: Filtering, Optimal control, Robustness, Nonlinear expectation.
Mathematics Subject Classification: 62M20;60G35;93E11.
Citation: Samuel N. Cohen. Uncertainty and filtering of hidden Markov models in discrete time. Probability, Uncertainty and Quantitative Risk, 2020, 5 (0) : 4-. doi: 10.1186/s41546-020-00046-x
References:
[1]

Allan, A.L. and S.N. Cohen. (2019a). Parameter uncertainty in the Kalman–Bucy filter, SIAM J. Control Optim. 57, no. 3, 1646–1671.,

[2]

Allan, A.L. and S.N. Cohen. (2020). Pathwise Stochastic Control with Applications to Robust Filtering, Ann. Appl. Prob. arXiv::1902.05434.,

[3]

Artzner, P., F. Delbaen, J.-M. Eber, and D. Heath. (1999). Coherent measures of risk, Math. Finan. 9, no. 3, 203–228.,

[4]

Başar, T. and P. Bernhard. (1991). H∞-Optimal Control and Related Minimax Design Problems, A Dynamic Game Approach, Birkhäuser, Basel.,

[5]

Bain, A. and D. Crisan. (2009). Fundamentals of Stochastic Filtering, Springer, Berlin–Heidelberg–New York.,

[6]

Bielecki, T.R., T. Chen, and I. Cialenco. (2017). Recursive construction of confidence regions, Electron. J. Stat. 11, no. 2, 4674–4700.,

[7]

Boel, R.K., M.R. James, and I.R. Petersen. (2002). Robustness and risk-sensitive filtering, IEEE Trans. Autom. Control 47, no. 3, 451–461.,

[8]

Cohen, S.N. and R.J. Elliott. (2010). A general theory of finite state backward stochastic difference equations, Stoch. Process. Appl. 120, no. 4, 442–466.,

[9]

Cohen, S.N. and R.J. Elliott. (2011). Backward stochastic difference equations and nearly-time-consistent nonlinear expectations, SIAM J. Control Optim. 49, no. 1, 125–139.,

[10]

Cohen, S.N. and R.J. Elliott. (2015). Stochastic Calculus and Applications, 2nd ed., Birkhäuser, New York.,

[11]

Cohen, S.N. (2017). Data-driven nonlinear expectations for statistical uncertainty in decisions, Electron. J. Stat. 11, no. 1, 1858–1889.,

[12]

Delbaen, F., S. Peng, and E. Rosazza Gianin. (2010). Representation of the penalty term of dynamic concave utilities, Finan. Stochast. 14, no. 3, 449–472.,

[13]

Dey, S. and J.B. Moore. (1995). Risk-sensitive filtering and smoothing for hidden Markov models, Syst. Control Lett. 25, 361–366.,

[14]

Douc, R., E. Moulines, J. Olsson, and R. van Handel. (2011). Consistency of the maximum likelihood estimator for general hidden Markov models, Ann. Stat. 39, no. 1, 474–513.,

[15]

Duffie, D. and L.G. Epstein. (1992). Asset pricing with stochastic differential utility, Rev. Finan. Stud. 5, no. 3, 411–436.,

[16]

El Karoui, N., S. Peng, and M.C. Quenez. (1997). Backward stochastic differential equations in finance, Math. Finan. 7, no. 1, 1–71.,

[17]

Epstein, L.G. and M. Schneider. (2003). Recursive multiple-priors, J. Econ. Theory 113, 1–31.,

[18]

Fagin, R. and J. Halpern. (1990). A new approach to updating beliefs, AUAI Press, Corvallis.,

[19]

Föllmer, H. and A. Schied. (2002a). Convex measures of risk and trading constraints, Finan. Stochast. 6, 429–447.,

[20]

Föllmer, H. and A. Schied. (2002b). Stochastic Finance: An Introduction in Discrete Time. Studies in Mathematics 27, de Gruyter, Berlin-New York.,

[21]

Frittelli, M. and E. Rosazza Gianin. (2002). Putting order in risk measures, J. Bank. Financ. 26, no. 7, 1473–1486.,

[22]

Graf, S. (1980). A Radon–Nikodym theorem for capacities, J. für die reine und angewandte Mathematik 320, 192–214.,

[23]

Grimble, M.J. and A. El Sayed. (1990). Solution of the H∞ optimal linear filtering problem for discretetime systems, Trans. Acoust. Speech Sig. Process. IEEE 38, no. 7.,

[24]

Hansen, L.P. and T.J. Sargent. (2005). Robust estimation and control under commitment, J. Econ. Theory 124, 258–301.,

[25]

Hansen, L.P. and T.J. Sargent. (2007). Recursive robust estimation and control without commitment, J. Econ. Theory 136, no. 1, 1–27.,

[26]

Hansen, L.P. and T.J. Sargent. (2008). Robustness, Princeton University Press, Princeton.,

[27]

Huber, P.J. and E.M. Roncetti. (2009). Robust Statistics, 2nd edn., Wiley, Hoboken.,

[28]

James, M.R., J.S. Baras, and R.J. Elliott. (1994). Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems, Trans. Autom. Control IEEE 39, no. 4, 780–792. https://doi.org/10.1109/9.286253.,

[29]

Kalman, R.E. (1960). A new approach to linear filtering and prediction problems, J. Basic Eng. ASME 82, 33–45.,

[30]

Kalman, R.E. and R.S. Bucy. (1961). New results in linear filtering and prediction theory, J. Basic Eng. ASME 83, 95–108.,

[31]

Keynes, J.M. (1921). A Treatise on Probability, Macmillan and Co., New York. Reprint BN Publishing, 2008.,

[32]

Knight, F.H. (1921). Risk, Uncertainty and Profit, Houghton Mifflin, Boston. reprint Dover 2006.,

[33]

Kupper, M. and W. Schachermayer. (2009). Representation results for law invariant time consistent functions, Math. Financ. Econ. 2, no. 3, 189–210.,

[34]

Leroux, B.G. (1992). Maximum-likelihood estimation for hidden Markov models, Stoch. Process. Appl. 40, 127–143.,

[35]

Peng, S. (2010). Nonlinear Expectations and Stochastic Calculus under Uncertainty, arxiv::1002.4546v1.,

[36]

Riedel, F. (2004). Dynamic coherent risk measures, Stochast. Process. Appl. 112, no. 2, 185–200.,

[37]

Rockafellar, R.T., S. Uryasev, and M. Zabarankin. (2006). Generalized deviations in risk analysis, Finan. Stochast. 10, 51–74.,

[38]

Wald, A. (1945). Statistical decision functions which minimize the maximum risk, Ann. Math. 46, no. 2, 265–280.,

[39]

Walley, P. (1991). Statistical Reasoning with Imprecise Probabilities, Chapman and Hall, London.,

[40]

Wonham, W.N. (1965). Some applications of stochastic differential equations to optimal nonlinear filtering, SIAM J. Control 2, 347–369.,

[41]

Zhang, J., Y. Xia, and P. Shi. (2009). Parameter-dependent robust H∞ filtering for uncertain discrete-time systems, Automatica 45, 560–565.,

show all references

References:
[1]

Allan, A.L. and S.N. Cohen. (2019a). Parameter uncertainty in the Kalman–Bucy filter, SIAM J. Control Optim. 57, no. 3, 1646–1671.,

[2]

Allan, A.L. and S.N. Cohen. (2020). Pathwise Stochastic Control with Applications to Robust Filtering, Ann. Appl. Prob. arXiv::1902.05434.,

[3]

Artzner, P., F. Delbaen, J.-M. Eber, and D. Heath. (1999). Coherent measures of risk, Math. Finan. 9, no. 3, 203–228.,

[4]

Başar, T. and P. Bernhard. (1991). H∞-Optimal Control and Related Minimax Design Problems, A Dynamic Game Approach, Birkhäuser, Basel.,

[5]

Bain, A. and D. Crisan. (2009). Fundamentals of Stochastic Filtering, Springer, Berlin–Heidelberg–New York.,

[6]

Bielecki, T.R., T. Chen, and I. Cialenco. (2017). Recursive construction of confidence regions, Electron. J. Stat. 11, no. 2, 4674–4700.,

[7]

Boel, R.K., M.R. James, and I.R. Petersen. (2002). Robustness and risk-sensitive filtering, IEEE Trans. Autom. Control 47, no. 3, 451–461.,

[8]

Cohen, S.N. and R.J. Elliott. (2010). A general theory of finite state backward stochastic difference equations, Stoch. Process. Appl. 120, no. 4, 442–466.,

[9]

Cohen, S.N. and R.J. Elliott. (2011). Backward stochastic difference equations and nearly-time-consistent nonlinear expectations, SIAM J. Control Optim. 49, no. 1, 125–139.,

[10]

Cohen, S.N. and R.J. Elliott. (2015). Stochastic Calculus and Applications, 2nd ed., Birkhäuser, New York.,

[11]

Cohen, S.N. (2017). Data-driven nonlinear expectations for statistical uncertainty in decisions, Electron. J. Stat. 11, no. 1, 1858–1889.,

[12]

Delbaen, F., S. Peng, and E. Rosazza Gianin. (2010). Representation of the penalty term of dynamic concave utilities, Finan. Stochast. 14, no. 3, 449–472.,

[13]

Dey, S. and J.B. Moore. (1995). Risk-sensitive filtering and smoothing for hidden Markov models, Syst. Control Lett. 25, 361–366.,

[14]

Douc, R., E. Moulines, J. Olsson, and R. van Handel. (2011). Consistency of the maximum likelihood estimator for general hidden Markov models, Ann. Stat. 39, no. 1, 474–513.,

[15]

Duffie, D. and L.G. Epstein. (1992). Asset pricing with stochastic differential utility, Rev. Finan. Stud. 5, no. 3, 411–436.,

[16]

El Karoui, N., S. Peng, and M.C. Quenez. (1997). Backward stochastic differential equations in finance, Math. Finan. 7, no. 1, 1–71.,

[17]

Epstein, L.G. and M. Schneider. (2003). Recursive multiple-priors, J. Econ. Theory 113, 1–31.,

[18]

Fagin, R. and J. Halpern. (1990). A new approach to updating beliefs, AUAI Press, Corvallis.,

[19]

Föllmer, H. and A. Schied. (2002a). Convex measures of risk and trading constraints, Finan. Stochast. 6, 429–447.,

[20]

Föllmer, H. and A. Schied. (2002b). Stochastic Finance: An Introduction in Discrete Time. Studies in Mathematics 27, de Gruyter, Berlin-New York.,

[21]

Frittelli, M. and E. Rosazza Gianin. (2002). Putting order in risk measures, J. Bank. Financ. 26, no. 7, 1473–1486.,

[22]

Graf, S. (1980). A Radon–Nikodym theorem for capacities, J. für die reine und angewandte Mathematik 320, 192–214.,

[23]

Grimble, M.J. and A. El Sayed. (1990). Solution of the H∞ optimal linear filtering problem for discretetime systems, Trans. Acoust. Speech Sig. Process. IEEE 38, no. 7.,

[24]

Hansen, L.P. and T.J. Sargent. (2005). Robust estimation and control under commitment, J. Econ. Theory 124, 258–301.,

[25]

Hansen, L.P. and T.J. Sargent. (2007). Recursive robust estimation and control without commitment, J. Econ. Theory 136, no. 1, 1–27.,

[26]

Hansen, L.P. and T.J. Sargent. (2008). Robustness, Princeton University Press, Princeton.,

[27]

Huber, P.J. and E.M. Roncetti. (2009). Robust Statistics, 2nd edn., Wiley, Hoboken.,

[28]

James, M.R., J.S. Baras, and R.J. Elliott. (1994). Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems, Trans. Autom. Control IEEE 39, no. 4, 780–792. https://doi.org/10.1109/9.286253.,

[29]

Kalman, R.E. (1960). A new approach to linear filtering and prediction problems, J. Basic Eng. ASME 82, 33–45.,

[30]

Kalman, R.E. and R.S. Bucy. (1961). New results in linear filtering and prediction theory, J. Basic Eng. ASME 83, 95–108.,

[31]

Keynes, J.M. (1921). A Treatise on Probability, Macmillan and Co., New York. Reprint BN Publishing, 2008.,

[32]

Knight, F.H. (1921). Risk, Uncertainty and Profit, Houghton Mifflin, Boston. reprint Dover 2006.,

[33]

Kupper, M. and W. Schachermayer. (2009). Representation results for law invariant time consistent functions, Math. Financ. Econ. 2, no. 3, 189–210.,

[34]

Leroux, B.G. (1992). Maximum-likelihood estimation for hidden Markov models, Stoch. Process. Appl. 40, 127–143.,

[35]

Peng, S. (2010). Nonlinear Expectations and Stochastic Calculus under Uncertainty, arxiv::1002.4546v1.,

[36]

Riedel, F. (2004). Dynamic coherent risk measures, Stochast. Process. Appl. 112, no. 2, 185–200.,

[37]

Rockafellar, R.T., S. Uryasev, and M. Zabarankin. (2006). Generalized deviations in risk analysis, Finan. Stochast. 10, 51–74.,

[38]

Wald, A. (1945). Statistical decision functions which minimize the maximum risk, Ann. Math. 46, no. 2, 265–280.,

[39]

Walley, P. (1991). Statistical Reasoning with Imprecise Probabilities, Chapman and Hall, London.,

[40]

Wonham, W.N. (1965). Some applications of stochastic differential equations to optimal nonlinear filtering, SIAM J. Control 2, 347–369.,

[41]

Zhang, J., Y. Xia, and P. Shi. (2009). Parameter-dependent robust H∞ filtering for uncertain discrete-time systems, Automatica 45, 560–565.,

[1]

Hong Niu, Zhijiang Feng, Qijin Xiao, Yajun Zhang. A PID control method based on optimal control strategy. Numerical Algebra, Control & Optimization, 2021, 11 (1) : 117-126. doi: 10.3934/naco.2020019
[2]

Zuliang Lu, Fei Huang, Xiankui Wu, Lin Li, Shang Liu. Convergence and quasi-optimality of $ L^2- $norms based an adaptive finite element method for nonlinear optimal control problems. Electronic Research Archive, 2020, 28 (4) : 1459-1486. doi: 10.3934/era.2020077
[3]

Lars Grüne, Matthias A. Müller, Christopher M. Kellett, Steven R. Weller. Strict dissipativity for discrete time discounted optimal control problems. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020046
[4]

Hai Huang, Xianlong Fu. Optimal control problems for a neutral integro-differential system with infinite delay. Evolution Equations & Control Theory, 2020  doi: 10.3934/eect.2020107
[5]

Vaibhav Mehandiratta, Mani Mehra, Günter Leugering. Fractional optimal control problems on a star graph: Optimality system and numerical solution. Mathematical Control & Related Fields, 2021, 11 (1) : 189-209. doi: 10.3934/mcrf.2020033
[6]

Christian Clason, Vu Huu Nhu, Arnd Rösch. Optimal control of a non-smooth quasilinear elliptic equation. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020052
[7]

Hongbo Guan, Yong Yang, Huiqing Zhu. A nonuniform anisotropic FEM for elliptic boundary layer optimal control problems. Discrete & Continuous Dynamical Systems - B, 2021, 26 (3) : 1711-1722. doi: 10.3934/dcdsb.2020179
[8]

A. Alessandri, F. Bedouhene, D. Bouhadjra, A. Zemouche, P. Bagnerini. Observer-based control for a class of hybrid linear and nonlinear systems. Discrete & Continuous Dynamical Systems - S, 2021, 14 (4) : 1213-1231. doi: 10.3934/dcdss.2020376
[9]

Youming Guo, Tingting Li. Optimal control strategies for an online game addiction model with low and high risk exposure. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020347
[10]

Pierluigi Colli, Gianni Gilardi, Jürgen Sprekels. Deep quench approximation and optimal control of general Cahn–Hilliard systems with fractional operators and double obstacle potentials. Discrete & Continuous Dynamical Systems - S, 2021, 14 (1) : 243-271. doi: 10.3934/dcdss.2020213
[11]

Stefan Doboszczak, Manil T. Mohan, Sivaguru S. Sritharan. Pontryagin maximum principle for the optimal control of linearized compressible navier-stokes equations with state constraints. Evolution Equations & Control Theory, 2020  doi: 10.3934/eect.2020110
[12]

Elimhan N. Mahmudov. Infimal convolution and duality in convex optimal control problems with second order evolution differential inclusions. Evolution Equations & Control Theory, 2021, 10 (1) : 37-59. doi: 10.3934/eect.2020051
[13]

Lars Grüne, Roberto Guglielmi. On the relation between turnpike properties and dissipativity for continuous time linear quadratic optimal control problems. Mathematical Control & Related Fields, 2021, 11 (1) : 169-188. doi: 10.3934/mcrf.2020032
[14]

Jingrui Sun, Hanxiao Wang. Mean-field stochastic linear-quadratic optimal control problems: Weak closed-loop solvability. Mathematical Control & Related Fields, 2021, 11 (1) : 47-71. doi: 10.3934/mcrf.2020026
[15]

Arthur Fleig, Lars Grüne. Strict dissipativity analysis for classes of optimal control problems involving probability density functions. Mathematical Control & Related Fields, 2020  doi: 10.3934/mcrf.2020053
[16]

Yuan Xu, Xin Jin, Saiwei Wang, Yang Tang. Optimal synchronization control of multiple euler-lagrange systems via event-triggered reinforcement learning. Discrete & Continuous Dynamical Systems - S, 2021, 14 (4) : 1495-1518. doi: 10.3934/dcdss.2020377
[17]

Héctor Barge. Čech cohomology, homoclinic trajectories and robustness of non-saddle sets. Discrete & Continuous Dynamical Systems - A, 2020  doi: 10.3934/dcds.2020381
[18]

Bernard Bonnard, Jérémy Rouot. Geometric optimal techniques to control the muscular force response to functional electrical stimulation using a non-isometric force-fatigue model. Journal of Geometric Mechanics, 2020  doi: 10.3934/jgm.2020032
[19]

Dominique Chapelle, Philippe Moireau, Patrick Le Tallec. Robust filtering for joint state-parameter estimation in distributed mechanical systems. Discrete & Continuous Dynamical Systems - A, 2009, 23 (1&2) : 65-84. doi: 10.3934/dcds.2009.23.65
[20]

Kalikinkar Mandal, Guang Gong. On ideal $ t $-tuple distribution of orthogonal functions in filtering de bruijn generators. Advances in Mathematics of Communications, 2020  doi: 10.3934/amc.2020125

 Impact Factor: 

Tools

Metrics

  • PDF downloads (19)
  • HTML views (97)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences