American Institute of Mathematical Sciences

Advanced
January  2020, 5: 4 doi: 10.1186/s41546-020-00046-x

Uncertainty and filtering of hidden Markov models in discrete time

Samuel N. Cohen

Mathematical Institute, University of Oxford, Woodstock Road, Oxford, UK

Received  June 13, 2018 Published  June 2020

We consider the problem of filtering an unseen Markov chain from noisy observations, in the presence of uncertainty regarding the parameters of the processes involved. Using the theory of nonlinear expectations, we describe the uncertainty in terms of a penalty function, which can be propagated forward in time in the place of the filter. We also investigate a simple control problem in this context.
Keywords: Filtering, Optimal control, Robustness, Nonlinear expectation.
Mathematics Subject Classification: 62M20;60G35;93E11.
Citation: Samuel N. Cohen. Uncertainty and filtering of hidden Markov models in discrete time. Probability, Uncertainty and Quantitative Risk, 2020, 5 (0) : 4-. doi: 10.1186/s41546-020-00046-x
References:
[1]

Allan, A.L. and S.N. Cohen. (2019a). Parameter uncertainty in the Kalman–Bucy filter, SIAM J. Control Optim. 57, no. 3, 1646–1671. Google Scholar

[2]

Allan, A.L. and S.N. Cohen. (2020). Pathwise Stochastic Control with Applications to Robust Filtering, Ann. Appl. Prob. arXiv::1902.05434. Google Scholar

[3]

Artzner, P., F. Delbaen, J.-M. Eber, and D. Heath. (1999). Coherent measures of risk, Math. Finan. 9, no. 3, 203–228. Google Scholar

[4]

Başar, T. and P. Bernhard. (1991). H∞-Optimal Control and Related Minimax Design Problems, A Dynamic Game Approach, Birkhäuser, Basel. Google Scholar

[5]

Bain, A. and D. Crisan. (2009). Fundamentals of Stochastic Filtering, Springer, Berlin–Heidelberg–New York. Google Scholar

[6]

Bielecki, T.R., T. Chen, and I. Cialenco. (2017). Recursive construction of confidence regions, Electron. J. Stat. 11, no. 2, 4674–4700. Google Scholar

[7]

Boel, R.K., M.R. James, and I.R. Petersen. (2002). Robustness and risk-sensitive filtering, IEEE Trans. Autom. Control 47, no. 3, 451–461. Google Scholar

[8]

Cohen, S.N. and R.J. Elliott. (2010). A general theory of finite state backward stochastic difference equations, Stoch. Process. Appl. 120, no. 4, 442–466. Google Scholar

[9]

Cohen, S.N. and R.J. Elliott. (2011). Backward stochastic difference equations and nearly-time-consistent nonlinear expectations, SIAM J. Control Optim. 49, no. 1, 125–139. Google Scholar

[10]

Cohen, S.N. and R.J. Elliott. (2015). Stochastic Calculus and Applications, 2nd ed., Birkhäuser, New York. Google Scholar

[11]

Cohen, S.N. (2017). Data-driven nonlinear expectations for statistical uncertainty in decisions, Electron. J. Stat. 11, no. 1, 1858–1889. Google Scholar

[12]

Delbaen, F., S. Peng, and E. Rosazza Gianin. (2010). Representation of the penalty term of dynamic concave utilities, Finan. Stochast. 14, no. 3, 449–472. Google Scholar

[13]

Dey, S. and J.B. Moore. (1995). Risk-sensitive filtering and smoothing for hidden Markov models, Syst. Control Lett. 25, 361–366. Google Scholar

[14]

Douc, R., E. Moulines, J. Olsson, and R. van Handel. (2011). Consistency of the maximum likelihood estimator for general hidden Markov models, Ann. Stat. 39, no. 1, 474–513. Google Scholar

[15]

Duffie, D. and L.G. Epstein. (1992). Asset pricing with stochastic differential utility, Rev. Finan. Stud. 5, no. 3, 411–436. Google Scholar

[16]

El Karoui, N., S. Peng, and M.C. Quenez. (1997). Backward stochastic differential equations in finance, Math. Finan. 7, no. 1, 1–71. Google Scholar

[17]

Epstein, L.G. and M. Schneider. (2003). Recursive multiple-priors, J. Econ. Theory 113, 1–31. Google Scholar

[18]

Fagin, R. and J. Halpern. (1990). A new approach to updating beliefs, AUAI Press, Corvallis. Google Scholar

[19]

Föllmer, H. and A. Schied. (2002a). Convex measures of risk and trading constraints, Finan. Stochast. 6, 429–447. Google Scholar

[20]

Föllmer, H. and A. Schied. (2002b). Stochastic Finance: An Introduction in Discrete Time. Studies in Mathematics 27, de Gruyter, Berlin-New York. Google Scholar

[21]

Frittelli, M. and E. Rosazza Gianin. (2002). Putting order in risk measures, J. Bank. Financ. 26, no. 7, 1473–1486. Google Scholar

[22]

Graf, S. (1980). A Radon–Nikodym theorem for capacities, J. für die reine und angewandte Mathematik 320, 192–214. Google Scholar

[23]

Grimble, M.J. and A. El Sayed. (1990). Solution of the H∞ optimal linear filtering problem for discretetime systems, Trans. Acoust. Speech Sig. Process. IEEE 38, no. 7. Google Scholar

[24]

Hansen, L.P. and T.J. Sargent. (2005). Robust estimation and control under commitment, J. Econ. Theory 124, 258–301. Google Scholar

[25]

Hansen, L.P. and T.J. Sargent. (2007). Recursive robust estimation and control without commitment, J. Econ. Theory 136, no. 1, 1–27. Google Scholar

[26]

Hansen, L.P. and T.J. Sargent. (2008). Robustness, Princeton University Press, Princeton. Google Scholar

[27]

Huber, P.J. and E.M. Roncetti. (2009). Robust Statistics, 2nd edn., Wiley, Hoboken. Google Scholar

[28]

James, M.R., J.S. Baras, and R.J. Elliott. (1994). Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems, Trans. Autom. Control IEEE 39, no. 4, 780–792. https://doi.org/10.1109/9.286253. Google Scholar

[29]

Kalman, R.E. (1960). A new approach to linear filtering and prediction problems, J. Basic Eng. ASME 82, 33–45. Google Scholar

[30]

Kalman, R.E. and R.S. Bucy. (1961). New results in linear filtering and prediction theory, J. Basic Eng. ASME 83, 95–108. Google Scholar

[31]

Keynes, J.M. (1921). A Treatise on Probability, Macmillan and Co., New York. Reprint BN Publishing, 2008. Google Scholar

[32]

Knight, F.H. (1921). Risk, Uncertainty and Profit, Houghton Mifflin, Boston. reprint Dover 2006. Google Scholar

[33]

Kupper, M. and W. Schachermayer. (2009). Representation results for law invariant time consistent functions, Math. Financ. Econ. 2, no. 3, 189–210. Google Scholar

[34]

Leroux, B.G. (1992). Maximum-likelihood estimation for hidden Markov models, Stoch. Process. Appl. 40, 127–143. Google Scholar

[35]

Peng, S. (2010). Nonlinear Expectations and Stochastic Calculus under Uncertainty, arxiv::1002.4546v1. Google Scholar

[36]

Riedel, F. (2004). Dynamic coherent risk measures, Stochast. Process. Appl. 112, no. 2, 185–200. Google Scholar

[37]

Rockafellar, R.T., S. Uryasev, and M. Zabarankin. (2006). Generalized deviations in risk analysis, Finan. Stochast. 10, 51–74. Google Scholar

[38]

Wald, A. (1945). Statistical decision functions which minimize the maximum risk, Ann. Math. 46, no. 2, 265–280. Google Scholar

[39]

Walley, P. (1991). Statistical Reasoning with Imprecise Probabilities, Chapman and Hall, London. Google Scholar

[40]

Wonham, W.N. (1965). Some applications of stochastic differential equations to optimal nonlinear filtering, SIAM J. Control 2, 347–369. Google Scholar

[41]

Zhang, J., Y. Xia, and P. Shi. (2009). Parameter-dependent robust H∞ filtering for uncertain discrete-time systems, Automatica 45, 560–565. Google Scholar

show all references

References:
[1]

Allan, A.L. and S.N. Cohen. (2019a). Parameter uncertainty in the Kalman–Bucy filter, SIAM J. Control Optim. 57, no. 3, 1646–1671. Google Scholar

[2]

Allan, A.L. and S.N. Cohen. (2020). Pathwise Stochastic Control with Applications to Robust Filtering, Ann. Appl. Prob. arXiv::1902.05434. Google Scholar

[3]

Artzner, P., F. Delbaen, J.-M. Eber, and D. Heath. (1999). Coherent measures of risk, Math. Finan. 9, no. 3, 203–228. Google Scholar

[4]

Başar, T. and P. Bernhard. (1991). H∞-Optimal Control and Related Minimax Design Problems, A Dynamic Game Approach, Birkhäuser, Basel. Google Scholar

[5]

Bain, A. and D. Crisan. (2009). Fundamentals of Stochastic Filtering, Springer, Berlin–Heidelberg–New York. Google Scholar

[6]

Bielecki, T.R., T. Chen, and I. Cialenco. (2017). Recursive construction of confidence regions, Electron. J. Stat. 11, no. 2, 4674–4700. Google Scholar

[7]

Boel, R.K., M.R. James, and I.R. Petersen. (2002). Robustness and risk-sensitive filtering, IEEE Trans. Autom. Control 47, no. 3, 451–461. Google Scholar

[8]

Cohen, S.N. and R.J. Elliott. (2010). A general theory of finite state backward stochastic difference equations, Stoch. Process. Appl. 120, no. 4, 442–466. Google Scholar

[9]

Cohen, S.N. and R.J. Elliott. (2011). Backward stochastic difference equations and nearly-time-consistent nonlinear expectations, SIAM J. Control Optim. 49, no. 1, 125–139. Google Scholar

[10]

Cohen, S.N. and R.J. Elliott. (2015). Stochastic Calculus and Applications, 2nd ed., Birkhäuser, New York. Google Scholar

[11]

Cohen, S.N. (2017). Data-driven nonlinear expectations for statistical uncertainty in decisions, Electron. J. Stat. 11, no. 1, 1858–1889. Google Scholar

[12]

Delbaen, F., S. Peng, and E. Rosazza Gianin. (2010). Representation of the penalty term of dynamic concave utilities, Finan. Stochast. 14, no. 3, 449–472. Google Scholar

[13]

Dey, S. and J.B. Moore. (1995). Risk-sensitive filtering and smoothing for hidden Markov models, Syst. Control Lett. 25, 361–366. Google Scholar

[14]

Douc, R., E. Moulines, J. Olsson, and R. van Handel. (2011). Consistency of the maximum likelihood estimator for general hidden Markov models, Ann. Stat. 39, no. 1, 474–513. Google Scholar

[15]

Duffie, D. and L.G. Epstein. (1992). Asset pricing with stochastic differential utility, Rev. Finan. Stud. 5, no. 3, 411–436. Google Scholar

[16]

El Karoui, N., S. Peng, and M.C. Quenez. (1997). Backward stochastic differential equations in finance, Math. Finan. 7, no. 1, 1–71. Google Scholar

[17]

Epstein, L.G. and M. Schneider. (2003). Recursive multiple-priors, J. Econ. Theory 113, 1–31. Google Scholar

[18]

Fagin, R. and J. Halpern. (1990). A new approach to updating beliefs, AUAI Press, Corvallis. Google Scholar

[19]

Föllmer, H. and A. Schied. (2002a). Convex measures of risk and trading constraints, Finan. Stochast. 6, 429–447. Google Scholar

[20]

Föllmer, H. and A. Schied. (2002b). Stochastic Finance: An Introduction in Discrete Time. Studies in Mathematics 27, de Gruyter, Berlin-New York. Google Scholar

[21]

Frittelli, M. and E. Rosazza Gianin. (2002). Putting order in risk measures, J. Bank. Financ. 26, no. 7, 1473–1486. Google Scholar

[22]

Graf, S. (1980). A Radon–Nikodym theorem for capacities, J. für die reine und angewandte Mathematik 320, 192–214. Google Scholar

[23]

Grimble, M.J. and A. El Sayed. (1990). Solution of the H∞ optimal linear filtering problem for discretetime systems, Trans. Acoust. Speech Sig. Process. IEEE 38, no. 7. Google Scholar

[24]

Hansen, L.P. and T.J. Sargent. (2005). Robust estimation and control under commitment, J. Econ. Theory 124, 258–301. Google Scholar

[25]

Hansen, L.P. and T.J. Sargent. (2007). Recursive robust estimation and control without commitment, J. Econ. Theory 136, no. 1, 1–27. Google Scholar

[26]

Hansen, L.P. and T.J. Sargent. (2008). Robustness, Princeton University Press, Princeton. Google Scholar

[27]

Huber, P.J. and E.M. Roncetti. (2009). Robust Statistics, 2nd edn., Wiley, Hoboken. Google Scholar

[28]

James, M.R., J.S. Baras, and R.J. Elliott. (1994). Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems, Trans. Autom. Control IEEE 39, no. 4, 780–792. https://doi.org/10.1109/9.286253. Google Scholar

[29]

Kalman, R.E. (1960). A new approach to linear filtering and prediction problems, J. Basic Eng. ASME 82, 33–45. Google Scholar

[30]

Kalman, R.E. and R.S. Bucy. (1961). New results in linear filtering and prediction theory, J. Basic Eng. ASME 83, 95–108. Google Scholar

[31]

Keynes, J.M. (1921). A Treatise on Probability, Macmillan and Co., New York. Reprint BN Publishing, 2008. Google Scholar

[32]

Knight, F.H. (1921). Risk, Uncertainty and Profit, Houghton Mifflin, Boston. reprint Dover 2006. Google Scholar

[33]

Kupper, M. and W. Schachermayer. (2009). Representation results for law invariant time consistent functions, Math. Financ. Econ. 2, no. 3, 189–210. Google Scholar

[34]

Leroux, B.G. (1992). Maximum-likelihood estimation for hidden Markov models, Stoch. Process. Appl. 40, 127–143. Google Scholar

[35]

Peng, S. (2010). Nonlinear Expectations and Stochastic Calculus under Uncertainty, arxiv::1002.4546v1. Google Scholar

[36]

Riedel, F. (2004). Dynamic coherent risk measures, Stochast. Process. Appl. 112, no. 2, 185–200. Google Scholar

[37]

Rockafellar, R.T., S. Uryasev, and M. Zabarankin. (2006). Generalized deviations in risk analysis, Finan. Stochast. 10, 51–74. Google Scholar

[38]

Wald, A. (1945). Statistical decision functions which minimize the maximum risk, Ann. Math. 46, no. 2, 265–280. Google Scholar

[39]

Walley, P. (1991). Statistical Reasoning with Imprecise Probabilities, Chapman and Hall, London. Google Scholar

[40]

Wonham, W.N. (1965). Some applications of stochastic differential equations to optimal nonlinear filtering, SIAM J. Control 2, 347–369. Google Scholar

[41]

Zhang, J., Y. Xia, and P. Shi. (2009). Parameter-dependent robust H∞ filtering for uncertain discrete-time systems, Automatica 45, 560–565. Google Scholar

[1]

Sie Long Kek, Kok Lay Teo, Mohd Ismail Abd Aziz. Filtering solution of nonlinear stochastic optimal control problem in discrete-time with model-reality differences. Numerical Algebra, Control & Optimization, 2012, 2 (1) : 207-222. doi: 10.3934/naco.2012.2.207
[2]

Qun Lin, Ryan Loxton, Kok Lay Teo. The control parameterization method for nonlinear optimal control: A survey. Journal of Industrial & Management Optimization, 2014, 10 (1) : 275-309. doi: 10.3934/jimo.2014.10.275
[3]

Z. G. Feng, Kok Lay Teo, N. U. Ahmed, Yulin Zhao, W. Y. Yan. Optimal fusion of sensor data for Kalman filtering. Discrete & Continuous Dynamical Systems, 2006, 14 (3) : 483-503. doi: 10.3934/dcds.2006.14.483
[4]

Ellina Grigorieva, Evgenii Khailov. Optimal control of a nonlinear model of economic growth. Conference Publications, 2007, 2007 (Special) : 456-466. doi: 10.3934/proc.2007.2007.456
[5]

Piermarco Cannarsa, Carlo Sinestrari. On a class of nonlinear time optimal control problems. Discrete & Continuous Dynamical Systems, 1995, 1 (2) : 285-300. doi: 10.3934/dcds.1995.1.285
[6]

H. Thomas Banks, Shuhua Hu, Zackary R. Kenz, Hien T. Tran. A comparison of nonlinear filtering approaches in the context of an HIV model. Mathematical Biosciences & Engineering, 2010, 7 (2) : 213-236. doi: 10.3934/mbe.2010.7.213
[7]

Ying Zhang, Changjun Yu, Yingtao Xu, Yanqin Bai. Minimizing almost smooth control variation in nonlinear optimal control problems. Journal of Industrial & Management Optimization, 2020, 16 (4) : 1663-1683. doi: 10.3934/jimo.2019023
[8]

Bin Li, Kok Lay Teo, Cheng-Chew Lim, Guang Ren Duan. An optimal PID controller design for nonlinear constrained optimal control problems. Discrete & Continuous Dynamical Systems - B, 2011, 16 (4) : 1101-1117. doi: 10.3934/dcdsb.2011.16.1101
[9]

Jingang Zhai, Guangmao Jiang, Jianxiong Ye. Optimal dilution strategy for a microbial continuous culture based on the biological robustness. Numerical Algebra, Control & Optimization, 2015, 5 (1) : 59-69. doi: 10.3934/naco.2015.5.59
[10]

Heinz Schättler, Urszula Ledzewicz, Benjamin Cardwell. Robustness of optimal controls for a class of mathematical models for tumor anti-angiogenesis. Mathematical Biosciences & Engineering, 2011, 8 (2) : 355-369. doi: 10.3934/mbe.2011.8.355
[11]

Rong Liu, Feng-Qin Zhang, Yuming Chen. Optimal contraception control for a nonlinear population model with size structure and a separable mortality. Discrete & Continuous Dynamical Systems - B, 2016, 21 (10) : 3603-3618. doi: 10.3934/dcdsb.2016112
[12]

Leszek Gasiński, Nikolaos S. Papageorgiou. Relaxation of optimal control problems driven by nonlinear evolution equations. Evolution Equations & Control Theory, 2020, 9 (4) : 1027-1040. doi: 10.3934/eect.2020050
[13]

Gabriella Zecca. An optimal control problem for some nonlinear elliptic equations with unbounded coefficients. Discrete & Continuous Dynamical Systems - B, 2019, 24 (3) : 1393-1409. doi: 10.3934/dcdsb.2019021
[14]

M. Alipour, M. A. Vali, A. H. Borzabadi. A hybrid parametrization approach for a class of nonlinear optimal control problems. Numerical Algebra, Control & Optimization, 2019, 9 (4) : 493-506. doi: 10.3934/naco.2019037
[15]

Shu Luan. On the existence of optimal control for semilinear elliptic equations with nonlinear neumann boundary conditions. Mathematical Control & Related Fields, 2017, 7 (3) : 493-506. doi: 10.3934/mcrf.2017018
[16]

Z.-R. He, M.-S. Wang, Z.-E. Ma. Optimal birth control problems for nonlinear age-structured population dynamics. Discrete & Continuous Dynamical Systems - B, 2004, 4 (3) : 589-594. doi: 10.3934/dcdsb.2004.4.589
[17]

Wensheng Yin, Jinde Cao, Guoqiang Zheng. Further results on stabilization of stochastic differential equations with delayed feedback control under $ G $-expectation framework. Discrete & Continuous Dynamical Systems - B, 2021  doi: 10.3934/dcdsb.2021072
[18]

Sie Long Kek, Mohd Ismail Abd Aziz, Kok Lay Teo, Rohanin Ahmad. An iterative algorithm based on model-reality differences for discrete-time nonlinear stochastic optimal control problems. Numerical Algebra, Control & Optimization, 2013, 3 (1) : 109-125. doi: 10.3934/naco.2013.3.109
[19]

Chongyang Liu, Zhaohua Gong, Enmin Feng, Hongchao Yin. Modelling and optimal control for nonlinear multistage dynamical system of microbial fed-batch culture. Journal of Industrial & Management Optimization, 2009, 5 (4) : 835-850. doi: 10.3934/jimo.2009.5.835
[20]

Hongyong Deng, Wei Wei. Existence and stability analysis for nonlinear optimal control problems with $1$-mean equicontinuous controls. Journal of Industrial & Management Optimization, 2015, 11 (4) : 1409-1422. doi: 10.3934/jimo.2015.11.1409

 Impact Factor: 

Tools

Metrics

  • PDF downloads (40)
  • HTML views (190)
  • Cited by (1)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences