Decision Theory and large deviations for dynamical hypotheses tests:The Neyman-Pearson Lemma, Min-Max and Bayesian tests

Hermes H. Ferreira; Artur O. Lopes; Silvia R. C. Lopes

doi:10.3934/jdg.2021031

Article Contents

2022, Volume 9, Issue 2: 123-150. Doi: 10.3934/jdg.2021031

This issue Previous Article Next Article Properties of contests: Constructing contest success functions from best-responses

Decision Theory and large deviations for dynamical hypotheses tests: The Neyman-Pearson Lemma, Min-Max and Bayesian tests

IME - UFRGS, Av. Bento Goncalves 9500 - 91509-900 - Porto Alegre, Brazil

^*Corresponding author: Artur O. Lopes
^*Corresponding author: Artur O. Lopes

Received: December 31, 2020

Revised: July 31, 2021

Early access: January 2022

Published: April 2022

H.H. Ferreira was supported by CAPES-Brazil scholarship. A.O. Lopes and S.R.C. Lopes were partially supported by grant of CNPq-Brazil

Abstract Full Text(HTML) Figure(4) Related Papers Cited by

Abstract

We analyze hypotheses tests using classical results on large deviations to compare two models, each one described by a different Hölder Gibbs probability measure. One main difference to the classical hypothesis tests in Decision Theory is that here the two measures are singular with respect to each other. Among other objectives, we are interested in the decay rate of the wrong decisions probability, when the sample size $ n $ goes to infinity. We show a dynamical version of the Neyman-Pearson Lemma displaying the ideal test within a certain class of similar tests. This test becomes exponentially better, compared to other alternative tests, when the sample size goes to infinity. We are able to present the explicit exponential decay rate. We also consider both, the Min-Max and a certain type of Bayesian hypotheses tests. We shall consider these tests in the log likelihood framework by using several tools of Thermodynamic Formalism. Versions of the Stein's Lemma and Chernoff's information are also presented.

Keywords:

Mathematics Subject Classification: 62C20, 62C10, 37D35.

Citation:

FullText(HTML)

Figure 1. Graphs of $ P_0 $ (in solid line) and $ P_1 $ (in dashed line) for the functions defined in (3.27). For these plots we consider the data from the example in Section 7

Download: Full-size image PowerPoint slide

Figure 2. The large deviation rate function $ I_1(\cdot) $ at points $ v_1 $, $ G_1 $ and zero, where $ G_1 = E - \left(\int \log J_0 \, d \mu_{0} \, - \int \log \, J_1\, \, d\mu_0\, \right) $

Download: Full-size image PowerPoint slide

Figure 3. Graph of the function $ R(\lambda) = I_0^{E_{\lambda, \lambda}}(0) $, when $ 0\leq \lambda\leq \lambda_s $, using the stochastic matrix $ \mathcal{P}_{j} $, for $ j = 0, 1 $, from the example in Section 7

Download: Full-size image PowerPoint slide

Figure 4. Graphs of the functions $ \lambda \to \int \log J_\lambda d \mu_0 $ (in dotted line) and $ \lambda \to \int \log J_\lambda d \mu_1 $ (in dashed and dotted line) together with the graph of the values $ E_\lambda $ (in solid line), as a function of $ \lambda $, when $ 0\leq \lambda\leq \lambda_s $. The stochastic matrix $ \mathcal{P}_{j} $, for $ j = 0, 1 $, is from the example in Section 7

Download: Full-size image PowerPoint slide

Related Papers

Cited by

References

[1]	F. Abramovich and Y. Ritov, Statistical Theory: A Concise Introduction, Boca Raton, CRC Press, 2013. doi: 10.1201/b14755.
[2]	R. R. Bahadur, Large deviations of the maximum likelihood estimate in the Markov chain case, In J. S. Rostag, M. H. Rizvi and D. Siegmund, editors, Recent Advances in Statistics, 273-283. Boston, Academic Press, 1983. doi: 10.1016/B978-0-12-589320-6.50017-4.
[3]	A. Baraviera, R. Leplaideur and A. O. Lopes, Ergodic Optimization, Zero Temperature and the Max-Plus Algebra, $23^{\text{o}}$ Colóquio Brasileiro de Matemática, IMPA, Rio de Janeiro, 2013.
[4]	M. Barni and B. Tondi, Binary hypothesis testing game with training data, IEEE Transactions on Information Theory, 60 (2014), 4848-4866. doi: 10.1109/TIT.2014.2325571.
[5]	T. Benoist, V. Jakšić, Y. Pautrat and C.-A. Pillet, On entropy production of repeated quantum measurements I. General theory, Commun. Math. Phys., 357 (2018), 77-123. doi: 10.1007/s00220-017-2947-1.
[6]	D. Blackwell and M. A. Girshick, Theory of Games and Statistical Decisions, Dover publications, 1979.
[7]	D. Bohle, A. Marynych and M. Meiners, A fundamental problem of hypotesis testing with finite e-commerce, Appl. Stoch. Models Bus. Ind., 37 (2021), 454-474. preprint, arXiv: 2006.05786. doi: 10.1002/asmb.2574.
[8]	L. D. Broemeling, Bayesian Inference for Stochastic Processes, Boca Raton, CRC Press, 2018.
[9]	J. A. Bucklew, Large Deviation Techniques in Decision, Simulation and Estimation, New York, Wiley, 1990.
[10]	A. Caticha, Lectures on probability, entropy and statistical physics, Entropic Physics, preprint, arXiv: 0808.0012.
[11]	J.-R. Chazottes and D. Gabrielli, Large deviations for empirical entropies of g-measures, Nonlinearity, 18 (2005), 2545-2563. doi: 10.1088/0951-7715/18/6/007.
[12]	T. M. Cover and J. A. Thomas, Elements of Information Theory, second edition, New York, Wiley Press, 2006.
[13]	G. B. Cybis, S. R. C. Lopes and H. P. Pinheiro, Power of the likelihood ratio test for models of DNA base substitution, Journal of Applied Statistics, 38 (2011), 2723-2737. doi: 10.1080/02664763.2011.567253.
[14]	R. Dakovic, M. Denker and M. Gordin, Circular unitary ensembles: Parametric models and their asymptotic maximum likelihood estimates, Journal of Mathematical Sciences, 219 (2016), 714-730. doi: 10.1007/s10958-016-3141-2.
[15]	A. Dembo and O. Zeitouni, Large Deviation Techniques and Applications, New York, Springer Verlag, 2010. doi: 10.1007/978-3-642-03311-7.
[16]	M. Denker, Basics of Thermodynamics, Lecture Notes - Penn State Univ., 2011.
[17]	M. Denker and W. Woyczynski, Introductory Statistics and Random Phenomena: Uncertainty, Complexity and Chaotic Behavior in Engineering and Science, New York, Birkhäuser, 2012. doi: 10.1007/978-3-319-66152-0.
[18]	R. S. Ellis, Entropy, Large Deviations, and Statistical Mechanics, New York, Springer Verlag, 2006. doi: 10.1007/3-540-29060-5.
[19]	H. H. Ferreira, A. O. Lopes and E. R. Oliveira, An iteration process for approximating subactions, Modeling, Dynamics, Optimization and Bioeconomics IV, Editors: Alberto Pinto and David Zilberman, Springer Proceedings in Mathematics and Statistics, New York, Springer Verlag (2021), 187-212.
[20]	V. Girardin, L. Lhote and P. Regnault, Different closed-form expressions for generalized entropy rates of Markov chains, Methodology and Computing in Applied Probability, 21 (2019), 1431-1452. doi: 10.1007/s11009-018-9679-3.
[21]	V. Girardin and P. Regnault, Escort distributions minimizing the Kullback-Leibler divergence for a large deviations principle and tests of entropy level, Ann Inst Stat Math., 68 (2016), 439-468. doi: 10.1007/s10463-014-0501-x.
[22]	M. J. Karling, S. R. C. Lopes and R. M. de Souza, A Bayesian approach for estimating the parameters of an $\alpha$-stable distribution, Journal of Statistical Computation and Simulation, 91 (2021), 1713-1748. doi: 10.1080/00949655.2020.1865958.
[23]	Y. Kifer, Large deviations in dynamical systems and stochastic processes, Trans. Amer. Math. Soc., 321 (1990), 505-524. doi: 10.1090/S0002-9947-1990-1025756-7.
[24]	A. Lopes, Entropy, pressure and large deviation, Cellular Automata, Dynamical Systems and Neural Networks, E. Goles e S. Martinez (eds.), Kluwer, Massachusets, (1994), 79-146.
[25]	A. O. Lopes, Thermodynamic formalism, maximizing probabilities and large deviations, Preprint - UFRGS.
[26]	A. O. Lopes, Entropy and large deviation, NonLinearity, 3 (1990), 527-546. doi: 10.1088/0951-7715/3/2/013.
[27]	A. O. Lopes, S. R. C. Lopes and P. Varandas, Bayes posterior convergence for loss functions via almost additive thermodynamic formalism, to appear in Journ. of Statis. Physics.
[28]	A. O. Lopes and J. K. Mengue, On information gain, Kullback-Leibler divergence, entropy production and the involution kernel, to appear in Disc. and Cont. Dyn. Syst. Series A.
[29]	A. O. Lopes and R. Ruggiero, Nonequilibrium in thermodynamic formalism: The second law, gases and information geometry, Qualitative Theory of Dynamical Systems, 21 (2022). doi: 10.1007/s12346-021-00551-0.
[30]	K. McGoff, S. Mukherjee and A. Nobel, Gibbs posterior convergence and thermodynamic formalism, to appear in Adv. in Appl. Prob.
[31]	W. Parry and M. Pollicott, Zeta functions and the periodic orbit structure of hyperbolic dynamics, Astérisque, (1990), 187-188.
[32]	V. K. Rohatgi, An Introduction to Probability Theory and Mathematical Statistics, New York: Wiley, 1976.
[33]	T. Sagawa, Entropy, divergence and majorization in classical and quantum theory, arXiv: 2007.09974.
[34]	Y. Suhov and M. Kelbert, Probability and Statistics by Example. I, Cambridge, Cambridge University Press, 2014. doi: 10.1017/CBO9781139087773.
[35]	D. A. van Dyk, The Role of statistics in the discovery of a higgs boson, Annual Review of Statistics and Its Application, 1 (2014), 41-59. doi: 10.1146/annurev-statistics-062713-085841.
[36]	A. C. D. van Enter, A. O. Lopes, S. R. C Lopes and J. K. Mengue, How to get the Bayesian a posteriori probability from an a priori probability via thermodynamic formalism for plans; the connection to disordered systems, work in progress.
[37]	W. von der Linden, V. Dose and U. von Toussaint, Bayesian Probability Theory Applications in the Physical Sciences, Cambridge, Cambridge University Press, 2014. doi: 10.1017/CBO9781139565608.