doi: 10.3934/jdg.2021031
Online First

Online First articles are published articles within a journal that have not yet been assigned to a formal issue. This means they do not yet have a volume number, issue number, or page numbers assigned to them, however, they can still be found and cited using their DOI (Digital Object Identifier). Online First publication benefits the research community by making new scientific discoveries known as quickly as possible.

Readers can access Online First articles via the “Online First” tab for the selected journal.

Decision Theory and large deviations for dynamical hypotheses tests: The Neyman-Pearson Lemma, Min-Max and Bayesian tests

IME - UFRGS, Av. Bento Goncalves 9500 - 91509-900 - Porto Alegre, Brazil

*Corresponding author: Artur O. Lopes

Received  January 2021 Revised  August 2021 Early access January 2022

Fund Project: H.H. Ferreira was supported by CAPES-Brazil scholarship. A.O. Lopes and S.R.C. Lopes were partially supported by grant of CNPq-Brazil

We analyze hypotheses tests using classical results on large deviations to compare two models, each one described by a different Hölder Gibbs probability measure. One main difference to the classical hypothesis tests in Decision Theory is that here the two measures are singular with respect to each other. Among other objectives, we are interested in the decay rate of the wrong decisions probability, when the sample size $ n $ goes to infinity. We show a dynamical version of the Neyman-Pearson Lemma displaying the ideal test within a certain class of similar tests. This test becomes exponentially better, compared to other alternative tests, when the sample size goes to infinity. We are able to present the explicit exponential decay rate. We also consider both, the Min-Max and a certain type of Bayesian hypotheses tests. We shall consider these tests in the log likelihood framework by using several tools of Thermodynamic Formalism. Versions of the Stein's Lemma and Chernoff's information are also presented.

Citation: Hermes H. Ferreira, Artur O. Lopes, Silvia R. C. Lopes. Decision Theory and large deviations for dynamical hypotheses tests: The Neyman-Pearson Lemma, Min-Max and Bayesian tests. Journal of Dynamics & Games, doi: 10.3934/jdg.2021031
References:
[1] F. Abramovich and Y. Ritov, Statistical Theory: A Concise Introduction, Boca Raton, CRC Press, 2013.  doi: 10.1201/b14755.  Google Scholar
[2]

R. R. Bahadur, Large deviations of the maximum likelihood estimate in the Markov chain case, In J. S. Rostag, M. H. Rizvi and D. Siegmund, editors, Recent Advances in Statistics, 273-283. Boston, Academic Press, 1983. doi: 10.1016/B978-0-12-589320-6.50017-4.  Google Scholar

[3]

A. Baraviera, R. Leplaideur and A. O. Lopes, Ergodic Optimization, Zero Temperature and the Max-Plus Algebra, $23^{\text{o}}$ Colóquio Brasileiro de Matemática, IMPA, Rio de Janeiro, 2013.  Google Scholar

[4]

M. Barni and B. Tondi, Binary hypothesis testing game with training data, IEEE Transactions on Information Theory, 60 (2014), 4848-4866.  doi: 10.1109/TIT.2014.2325571.  Google Scholar

[5]

T. BenoistV. JakšićY. Pautrat and C.-A. Pillet, On entropy production of repeated quantum measurements I. General theory, Commun. Math. Phys., 357 (2018), 77-123.  doi: 10.1007/s00220-017-2947-1.  Google Scholar

[6]

D. Blackwell and M. A. Girshick, Theory of Games and Statistical Decisions, Dover publications, 1979.  Google Scholar

[7]

D. Bohle, A. Marynych and M. Meiners, A fundamental problem of hypotesis testing with finite e-commerce, Appl. Stoch. Models Bus. Ind., 37 (2021), 454-474. preprint, arXiv: 2006.05786. doi: 10.1002/asmb.2574.  Google Scholar

[8] L. D. Broemeling, Bayesian Inference for Stochastic Processes, Boca Raton, CRC Press, 2018.   Google Scholar
[9]

J. A. Bucklew, Large Deviation Techniques in Decision, Simulation and Estimation, New York, Wiley, 1990.  Google Scholar

[10]

A. Caticha, Lectures on probability, entropy and statistical physics, Entropic Physics, preprint, arXiv: 0808.0012. Google Scholar

[11]

J.-R. Chazottes and D. Gabrielli, Large deviations for empirical entropies of g-measures, Nonlinearity, 18 (2005), 2545-2563.  doi: 10.1088/0951-7715/18/6/007.  Google Scholar

[12] T. M. Cover and J. A. Thomas, Elements of Information Theory, second edition, New York, Wiley Press, 2006.   Google Scholar
[13]

G. B. CybisS. R. C. Lopes and H. P. Pinheiro, Power of the likelihood ratio test for models of DNA base substitution, Journal of Applied Statistics, 38 (2011), 2723-2737.  doi: 10.1080/02664763.2011.567253.  Google Scholar

[14]

R. DakovicM. Denker and M. Gordin, Circular unitary ensembles: Parametric models and their asymptotic maximum likelihood estimates, Journal of Mathematical Sciences, 219 (2016), 714-730.  doi: 10.1007/s10958-016-3141-2.  Google Scholar

[15]

A. Dembo and O. Zeitouni, Large Deviation Techniques and Applications, New York, Springer Verlag, 2010. doi: 10.1007/978-3-642-03311-7.  Google Scholar

[16]

M. Denker, Basics of Thermodynamics, Lecture Notes - Penn State Univ., 2011. Google Scholar

[17]

M. Denker and W. Woyczynski, Introductory Statistics and Random Phenomena: Uncertainty, Complexity and Chaotic Behavior in Engineering and Science, New York, Birkhäuser, 2012. doi: 10.1007/978-3-319-66152-0.  Google Scholar

[18]

R. S. Ellis, Entropy, Large Deviations, and Statistical Mechanics, New York, Springer Verlag, 2006. doi: 10.1007/3-540-29060-5.  Google Scholar

[19]

H. H. Ferreira, A. O. Lopes and E. R. Oliveira, An iteration process for approximating subactions, Modeling, Dynamics, Optimization and Bioeconomics IV, Editors: Alberto Pinto and David Zilberman, Springer Proceedings in Mathematics and Statistics, New York, Springer Verlag (2021), 187-212. Google Scholar

[20]

V. GirardinL. Lhote and P. Regnault, Different closed-form expressions for generalized entropy rates of Markov chains, Methodology and Computing in Applied Probability, 21 (2019), 1431-1452.  doi: 10.1007/s11009-018-9679-3.  Google Scholar

[21]

V. Girardin and P. Regnault, Escort distributions minimizing the Kullback-Leibler divergence for a large deviations principle and tests of entropy level, Ann Inst Stat Math., 68 (2016), 439-468.  doi: 10.1007/s10463-014-0501-x.  Google Scholar

[22]

M. J. KarlingS. R. C. Lopes and R. M. de Souza, A Bayesian approach for estimating the parameters of an $\alpha$-stable distribution, Journal of Statistical Computation and Simulation, 91 (2021), 1713-1748.  doi: 10.1080/00949655.2020.1865958.  Google Scholar

[23]

Y. Kifer, Large deviations in dynamical systems and stochastic processes, Trans. Amer. Math. Soc., 321 (1990), 505-524.  doi: 10.1090/S0002-9947-1990-1025756-7.  Google Scholar

[24]

A. Lopes, Entropy, pressure and large deviation, Cellular Automata, Dynamical Systems and Neural Networks, E. Goles e S. Martinez (eds.), Kluwer, Massachusets, (1994), 79-146.  Google Scholar

[25]

A. O. Lopes, Thermodynamic formalism, maximizing probabilities and large deviations, Preprint - UFRGS. Google Scholar

[26]

A. O. Lopes, Entropy and large deviation, NonLinearity, 3 (1990), 527-546.  doi: 10.1088/0951-7715/3/2/013.  Google Scholar

[27]

A. O. Lopes, S. R. C. Lopes and P. Varandas, Bayes posterior convergence for loss functions via almost additive thermodynamic formalism, to appear in Journ. of Statis. Physics. Google Scholar

[28]

A. O. Lopes and J. K. Mengue, On information gain, Kullback-Leibler divergence, entropy production and the involution kernel, to appear in Disc. and Cont. Dyn. Syst. Series A. Google Scholar

[29]

A. O. Lopes and R. Ruggiero, Nonequilibrium in thermodynamic formalism: The second law, gases and information geometry, Qualitative Theory of Dynamical Systems, 21 (2022). doi: 10.1007/s12346-021-00551-0.  Google Scholar

[30]

K. McGoff, S. Mukherjee and A. Nobel, Gibbs posterior convergence and thermodynamic formalism, to appear in Adv. in Appl. Prob. Google Scholar

[31]

W. Parry and M. Pollicott, Zeta functions and the periodic orbit structure of hyperbolic dynamics, Astérisque, (1990), 187-188.  Google Scholar

[32]

V. K. Rohatgi, An Introduction to Probability Theory and Mathematical Statistics, New York: Wiley, 1976.  Google Scholar

[33]

T. Sagawa, Entropy, divergence and majorization in classical and quantum theory, arXiv: 2007.09974. Google Scholar

[34] Y. Suhov and M. Kelbert, Probability and Statistics by Example. I, Cambridge, Cambridge University Press, 2014.  doi: 10.1017/CBO9781139087773.  Google Scholar
[35]

D. A. van Dyk, The Role of statistics in the discovery of a higgs boson, Annual Review of Statistics and Its Application, 1 (2014), 41-59.  doi: 10.1146/annurev-statistics-062713-085841.  Google Scholar

[36]

A. C. D. van Enter, A. O. Lopes, S. R. C Lopes and J. K. Mengue, How to get the Bayesian a posteriori probability from an a priori probability via thermodynamic formalism for plans; the connection to disordered systems, work in progress. Google Scholar

[37] W. von der LindenV. Dose and U. von Toussaint, Bayesian Probability Theory Applications in the Physical Sciences, Cambridge, Cambridge University Press, 2014.  doi: 10.1017/CBO9781139565608.  Google Scholar

show all references

References:
[1] F. Abramovich and Y. Ritov, Statistical Theory: A Concise Introduction, Boca Raton, CRC Press, 2013.  doi: 10.1201/b14755.  Google Scholar
[2]

R. R. Bahadur, Large deviations of the maximum likelihood estimate in the Markov chain case, In J. S. Rostag, M. H. Rizvi and D. Siegmund, editors, Recent Advances in Statistics, 273-283. Boston, Academic Press, 1983. doi: 10.1016/B978-0-12-589320-6.50017-4.  Google Scholar

[3]

A. Baraviera, R. Leplaideur and A. O. Lopes, Ergodic Optimization, Zero Temperature and the Max-Plus Algebra, $23^{\text{o}}$ Colóquio Brasileiro de Matemática, IMPA, Rio de Janeiro, 2013.  Google Scholar

[4]

M. Barni and B. Tondi, Binary hypothesis testing game with training data, IEEE Transactions on Information Theory, 60 (2014), 4848-4866.  doi: 10.1109/TIT.2014.2325571.  Google Scholar

[5]

T. BenoistV. JakšićY. Pautrat and C.-A. Pillet, On entropy production of repeated quantum measurements I. General theory, Commun. Math. Phys., 357 (2018), 77-123.  doi: 10.1007/s00220-017-2947-1.  Google Scholar

[6]

D. Blackwell and M. A. Girshick, Theory of Games and Statistical Decisions, Dover publications, 1979.  Google Scholar

[7]

D. Bohle, A. Marynych and M. Meiners, A fundamental problem of hypotesis testing with finite e-commerce, Appl. Stoch. Models Bus. Ind., 37 (2021), 454-474. preprint, arXiv: 2006.05786. doi: 10.1002/asmb.2574.  Google Scholar

[8] L. D. Broemeling, Bayesian Inference for Stochastic Processes, Boca Raton, CRC Press, 2018.   Google Scholar
[9]

J. A. Bucklew, Large Deviation Techniques in Decision, Simulation and Estimation, New York, Wiley, 1990.  Google Scholar

[10]

A. Caticha, Lectures on probability, entropy and statistical physics, Entropic Physics, preprint, arXiv: 0808.0012. Google Scholar

[11]

J.-R. Chazottes and D. Gabrielli, Large deviations for empirical entropies of g-measures, Nonlinearity, 18 (2005), 2545-2563.  doi: 10.1088/0951-7715/18/6/007.  Google Scholar

[12] T. M. Cover and J. A. Thomas, Elements of Information Theory, second edition, New York, Wiley Press, 2006.   Google Scholar
[13]

G. B. CybisS. R. C. Lopes and H. P. Pinheiro, Power of the likelihood ratio test for models of DNA base substitution, Journal of Applied Statistics, 38 (2011), 2723-2737.  doi: 10.1080/02664763.2011.567253.  Google Scholar

[14]

R. DakovicM. Denker and M. Gordin, Circular unitary ensembles: Parametric models and their asymptotic maximum likelihood estimates, Journal of Mathematical Sciences, 219 (2016), 714-730.  doi: 10.1007/s10958-016-3141-2.  Google Scholar

[15]

A. Dembo and O. Zeitouni, Large Deviation Techniques and Applications, New York, Springer Verlag, 2010. doi: 10.1007/978-3-642-03311-7.  Google Scholar

[16]

M. Denker, Basics of Thermodynamics, Lecture Notes - Penn State Univ., 2011. Google Scholar

[17]

M. Denker and W. Woyczynski, Introductory Statistics and Random Phenomena: Uncertainty, Complexity and Chaotic Behavior in Engineering and Science, New York, Birkhäuser, 2012. doi: 10.1007/978-3-319-66152-0.  Google Scholar

[18]

R. S. Ellis, Entropy, Large Deviations, and Statistical Mechanics, New York, Springer Verlag, 2006. doi: 10.1007/3-540-29060-5.  Google Scholar

[19]

H. H. Ferreira, A. O. Lopes and E. R. Oliveira, An iteration process for approximating subactions, Modeling, Dynamics, Optimization and Bioeconomics IV, Editors: Alberto Pinto and David Zilberman, Springer Proceedings in Mathematics and Statistics, New York, Springer Verlag (2021), 187-212. Google Scholar

[20]

V. GirardinL. Lhote and P. Regnault, Different closed-form expressions for generalized entropy rates of Markov chains, Methodology and Computing in Applied Probability, 21 (2019), 1431-1452.  doi: 10.1007/s11009-018-9679-3.  Google Scholar

[21]

V. Girardin and P. Regnault, Escort distributions minimizing the Kullback-Leibler divergence for a large deviations principle and tests of entropy level, Ann Inst Stat Math., 68 (2016), 439-468.  doi: 10.1007/s10463-014-0501-x.  Google Scholar

[22]

M. J. KarlingS. R. C. Lopes and R. M. de Souza, A Bayesian approach for estimating the parameters of an $\alpha$-stable distribution, Journal of Statistical Computation and Simulation, 91 (2021), 1713-1748.  doi: 10.1080/00949655.2020.1865958.  Google Scholar

[23]

Y. Kifer, Large deviations in dynamical systems and stochastic processes, Trans. Amer. Math. Soc., 321 (1990), 505-524.  doi: 10.1090/S0002-9947-1990-1025756-7.  Google Scholar

[24]

A. Lopes, Entropy, pressure and large deviation, Cellular Automata, Dynamical Systems and Neural Networks, E. Goles e S. Martinez (eds.), Kluwer, Massachusets, (1994), 79-146.  Google Scholar

[25]

A. O. Lopes, Thermodynamic formalism, maximizing probabilities and large deviations, Preprint - UFRGS. Google Scholar

[26]

A. O. Lopes, Entropy and large deviation, NonLinearity, 3 (1990), 527-546.  doi: 10.1088/0951-7715/3/2/013.  Google Scholar

[27]

A. O. Lopes, S. R. C. Lopes and P. Varandas, Bayes posterior convergence for loss functions via almost additive thermodynamic formalism, to appear in Journ. of Statis. Physics. Google Scholar

[28]

A. O. Lopes and J. K. Mengue, On information gain, Kullback-Leibler divergence, entropy production and the involution kernel, to appear in Disc. and Cont. Dyn. Syst. Series A. Google Scholar

[29]

A. O. Lopes and R. Ruggiero, Nonequilibrium in thermodynamic formalism: The second law, gases and information geometry, Qualitative Theory of Dynamical Systems, 21 (2022). doi: 10.1007/s12346-021-00551-0.  Google Scholar

[30]

K. McGoff, S. Mukherjee and A. Nobel, Gibbs posterior convergence and thermodynamic formalism, to appear in Adv. in Appl. Prob. Google Scholar

[31]

W. Parry and M. Pollicott, Zeta functions and the periodic orbit structure of hyperbolic dynamics, Astérisque, (1990), 187-188.  Google Scholar

[32]

V. K. Rohatgi, An Introduction to Probability Theory and Mathematical Statistics, New York: Wiley, 1976.  Google Scholar

[33]

T. Sagawa, Entropy, divergence and majorization in classical and quantum theory, arXiv: 2007.09974. Google Scholar

[34] Y. Suhov and M. Kelbert, Probability and Statistics by Example. I, Cambridge, Cambridge University Press, 2014.  doi: 10.1017/CBO9781139087773.  Google Scholar
[35]

D. A. van Dyk, The Role of statistics in the discovery of a higgs boson, Annual Review of Statistics and Its Application, 1 (2014), 41-59.  doi: 10.1146/annurev-statistics-062713-085841.  Google Scholar

[36]

A. C. D. van Enter, A. O. Lopes, S. R. C Lopes and J. K. Mengue, How to get the Bayesian a posteriori probability from an a priori probability via thermodynamic formalism for plans; the connection to disordered systems, work in progress. Google Scholar

[37] W. von der LindenV. Dose and U. von Toussaint, Bayesian Probability Theory Applications in the Physical Sciences, Cambridge, Cambridge University Press, 2014.  doi: 10.1017/CBO9781139565608.  Google Scholar
Figure 1.  Graphs of $ P_0 $ (in solid line) and $ P_1 $ (in dashed line) for the functions defined in (3.27). For these plots we consider the data from the example in Section 7
Figure 2.  The large deviation rate function $ I_1(\cdot) $ at points $ v_1 $, $ G_1 $ and zero, where $ G_1 = E - \left(\int \log J_0 \, d \mu_{0} \, - \int \log \, J_1\, \, d\mu_0\, \right) $
Figure 3.  Graph of the function $ R(\lambda) = I_0^{E_{\lambda, \lambda}}(0) $, when $ 0\leq \lambda\leq \lambda_s $, using the stochastic matrix $ \mathcal{P}_{j} $, for $ j = 0, 1 $, from the example in Section 7
Figure 4.  Graphs of the functions $ \lambda \to \int \log J_\lambda d \mu_0 $ (in dotted line) and $ \lambda \to \int \log J_\lambda d \mu_1 $ (in dashed and dotted line) together with the graph of the values $ E_\lambda $ (in solid line), as a function of $ \lambda $, when $ 0\leq \lambda\leq \lambda_s $. The stochastic matrix $ \mathcal{P}_{j} $, for $ j = 0, 1 $, is from the example in Section 7
[1]

Onur Teymur, Sarah Filippi. A Bayesian nonparametric test for conditional independence. Foundations of Data Science, 2020, 2 (2) : 155-172. doi: 10.3934/fods.2020009

[2]

Luis Barreira. Nonadditive thermodynamic formalism: Equilibrium and Gibbs measures. Discrete & Continuous Dynamical Systems, 2006, 16 (2) : 279-305. doi: 10.3934/dcds.2006.16.279

[3]

José M. Amigó, Ángel Giménez. Formulas for the topological entropy of multimodal maps based on min-max symbols. Discrete & Continuous Dynamical Systems - B, 2015, 20 (10) : 3415-3434. doi: 10.3934/dcdsb.2015.20.3415

[4]

Jinchuan Zhou, Changyu Wang, Naihua Xiu, Soonyi Wu. First-order optimality conditions for convex semi-infinite min-max programming with noncompact sets. Journal of Industrial & Management Optimization, 2009, 5 (4) : 851-866. doi: 10.3934/jimo.2009.5.851

[5]

Roberto Livrea, Salvatore A. Marano. A min-max principle for non-differentiable functions with a weak compactness condition. Communications on Pure & Applied Analysis, 2009, 8 (3) : 1019-1029. doi: 10.3934/cpaa.2009.8.1019

[6]

Meixia Li, Changyu Wang, Biao Qu. Non-convex semi-infinite min-max optimization with noncompact sets. Journal of Industrial & Management Optimization, 2017, 13 (4) : 1859-1881. doi: 10.3934/jimo.2017022

[7]

Alvaro Sandroni, Eran Shmaya. A prequential test for exchangeable theories. Journal of Dynamics & Games, 2014, 1 (3) : 497-505. doi: 10.3934/jdg.2014.1.497

[8]

Li Li, Xinzhen Zhang, Zheng-Hai Huang, Liqun Qi. Test of copositive tensors. Journal of Industrial & Management Optimization, 2019, 15 (2) : 881-891. doi: 10.3934/jimo.2018075

[9]

Lucian Coroianu, Sorin G. Gal. New approximation properties of the Bernstein max-min operators and Bernstein max-product operators. Mathematical Foundations of Computing, 2021  doi: 10.3934/mfc.2021034

[10]

Natalia Tokareva. On the number of bent functions from iterative constructions: lower bounds and hypotheses. Advances in Mathematics of Communications, 2011, 5 (4) : 609-621. doi: 10.3934/amc.2011.5.609

[11]

Vaughn Climenhaga. A note on two approaches to the thermodynamic formalism. Discrete & Continuous Dynamical Systems, 2010, 27 (3) : 995-1005. doi: 10.3934/dcds.2010.27.995

[12]

J. Mead. $ \chi^2 $ test for total variation regularization parameter selection. Inverse Problems & Imaging, 2020, 14 (3) : 401-421. doi: 10.3934/ipi.2020019

[13]

Segismundo S. Izquierdo, Luis R. Izquierdo. "Test two, choose the better" leads to high cooperation in the Centipede game. Journal of Dynamics & Games, 2021  doi: 10.3934/jdg.2021018

[14]

Yu-Chi Chen. Security analysis of public key encryption with filtered equality test. Advances in Mathematics of Communications, 2021  doi: 10.3934/amc.2021053

[15]

Michael Jakobson, Lucia D. Simonelli. Countable Markov partitions suitable for thermodynamic formalism. Journal of Modern Dynamics, 2018, 13: 199-219. doi: 10.3934/jmd.2018018

[16]

Manfred Denker, Yuri Kifer, Manuel Stadlbauer. Thermodynamic formalism for random countable Markov shifts. Discrete & Continuous Dynamical Systems, 2008, 22 (1&2) : 131-164. doi: 10.3934/dcds.2008.22.131

[17]

Yongluo Cao, De-Jun Feng, Wen Huang. The thermodynamic formalism for sub-additive potentials. Discrete & Continuous Dynamical Systems, 2008, 20 (3) : 639-657. doi: 10.3934/dcds.2008.20.639

[18]

Anna Mummert. The thermodynamic formalism for almost-additive sequences. Discrete & Continuous Dynamical Systems, 2006, 16 (2) : 435-454. doi: 10.3934/dcds.2006.16.435

[19]

Manfred Denker, Yuri Kifer, Manuel Stadlbauer. Corrigendum to: Thermodynamic formalism for random countable Markov shifts. Discrete & Continuous Dynamical Systems, 2015, 35 (1) : 593-594. doi: 10.3934/dcds.2015.35.593

[20]

Yakov Pesin. On the work of Sarig on countable Markov chains and thermodynamic formalism. Journal of Modern Dynamics, 2014, 8 (1) : 1-14. doi: 10.3934/jmd.2014.8.1

 Impact Factor: 

Article outline

Figures and Tables

[Back to Top]