Advanced Search
Article Contents
Article Contents

On entropy, entropy-like quantities, and applications

Abstract Related Papers Cited by
  • This is a review on entropy in various fields of mathematics and science. Its scope is to convey a unified vision of the classical entropies and some newer, related notions to a broad audience with an intermediate background in dynamical systems and ergodic theory. Due to the breadth and depth of the subject, we have opted for a compact exposition whose contents are a compromise between conceptual import and instrumental relevance. The intended technical level and the space limitation born furthermore upon the final selection of the topics, which cover the three items named in the title. Specifically, the first part is devoted to the avatars of entropy in the traditional contexts: many particle physics, information theory, and dynamical systems. This chronological order helps present the materials in a didactic manner. The axiomatic approach will be also considered at this stage to show that, quite remarkably, the essence of entropy can be encapsulated in a few basic properties. Inspired by the classical entropies, further akin quantities have been proposed in the course of time, mostly aimed at specific needs. A common denominator of those addressed in the second part of this review is their major impact on research. The final part shows that, along with its profound role in the theory, entropy has interesting practical applications beyond information theory and communications technology. For this sake we preferred examples from applied mathematics, although there are certainly nice applications in, say, physics, computer science and even social sciences. This review concludes with a representative list of references.
    Mathematics Subject Classification: Primary: 28D20, 37A35, 37B40, 94A17; Secondary: 37M25.


    \begin{equation} \\ \end{equation}
  • [1]

    S. Abe, Tsallis entropy: How unique?, Contin. Mech. Thermodyn., 16 (2004), 237-244.doi: 10.1007/s00161-003-0153-1.


    U. R. Acharya, O. Faust, N. Kannathal, T. Chua and S. Laxminarayan, Non-linear analysis of EEG signals at various sleep stages, Comput. Meth. Prog. Bio., 80 (2005), 37-45.doi: 10.1016/j.cmpb.2005.06.011.


    U. R. Acharya, K. P. Joseph, N. Kannathal, C. M. Lim and J. S. Suri, Heart rate variability: A review, Advances in Cardiac Signal Processing, (2007), 121-165.doi: 10.1007/978-3-540-36675-1_5.


    J. Aczél and Z. Daróczy, Charakterisierung der Entropien positiver Ordnung und der Shannonschen Entropie, Acta Math. Acad. Sci. Hung., 14 (1963), 95-121.doi: 10.1007/BF01901932.


    R. Adler, A. Konheim and M. McAndrew, Topological entropy, Trans. Amer. Mat. Soc., 114 (1965), 309-319.doi: 10.1090/S0002-9947-1965-0175106-9.


    R. L. Adler and B. Marcus, Topological entropy and the equivalence of dynamical systems, Mem. Amer. Math. Soc., 20 (1979), iv+84 pp.doi: 10.1090/memo/0219.


    V. Afraimovich, M. Courbage and L. Glebsky, Directional complexity and entropy for lift mappings, Discr. Contin. Dyn. Syst. B, 20 (2015), 3385-3401.


    L. Alsedà, J. Llibre and M. Misiurewicz, Combinatorial Dynamics and Entropy in Dimension One, World Scientific, Singapore, 2000.doi: 10.1142/4205.


    J. M. Amigó, L. Kocarev and J. Szczepanski, Order patterns and chaos, Phys. Lett. A, 355 (2006), 27-31.


    J. M. Amigó, S. Zambrano and M. A. F. Sanjuán, True and false forbidden patterns in deterministic and random dynamics, Eur. Phys. Lett., 79 (2007), 50001, 5pp.doi: 10.1209/0295-5075/79/50001.


    J. M. Amigó and M. B. Kennel, Forbidden ordinal patterns in higher dimensional dynamics, Physica D, 237 (2008), 2893-2899.doi: 10.1016/j.physd.2008.05.003.


    J. M. Amigó, S. Zambrano and M. A. F. Sanjuán, Combinatorial detection of determinism in noisy time series, Europhys. Lett., 82 (2008), 60005.


    J. M. Amigó, Permutation Complexity in Dynamical Systems-Ordinal Patterns, Permutation Entropy, and All That, Springer Series in Synergetics, Springer, Berlin Heidelberg, 2010.doi: 10.1007/978-3-642-04084-9.


    J. M. Amigó, S. Zambrano and M. A. F. Sanjuán, Detecting determinism in time series with ordinal patterns: A comparative study, Int. J. Bif. Chaos, 20 (2010), 2915-2924.doi: 10.1142/S0218127410027453.


    J. M. Amigó, R. Monetti, T. Aschenbrenner and W. Bunk, Transcripts: An algebraic approach to coupled time series, Chaos, 22 (2012), 013105, 13pp.doi: 10.1063/1.3673238.


    J. M. Amigó, The equality of Kolmogorov-Sinai entropy and metric permutation entropy generalized, Physica D, 241 (2012), 789-793.doi: 10.1016/j.physd.2012.01.004.


    J. M. Amigó and K. Keller, Permutation entropy: One concept, two approaches, Eur. Phys. J. Special Topics, 222 (2013), 263-273.


    J. M. Amigó, P. Kloeden and A. Giménez, Switching systems and entropy, J. Diff. Eq. Appl., 19 (2013), 1872-1888.doi: 10.1080/10236198.2013.788166.


    J. M. Amigó, P. E. Kloeden and A. Giménez, Entropy increase in switching systems, Entropy, 15 (2013), 2363-2383.doi: 10.3390/e15062363.


    J. M. Amigó, T. Aschenbrenner, W. Bunk and R. Monetti, Dimensional reduction of conditional algebraic multi-information via transcripts, Inform. Sciences, 278 (2014), 298-310.doi: 10.1016/j.ins.2014.03.054.


    J. M. Amigó and A. Giménez, A simplified algorithm for the topological entropy of multimodal maps, Entropy, 16 (2014), 627-644.doi: 10.3390/e16020627.


    J. M. Amigó, K. Keller and V. A. Unakafova, Ordinal symbolic analysis and its application to biomedical recordings}, Phil. Trans. R. Soc. A, 373 (2015), 20140091, 18pp.doi: 10.1098/rsta.2014.0091.


    J. M. Amigó and A. Giménez, Formulas for the topological entropy of multimodal maps based on min-max symbols, Discr. Contin. Dyn. Syst. B, 20 (2015), 3415-3434.


    R. G. Andrzejak, K. Lehnertz, F. Mormann, C. Rieke, P. David and C. E. Elger, Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: Dependence on recording region and brain state, Phys. Rev. E, 64 (2001), 061907.doi: 10.1103/PhysRevE.64.061907.


    A. Antoniouk, K. Keller and S. Maksymenko, Kolmogorov-Sinai entropy via separation properties of order-generated sigma-algebras, Discr. Contin. Dyn. Syst. A, 34 (2014), 1793-1809.


    R. B. Ash, Information Theory, Dover Publications, New York, 1990.


    C. Bandt, G. Keller and B. Pompe, Entropy of interval maps via permutations, Nonlinearity, 15 (2002) 1595-1602.doi: 10.1088/0951-7715/15/5/312.


    E. Beadle, J. Schroeder, B Moran and S. Suvorova, An overview of Rényi Entropy and some potential applications, in 42nd Asilomar Conference on Signals, Systems and Computers, Pacific Grove, 2008, IEEE Explore, 2008, 1698-1704.doi: 10.1109/ACSSC.2008.5074715.


    C. H. Bennet, Notes on Landauer's principle, reversible computation and Maxwell's demon, Stud. Hist. Philos. M. P., 34 (2003), 501-510.doi: 10.1016/S1355-2198(03)00039-X.


    G. D. Birkhoff, Proof of a recurrence theorem for strongly transitive systems, PNAS, 17 (1931), 650-655.doi: 10.1073/pnas.17.12.650.


    S. A. Borovkova, Estimation and Prediction for Nonlinear Time Series, Ph.D thesis, University of Groningen, 1998.


    M. Boyle and D. Lind, Expansive subdynamics, Trans. Amer. Math. Soc., 349 (1997), 55-102.doi: 10.1090/S0002-9947-97-01634-6.


    R. Bowen, Entropy for group endomorphisms and homogeneous spaces, Trans. Amer. Math. Soc., 153 (1971), 401-414.doi: 10.1090/S0002-9947-1971-0274707-X.


    L. Bowen, A measure-conjugacy invariant for free group actions, Ann. of Math., 171 (2010), 1387-1400.doi: 10.4007/annals.2010.171.1387.


    L. Bowen, Measure-conjugacy invariants for actions of countable sofic groups, J. Amer. Math. Soc., 23 (2010), 217-245.doi: 10.1090/S0894-0347-09-00637-7.


    H. W. Broer and F. Takens, Dynamical Systems and Chaos, Applied Mathematical Sciences, 172, Springer, New York, 2011.doi: 10.1007/978-1-4419-6870-8.


    M. Brin and A. Katok, On local entropy, in Geometric dynamics (ed. J. Palis), Lecture Notes in Mathematics, 1007, Springer, Berlin Heidelberg, 1983, 30-38.doi: 10.1007/BFb0061408.


    A. A. Bruzzo, B. Gesierich, M. Santi, C. A. Tassinari, N. Birbaumer and G. Rubboli, Permutation entropy to detect vigilance changes and preictal states from scalp EEG in epileptic patients. A preliminary study, Neurol. Sci., 29 (2008), 3-9.doi: 10.1007/s10072-008-0851-3.


    N. Burioka, M. Miyata, G. Cornélissen, F. Halberg, T. Takeshima, D. T. Kaplan, H. Suyama, M. Endo, Y. Maegaki and T. Nomura, et. al, Approximate entropy in the electroencephalogram during wake and sleep, Clin. EEG Neurosci., 36 (2005), 21-24.doi: 10.1177/155005940503600106.


    C. Cafaro, W. M. Lord, J. Sun and E. M. Bollt, Causation entropy from symbolic representations of dynamical systems, Chaos, 25 (2015), 043106.


    J. S. Cánovas and A. Guillamón, Permutations and time series analysis, Chaos, 19 (2009), 043103, 12pp.doi: 10.1063/1.3238256.


    Y. Cao, W.-W. Tung, J. B. Gao, V. A. Protopopescu and L. M. Hively, Detecting dynamical changes in time series using the permutation entropy, Phys. Rev. E, 70 (2004), 046217, 7pp.doi: 10.1103/PhysRevE.70.046217.


    A. Capurro, L. Diambra, D. Lorenzo, O. Macadar, M. T. Martin, C. Mostaccio, A. Plastino, E. Rofman, M. E. Torres and J. Velluti, Tsallis entropy and cortical dynamics: The analysis of EEG signals, Physica A, 257 (1998), 149-155.doi: 10.1016/S0378-4371(98)00137-X.


    A. Capurro, L. Diambra, D. Lorenzo, O. Macadar, M. T. Martin, C. Mostaccio, A. Plastino, J. Perez, E. Rofman and M. E. Torres, Human brain dynamics: The analysis of EEG signals with Tsallis information measure, Physica A, 265 (1999), 235-254.doi: 10.1016/S0378-4371(98)00471-3.


    D. Carrasco-Olivera, R. Metzger and C. A. Morales, Topological entropy for set-valued maps, Discr. Contin. Dyn. Syst. B, 20 (2015), 3461-3474.


    H. C. Choe, Computational Ergodic Theory, Algorithms and Computation in Mathematics, 13, Springer, Berlin Heidelberg, 2005.


    R. Clausius, The Mechanical Theory of Heat, MacMillan and Co., London, 1865.


    M. Courbage and B. Kamiński, Space-time directional Lyapunov exponents for cellular automata, J. Statis. Phys., 124 (2006), 1499-1509.doi: 10.1007/s10955-006-9172-1.


    T. M. Cover and J. A. Thomas, Elements of Information Theory, John Wiley & Sons, New York, 2006.


    B. Dai and B. Hu, Minimum conditional entropy clustering: A discriminative framework for clustering, in JMLR: Workshop and Conference Proceedings, 13, Tokyo, 2010, 47-62.


    K. Denbigh, How subjective is entropy, in Maxwell's Demon, Entropy, Information, Computing (eds. H. S. Leff and A. F. Rex), Princeton University Press, Princeton, 1990, 109-115.


    M. Denker, Finite generators for ergodic, measure-preserving transformation, Zeit. Wahr. ver. Geb., 29 (1974), 45-55.doi: 10.1007/BF00533186.


    M. Denker, C. Grillenberger and K. Sigmund, Ergodic Theory on Compact Spaces, Lecture Notes in Mathematics, {527}, Springer, Berlin Heidelberg, 1976.


    M. Denker and G. Keller, Rigorous statistical procedures for data from dynamical systems, J. Stat. Phys., 44 (1986), 67-93.doi: 10.1007/BF01010905.


    E. I. Dinaburg, The relation between topological entropy and metric entropy, Soviet Math., 11 (1970), 13-16.


    R. J. V. dos Santos, Generalization of Shannon's theorem for Tsallis entropy, J. Math. Phys., 38 (1997), 4104-4107.doi: 10.1063/1.532107.


    J.-P. Eckmann and D. Ruelle, Ergodic theory of chaos and strange attractors, Rev. Mod. Phys., 57 (1985), 617-656.doi: 10.1103/RevModPhys.57.617.


    M. Einsiedler and T. Ward, Ergodic Theory with a View Towards Number Theory, Graduate Texts in Mathematics, {259}, Springer, London, 2011.doi: 10.1007/978-0-85729-021-2.


    D. K. Faddeev, On the notion of entropy of finite probability distributions, Usp. Mat. Nauk (in Russian), 11 (1956), 227-231.


    F. Falniowski, On the connections of the generalized entropies and Kolmogorov-Sinai entropies, Entropy, 16 (2014), 3732-3753.doi: 10.3390/e16073732.


    B. Frank, B. Pompe, U. Schneider and D. Hoyer, Permutation entropy improves fetal behavioural state classification based on heart rate analysis from biomagnetic recordings in near term fetuses, Med. Biol. Eng. Comput., 44 (2006), 179-187.doi: 10.1007/s11517-005-0015-z.


    S. Furuichi, On uniqueness Theorems for Tsallis entropy and Tsallis relative entropy, IEEE Trans. Inf. Theory, 51 (2005), 3638-3645.doi: 10.1109/TIT.2005.855606.


    L. G. Gamero, A. Plastino and M. E. Torres, Wavelet analysis and nonlinear dynamics in a nonextensive setting, Physica A, 246 (1997), 487-509.doi: 10.1016/S0378-4371(97)00367-1.


    P. G. Gaspard and X. J. Wang, Noise, chaos, and $(\varepsilon,\tau)$-entropy per unit time, Phys. Rep., 235 (1993), 291-343.doi: 10.1016/0370-1573(93)90012-3.


    T. N. T. Goodman, Relating topological entropy and measure entropy, Bull. London Math. Soc., 3 (1971), 176-180.doi: 10.1112/blms/3.2.176.


    N. Gradojevic and R. Genccay, Financial applications of nonextensive entropy, IEEE Signal Process. Mag., 28 (2011), p116.


    B. Graff, G. Graff and A. Kaczkowska, Entropy Measures of Heart Rate Variability for Short ECG Datasets in Patients with Congestive Heart Failure, Acta Phys. Pol. B Proc. Suppl., 5 (2012), 153-158.


    C. W. J. Granger, Investigating causal relations by econometric models and cross-spectral methods, Econometrica, 37 (1969), 424-438.


    P. Grassberger and I. Procaccia, Dimensions and entropies of strange attractors from a fluctuating dynamics approach, Physica D, 13 (1984), 34-54.doi: 10.1016/0167-2789(84)90269-0.


    P. Grassberger and I. Procaccia, Measuring the strangeness of strange attractors, Physica D, 9 (1983), 189-208.doi: 10.1016/0167-2789(83)90298-1.


    P. Grassberger and I. Procaccia, Characterization of strange attractors, Phys. Rev. Lett., 50 (1983), 346-349.doi: 10.1103/PhysRevLett.50.346.


    D. W. Hahs and S. D. Pethel, Distinguishing anticipation from causality: Anticipatory bias in the estimation of information flow, Phys. Rev. Lett., 107 (2011), 128701.


    R. Hanel and S. Thurner, A comprehensive classification of complex statistical systems and an axiomatic derivation of their entropy and equidistribution functions, EPL, 93 (2011), 20006.


    R. Hanel, S. Thurner and M. Gell-Mann, Generalized entropies and logarithms and their duality relations, Proceed. Nat. Acad. Scie., 109 (2012), 19151-19154.doi: 10.1073/pnas.1216885109.


    B. Hasselblatt and A. Katok, Principal structures, in Handbook of Dynamical Systems (eds. B. Hasselblatt and A. Katok), 1A, North Holland, Amsterdam, 2002, 1-203.doi: 10.1016/S1874-575X(02)80003-0.


    J. Havrda and F. Charvát, Quantification method of classification processes. Concept of structural $\alpha$-entropy, Kybernetika, 3 (1967), 30-35.


    G. A. Hedlund, Endomorphisms and automorphisms of the shift dynamical system, Math. Syst. Theory, 3 (1969), 320-375.doi: 10.1007/BF01691062.


    H. G. E. Hentschel and I. Procaccia, The infinite number of generalized dimensions of fractals and strange attractors, Physica D, 8 (1983), 435-444.doi: 10.1016/0167-2789(83)90235-X.


    R. Hornero, D. Abasolo, J. Escuredo and C. Gomez, Nonlinear analysis of electroencephalogram and magnetoencephalogram recordings in patients with Alzheimer's disease, Phil. Trans. R. Soc. A, 367 (2009), 317-336.doi: 10.1098/rsta.2008.0197.


    V. M. Ilić, M. S. Stanković and E. H. Mulalić, Comments on "Generalization of Shannon-Khinchin axioms to nonextensive systems and the uniqueness theorem for the nonextensive entropy'', IEEE Trans. Inf. Theory, 59 (2013), 6950-6952.


    E. T. Jaynes, Information theory and statistical mechanics, Phys. Rev., 106 (1957), 620-630.doi: 10.1103/PhysRev.106.620.


    P. Jizba and T. Arimitsu, The world according to Rényi: Thermodynamics of multifractal systems, Ann. Phys., 312 (2004), 17-59.doi: 10.1016/j.aop.2004.01.002.


    C. C. Jouny and G. K. Bergey, Characterization of early partial seizure onset: Frequency, complexity and entropy, Clin. Neurophysiol., 123 (2012), 658-669.doi: 10.1016/j.clinph.2011.08.003.


    N. Kannathal, M. L. Choo, U. R. Acharya and P. K. Sadasivan, Entropies for detection of epilepsy in EEG, Comput. Meth. Prog. Bio., 80 (2005), 187-194.doi: 10.1016/j.cmpb.2005.06.012.


    H. Kantz and T. Schreiber, Nonlinear Time Series Analysis, Cambridge University Press, Cambridge, 2004.


    A. Katok, Fifty years of entropy in dynamics: 1978-2007, J. Mod. Dyn., 1 (2007), 545-596.doi: 10.3934/jmd.2007.1.545.


    K. Keller and H. Lauffer, Symbolic analysis of high-dimensional time series, Int. J. Bif. Chaos, 13 (2003), 2657-2668.doi: 10.1142/S0218127403008168.


    K. Keller, Permutations and the Kolmogorov-Sinai entropy, Discr. Contin. Dyn. Syst. A, 32 (2012), 891-900.doi: 10.3934/dcds.2012.32.891.


    K. Keller, A. M. Unakafov and V. A. Unakafova, Ordinal Patterns, Entropy, and EEG, Entropy, 16 (2014), 6212-6239.doi: 10.3390/e16126212.


    K. Keller, S. Maksymenko and I. Stolz, Entropy determination based on the ordinal structure of a dynamical system, Discr. Contin. Dyn. Syst. B, 20 (2015), 3507-3524.


    A. I. Khinchin, Mathematical Foundations of Information Theory, Dover, New York, 1957.


    A. N. Kolmogorov, A new metric invariant of transitive dynamical systems and Lebesgue space endomorphisms, Dokl. Acad. Sci. USSR, 119 (1958), 861-864.


    S. Kolyada and L. Snoha, Topological entropy of nonautonomous dynamical systems, Random Comput. Dynam., 4 (1996), 205-233.


    Z. S. Kowalski, Finite generators of ergodic endomorphisms, Colloq. Math., 49 (1984), 87-89.


    Z. S. Kowalski, Minimal generators for aperiodic endomorphisms, Commentat. Math. Univ. Carol., 36 (1995), 721-725.


    W. Krieger, On entropy and generators of measure-preserving transformations, Trans. Amer. Math. Soc., 149 (1970), 453-464.doi: 10.1090/S0002-9947-1970-0259068-3.


    J. Kurths, A. Voss, P. Saparin, A. Witt, H. J. Kleiner and N. Wessel, Quantitative analysis of heart rate variability, Chaos, 5 (1995), 88-94.doi: 10.1063/1.166090.


    D. E. Lake, J. S. Richman, M. P. Griffin and J. R. Moorman, Sample entropy analysis of neonatal heart rate variability, Am. J. Physiol.-Reg. I., 283 (2002), 789-797.doi: 10.1152/ajpregu.00069.2002.


    A. M. Law and W. D. Kelton, Simulation, Modeling, and Analysis, McGraw-Hill, Boston, 2000.


    F. Ledrappier and L.-S. Young, The metric entropy of diffeomorphisms. II. Relations betweenentropy, exponents and dimension, Ann. Math., 122 (1985), 540-574.doi: 10.2307/1971329.


    X. Li, G. Ouyang and D. A. Richards, Predictability analysis of absence seizures with permutation entropy, Epilepsy Res., 77 (2007), 70-74.doi: 10.1016/j.eplepsyres.2007.08.002.


    J. Li, J. Yan, X. Liu and G. Ouyang, Using permutation entropy to measure the changes in eeg signals during absence seizures, Entropy, 16 (2014), 3049-3061.doi: 10.3390/e16063049.


    H. Li, K. Zhang and T. Jiang, Minimum entropy clustering and applications to gene expression analysis, in Proc. IEEE Comput. Syst. Bioinform. Conf., 2004, 142-151.


    Z. Liang, Y. Wang, X. Sun, D. Li, L. J. Voss, J. W. Sleigh, S. Hagihira and X. Li, EEG entropy measures in anesthesia, Front. Comput. Neurosci., 9 (2015), p16.doi: 10.3389/fncom.2015.00016.


    D. Lind and B. Marcus, An Introduction to Symbolic Dynamics and Coding, Cambridge University Press, Cambridge, 1995.doi: 10.1017/CBO9780511626302.


    J. Llibre, Brief survey on the topological entropy, Discr. Contin. Dyn. Syst. B, 20 (2015), 3363-3374.


    N. Mammone, F. La Foresta and F. C. Morabito, Automatic artifact rejection from multichannel scalp EEG by wavelet ICA, IEEE Sens. J., 12 (2012), 533-542.doi: 10.1109/JSEN.2011.2115236.


    M. Matilla-García, A non-parametric test for independence based on symbolic dynamics, J. Econ. Dyn. Control, 31 (2007), 3889-3903.doi: 10.1016/j.jedc.2007.01.018.


    M. Matilla-García and M. Ruiz Marín, A non-parametric independence test using permutation entropy, J. Econ., 144 (2008), 139-155.doi: 10.1016/j.jeconom.2007.12.005.


    A. M. Mesón and F. Vericat, On the Kolmogorov-like generalization of Tsallis entropy, correlation entropies and multifractal analysis, J. Math. Phys., 43 (2002), 904-917.doi: 10.1063/1.1429323.


    R. Miles and T. Ward, Directional uniformities, periodic points, and entropy, Discr. Contin. Dyn. Syst. B, 20 (2015), 3525-3545.


    J. Milnor, On the entropy geometry of cellular automata, Complex Systems, 2 (1988), 357-385.


    J. Milnor and W. Thurston, On iterated maps of the interval, in Dynamical Systems (ed. J. C. Alexander), Lectures Notes in Mathematics, 1342, Springer, 1988, 465-563.doi: 10.1007/BFb0082847.


    M. Misiurewicz and W. Szlenk, Entropy of piecewise monotone mappings, Studia Math., 67 (1980), 45-63.


    M. Misiurewicz, Permutations and topological entropy for interval maps, Nonlinearity, 16 (2003), 971-976.doi: 10.1088/0951-7715/16/3/310.


    R. Monetti, J. M. Amigó, T. Aschenbrenner and W. Bunk, Permutation complexity of interacting dynamical systems, Eur. Phys. J. Special Topics, 222 (2013), 421-436.doi: 10.1140/epjst/e2013-01850-y.


    R. Monetti, W. Bunk, T. Aschenbrenner, S. Springer and J. M. Amigó, Information directionality in coupled time series using transcripts, Phys. Rev. E, 88 (2013), 022911.doi: 10.1103/PhysRevE.88.022911.


    F. C. Morabito, D. Labate, F. La Foresta, A. Bramanti, G. Morabito and I. Palamara, Multivariate multi-scale permutation entropy for complexity analysis of Alzheimer's disease EEG, Entropy, 14 (2012), 1186-1202.doi: 10.3390/e14071186.


    N. Nicolaou and J. Georgiou, The use of permutation entropy to characterize sleep electroencephalograms, Clin. EEG Neurosci., 42 (2011), 24-28.doi: 10.1177/155005941104200107.


    D. Ornstein, Two Bernoulli shifts with the same entropy are isomorphic, Adv. Math., 4 (1970), 337-352.doi: 10.1016/0001-8708(70)90029-0.


    D. Ornstein, Two Bernoulli shifts with infinite entropy are isomorphic, Adv. Math., 5 (1970), 339-348.doi: 10.1016/0001-8708(70)90008-3.


    D. Ornstein and B. Weiss, Entropy and isomorphism theorem for actions of amenable groups, J. Analyse Math., 48 (1987), 1-141.doi: 10.1007/BF02790325.


    D. Ornstein, Newton's law and coin tossing, Notices Amer. Math. Soc., 60 (2013), 450-459.doi: 10.1090/noti974.


    D. Ornstein and B. Weiss, Entropy is the only finitely observable invariant, J. Mod. Dyn., 1 (2007), 93-105.


    G. Ouyang, C. Dang, D. A. Richards and X. Li, Ordinal pattern based similarity analysis for EEG recordings, Clin. Neurophysiol, 121 (2010), 694-703.doi: 10.1016/j.clinph.2009.12.030.


    S. Y. Park and A. K. Bera, Maximum entropy autoregressive conditional heretoskedasticity model, J. Econometrics, 150 (2009), 219-230.doi: 10.1016/j.jeconom.2008.12.014.


    U. Parlitz, S. Berg, S. Luther, A. Schirdewan, J. Kurths and N. Wessel, Classifying cardiac biosignals using ordinal pattern statistics and symbolic dynamics, Comput. Biol. Med., 42 (2012), 319-327.doi: 10.1016/j.compbiomed.2011.03.017.


    W. Parry, Intrinsic Markov chains, Trans. Amer. Math. Soc., 112 (1964), 55-66.doi: 10.1090/S0002-9947-1964-0161372-1.


    Y. Pesin, Characteristic Lyapunov exponents and smooth ergodic theory, Russian Math. Surveys, 32 (1977), 55-112.


    S. M. Pincus, Approximate entropy as a measure of system complexity, PNAS, 88 (1991), 2297-2301.doi: 10.1073/pnas.88.6.2297.


    S. M. Pincus and R. R. Viscarello, Approximate entropy: A regularity measure for fetal heart rate analysis, Obstet. Gynecol., 79 (1992), 249-255.


    S. M. Pincus, Approximate entropy as a measure of irregularity for psychiatric serial metrics, Bipolar Disord., 8 (2006), 430-440.doi: 10.1111/j.1399-5618.2006.00375.x.


    B. Pompe and J. Runge, Momentary information transfer as a coupling measure of time series, Phys. Rev. E, 83 (2011), 051122.doi: 10.1103/PhysRevE.83.051122.


    J. Poza, R. Hornero, J. Escudero, A. Fernández and C. A. Sánchez, Regional analysis of spontaneous MEG rhythms in patients with Alzheimer's disease using spectral entropies, Ann. Biomed. Eng., 36 (2008), 141-152.doi: 10.1007/s10439-007-9402-y.


    J. C. Principe, Information Theoretic Learning. Renyi's Entropy and Kernel Perspectives, Springer, New York, 2010.doi: 10.1007/978-1-4419-1570-2.


    A. Rényi, On measures of entropy and information, in Proceedings of the 4th Berkeley Symposium on Mathematics, Statistics and Probability (ed. J. Neyman), University of California Press, Berkeley, 1961, 547-561.


    J. S. Richman and J. R. Moorman, Physiological time-series analysis using approximate entropy and sample entropy, Am. J. Physiol.-Heart C., 278 (2000), 2039-2049.


    S. J. Roberts, R. Everson and I. Rezek, Minimum entropy data partitioning, in Proceedings of International Conference on Artificial Neural Networks, 1999, 844-849.


    E. A. Robinson and A. Şahin, Rank-one $\mathbbZ^d$ actions and directional entropy, Ergod. Theor. Dynam. Syst., 31 (2011), 285-299.doi: 10.1017/S0143385709000911.


    O. A. Rosso, M. T. Martin and A. Plastino, Brain electrical activity analysis using wavelet-based informational tools, Physica A, 313 (2002), 587-608.doi: 10.1016/S0378-4371(02)00958-5.


    D. Rudolph and B. Weiss, Entropy and mixing for amenable group actions, Ann. of Math., 151 (2000), 1119-1150.doi: 10.2307/121130.


    D. Ruelle, Statistical mechanics on a compact set with $Z^{\nu}$ action satisfying expansiveness and specification, Trans. Amer. Math. Soc., 187 (1973), 237-251.


    J. Runge, J. Heitzig, N. Marwan and J. Kurths, Quantifying causal coupling strength: A lag-specific measure for multivariate time series related to transfer entropy, Phys. Rev. E, 86 (2012), 061121.


    X. San Liang, Unraveling the cause-effect relation between time series, Phys. Rev. E, 90 (2014), 052150.


    T. Schreiber, Measuring information transfer, Phys. Rev. Lett., 85 (2000), 461-464.doi: 10.1103/PhysRevLett.85.461.


    C. E. Shannon, A mathematical theory of communication, Bell Syst. Tech. J., 27 (1948), 379-423, 623-653.doi: 10.1002/j.1538-7305.1948.tb01338.x.


    Y. G. Sinai, On the notion of entropy of dynamical systems, Dokl. Acad. Sci. USSR, 125 (1959), 768-771.


    Y. G. Sinai, Flows with finite entropy, Dokl. Acad. Sci. USSR, 125 (1959), 1200-1202.


    Y. G. Sinai, Gibbs measures in ergodic theory, Uspehi Mat. Nauk, 27 (1972), 21-64.


    D. Smirnov, Spurious causalities with transfer entropy, Phys. Rev. E, 87 (2013), 042917.


    M. Smorodinsky, Ergodic Theory, Entropy, Lectures Notes in Mathematics, 214, Springer, Berlin Heidelberg, 1971.


    R. Sneddon, The Tsallis entropy of natural information, Physica A, 386 (2007), 101-118.doi: 10.1016/j.physa.2007.05.065.


    V. Srinivasan, C. Eswaran and N. Sriraam, Approximate entropy-based epileptic EEG detection using artificial neural networks, IEEE T. Inf. Technol. B., 11 (2007), 288-295.doi: 10.1109/TITB.2006.884369.


    H. Suyari, Generalization of Shannon-Khinchin axioms to nonextensive systems and the uniqueness theorem for the nonextensive entropy, IEEE T. Inform. Theory, 50 (2004), 1783-1787.doi: 10.1109/TIT.2004.831749.


    F. Takens, Detecting strange attractors in turbulence, in Dynamical Systems and Turbulence (eds. D. A. Rand and L. S. Young), Lecture Notes in Mathematics, 898, Springer, Berlin Heidelberg, 1981, 366-381.


    F. Takens and E. Verbitskiy, Rényi entropies of aperiodic dynamical systems, Israel J. Math., 127 (2002), 279-302.doi: 10.1007/BF02784535.


    F. Takens and E. Verbitskiy, Generalized entropies: Rényi and correlation integral approach, Nonlinearity, 11 (1998), 771-782.doi: 10.1088/0951-7715/11/4/001.


    S. Tong, A. Bezerianos, A. Malhotra, Y. Zhu and N. Thakor, Parameterized entropy analysis of EEG following hypoxicischemic brain injury, Phys. Lett. A, 314 (2003), 354-361.doi: 10.1016/S0375-9601(03)00949-6.


    M. E. Torres and L. G. Gamero, Relative complexity changes in time series using information measures, Physica A, 286 (2000), 457-473.doi: 10.1016/S0378-4371(00)00309-5.


    B. Tóthmérész, Comparison of different methods for diversity ordering, J. Veg. Sci., 6 (1995), 283-290.


    C. Tsallis, Possible generalization of Boltzmann-Gibbs statistics, J. Stat. Phys., 52 (1988), 479-487.doi: 10.1007/BF01016429.


    C. Tsallis, The nonadditive entropy Sq and its applications in physics and elsewhere: Some remarks, Entropy, 13 (2011), 1765-1804.doi: 10.3390/e13101765.


    A. M. Unakafov and K. Keller, Conditional entropy of ordinal patterns, Physica D, 269 (2014), 94-102.doi: 10.1016/j.physd.2013.11.015.


    V. A. Unakafova, Investigating Measures of Complexity for Dynamical Systems and for Time Series, Ph.D thesis, University of Luebeck, 2015.


    E. A. Verbitskiy, Generalized Entropies in Dynamical Systems, Ph.D thesis, University of Groningen, 2000.


    A. Voss, S. Schulz, R. Schroeder, M. Baumert and P. Caminal, Methods derived from nonlinear dynamics for analysing heart rate variability, Phil. Trans. R. Soc. A, 367 (2009), 277-296.doi: 10.1098/rsta.2008.0232.


    P. Walters, An Introduction to Ergodic Theory, Springer Verlag, New York, 1982.


    B. Weiss, Subshifts of finite type and sofic systems, Monats. Math., 77 (1973), 462-474.doi: 10.1007/BF01295322.


    B. Weiss, Entropy and actions of sofic groups, Discr. Contin. Dyn. Syst. B, 20 (2015), 3375-3383.


    N. Wiener, The theory of prediction, in Modern Mathematics for the Engineer (ed. E. F. Beckenbach), McGraw-Hill, New York, 1956.


    A. D. Wissner-Gross and C. E. Freer, Causal entropic forces, Phys. Rev. Lett., 110 (2013), 168702.doi: 10.1103/PhysRevLett.110.168702.


    M. Zanin, L. Zunino, O. A. Rosso and D. Papo, Permutation entropy and its main biomedical and econophysics applications: A review, Entropy, 14 (2012), 1553-1577.doi: 10.3390/e14081553.


    D. Zhang, X. Jia, H. Ding, D. Ye and N. V. Thakor, Application of Tsallis entropy to EEG: Quantifying the presence of burst suppression after asphyxial cardiac arrest in rats, IEEE Trans. Bio.-Med. Eng., 57 (2010), 867-874.

  • 加载中

Article Metrics

HTML views(875) PDF downloads(714) Cited by(0)

Access History



    DownLoad:  Full-Size Img  PowerPoint