American Institute of Mathematical Sciences

Advanced
  • Previous Article
    Stabilisation by delay feedback control for highly nonlinear hybrid stochastic differential equations
  • DCDS-B Home
  • This Issue
  • Next Article
    Stabilisation by noise on the boundary for a Chafee-Infante equation with dynamical boundary conditions
August  2019, 24(8): 4079-4097. doi: 10.3934/dcdsb.2019051

Detecting coupling directions with transcript mutual information: A comparative study

José M. Amigó 1,, , Beata Graff 2, , Grzegorz Graff 3, , Roberto Monetti 4, and  Katarzyna Tessmer 5,

1. 

Centro de Investigación Operativa, Universidad Miguel Hernández, 03202 Elche, Spain
2. 

Department of Hypertension and Diabetology, Medical University of Gdańsk, 80-952 Gdańsk, Poland
3. 

Faculty of Applied Physics and Mathematics, Gdańsk University of Technology, 80-233 Gdańsk, Poland
4. 

IngSoft GmbH. Irrerstrasse 17. 90403 Nuernberg, Germany
5. 

Faculty of Applied Physics and Mathematics, Gdańsk University of Technology, 80-233 Gdańsk, Poland

* Corresponding author: jm.amigo@umh.es

Dedicated to Peter E. Kloeden on the occasion of his 70th birthday

Received  March 2018 Revised  July 2018 Published  February 2019

Fund Project: J.M.A. was financially supported by the Spanish Ministry of Economy, Industry and Competitivity, grant MTM2016-74921-P (AEI/FEDER, EU). B.G. was financially supported by the National Science Centre, Poland, grant MAESTRO UMO-2011/02/A/NZ5/00329. G.G. was financially supported by the National Science Centre, Poland, grant UMO-2014/15/B/ST1/01710.

Causal relationships are important to understand the dynamics of coupled processes and, moreover, to influence or control the effects by acting on the causes. Among the different approaches to determine cause-effect relationships and, in particular, coupling directions in interacting random or deterministic processes, we focus in this paper on information-theoretic measures. So, we study in the theoretical part the difference between directionality indicators based on transfer entropy as well as on its dimensional reduction via transcripts in algebraic time series representations. In the applications we consider specifically the lowest dimensional case, i.e., 3-dimensional transfer entropy, which is currently one of the most popular causality indicators, and the (2-dimensional) mutual information of transcripts. Needless to say, the lower dimensionality of the transcript-based indicator can make a difference in practice, where datasets are usually small. To compare numerically the performance of both directionality indicators, synthetic data (obtained with random processes) and real world data (in the form of biomedical recordings) are used. As happened in previous related work, we found again that the transcript mutual information performs as good as, and in some cases even better than, the lowest dimensional binned and symbolic transfer entropy, the symbols being ordinal patterns.

Keywords: Information directionality, transfer entropy, time series analysis, algebraic symbolic representations, transcripts, dimensional reduction.
Mathematics Subject Classification: Primary: 37M10, 94A17; Secondary: 60G10.
Citation: José M. Amigó, Beata Graff, Grzegorz Graff, Roberto Monetti, Katarzyna Tessmer. Detecting coupling directions with transcript mutual information: A comparative study. Discrete & Continuous Dynamical Systems - B, 2019, 24 (8) : 4079-4097. doi: 10.3934/dcdsb.2019051
References:
[1]

J. M. Amigó and M. B. Kennel, Forbidden ordinal patterns in higher dimensional dynamics, Physica D, 237 (2008), 2893-2899.  doi: 10.1016/j.physd.2008.05.003.  Google Scholar

[2]

J. M. Amigó, S. Zambrano and M. A. F. Sanjuán., Detecting determinism with ordinal patterns: A comparative study, Int. J. Bifurcation and Chaos, 20 (2010), 2915-2924.  doi: 10.1142/S0218127410027453.  Google Scholar

[3]

J. M. Amigó, R. Monetti, T. Aschenbrenner and W. Bunk, Transcripts: An algebraic approach to coupled time series, Chaos, 22 (2012), 013105, 13pp. doi: 10.1063/1.3673238.  Google Scholar

[4]

J. M. Amigó, T. Aschenbrenner, W. Bunk and R. Monetti, Dimensional reduction of conditional algebraic multi-information via transcripts, Inform. Sci., 278 (2014), 298-310.  doi: 10.1016/j.ins.2014.03.054.  Google Scholar

[5]

J. M. Amigó, K. Keller and V. Unakafova, Ordinal symbolic analysis and its application to biomedical recordings, Phil. Trans. R. Soc. A, 373 (2015), 20140091, 18pp. doi: 10.1098/rsta.2014.0091.  Google Scholar

[6]

J. M. Amigó, R. Monetti, N. Tort-Colet and M. V. Sanchez-Vives, Infragranular layers lead information flow during slow oscillations according to information directionality indicators, J. Comput. Neurosci, 39 (2015), 53-62.   Google Scholar

[7]

J. M. Amigó, R. Monetti, B. Graff and G. Graff, Computing algebraic transfer entropy and coupling directions via transcripts, Chaos, 26 (2016), 113115, 12pp. doi: 10.1063/1.4967803.  Google Scholar

[8]

C. Bandt and B. Pompe, Permutation entropy: A natural complexity measure for time series, Phys. Rev. Lett., 88 (2002), 174102. Google Scholar

[9]

C. Cafaro, W. M. Lord, J. Sun and E. M. Bollt, Causation entropy from symbolic representations of dynamical systems, Chaos, 25 (2015), 043106, 10pp. doi: 10.1063/1.4916902.  Google Scholar

[10]

T. M. Cover and J. A. Thomas, Elements of Information Theory, (second ed.), John Wiley & Sons, Hoboken, 2006.  Google Scholar

[11]

European-Heart-Network, Cardiovascular Disease Statistics, 2017. Google Scholar

[12]

G. Graff, B. Graff, A. Kaczkowska, D. Makowiec, J. M. Amigó, J. Piskorski, K. Narkiewicz and P. Guzik, Ordinal pattern statistics for the assessment of heart rate variability, Eur. Phys. J. Special Topics, 222 (2013), 525-534.  doi: 10.1140/epjst/e2013-01857-4.  Google Scholar

[13]

B. Graff, G. Graff, D. Makowiec, A. Kaczkowska, D. Wejer, S. Budrejko, D. Kozƚ owski and K. Narkiewicz, Entropy measures in the assessment of heart rate variability in patients with cardiodepressive vasovagal syncope, Entropy, 17 (2015), 1007-1022.  doi: 10.3390/e17031007.  Google Scholar

[14]

C. W. J. Granger, Investigating causal relations by econometric models and cross-spectral methods, Econometrica, 37 (1969), 424-438.   Google Scholar

[15]

Y. Hirata, J. M. Amigó, Y. Matsuzaka, R. Yokota, H. Mushiake and K. Aihara, Detecting causality by combined use of multiple methods: Climate and brain examples, PLos One, 11 (2016), e0158572. doi: 10.1371/journal.pone.0158572.  Google Scholar

[16]

R. Monetti, W. Bunk, T. Aschenbrenner and F. Jamitzky, Characterizing synchronization in time series using information measures extracted from symbolic representations, Phys. Rev. E, 79 (2009), 046207. doi: 10.1103/PhysRevE.79.046207.  Google Scholar

[17]

R. Monetti, J. M. Amigó, T. Aschenbrenner and W. Bunk, Permutation complexity of interacting dynamical systems, Eur. Phys. J. Special Topics, 222 (2013), 421-436.  doi: 10.1140/epjst/e2013-01850-y.  Google Scholar

[18]

R. Monetti, W. Bunk, T. Aschenbrenner, S. Springer and J. M. Amigó, Information directionality in coupled time series using transcripts, Phys. Rev. E, 88 (2013), 022911. doi: 10.1103/PhysRevE.88.022911.  Google Scholar

[19]

U. Parlitz, S. Berg, S. Luther, A. Schirdewan, J. Kurths and N. Wessel, Classifying cardiac biosignals using ordinal pattern statistics and symbolic dynamics, Comput. Biol. Med., 42 (2012), 319-327.  doi: 10.1016/j.compbiomed.2011.03.017.  Google Scholar

[20]

A. Porta, A. Catai, A. Takahashi, V. Magagnin, T. Bassani, E. Tobaldini, P. van de Borne and N. Montano, Causal relationships between heart period and systolic arterial pressure during graded head-up tilt, Am. J. Physiol. Regul. Integr. Comp. Physiol, 300 (2011), R378–R386. doi: 10.1152/ajpregu.00553.2010.  Google Scholar

[21]

T. Schreiber, Measuring information transfer, Phys. Rev. Lett., 85 (2000), 461-464.  doi: 10.1103/PhysRevLett.85.461.  Google Scholar

[22]

D. Smirnov, Spurious causalities with transfer entropy, Phys. Rev. E, 87 (2013), 042917. doi: 10.1103/PhysRevE.87.042917.  Google Scholar

[23]

M. Staniek and K. Lehnertz, Symbolic transfer entropy, Phys. Rev. Lett. 100 (2008), 158101. doi: 10.1103/PhysRevLett.100.158101.  Google Scholar

[24]

N. Wiener, Modern Mathematics for Engineers, McGraw-Hill, New York, 1956.  Google Scholar

show all references

References:
[1]

J. M. Amigó and M. B. Kennel, Forbidden ordinal patterns in higher dimensional dynamics, Physica D, 237 (2008), 2893-2899.  doi: 10.1016/j.physd.2008.05.003.  Google Scholar

[2]

J. M. Amigó, S. Zambrano and M. A. F. Sanjuán., Detecting determinism with ordinal patterns: A comparative study, Int. J. Bifurcation and Chaos, 20 (2010), 2915-2924.  doi: 10.1142/S0218127410027453.  Google Scholar

[3]

J. M. Amigó, R. Monetti, T. Aschenbrenner and W. Bunk, Transcripts: An algebraic approach to coupled time series, Chaos, 22 (2012), 013105, 13pp. doi: 10.1063/1.3673238.  Google Scholar

[4]

J. M. Amigó, T. Aschenbrenner, W. Bunk and R. Monetti, Dimensional reduction of conditional algebraic multi-information via transcripts, Inform. Sci., 278 (2014), 298-310.  doi: 10.1016/j.ins.2014.03.054.  Google Scholar

[5]

J. M. Amigó, K. Keller and V. Unakafova, Ordinal symbolic analysis and its application to biomedical recordings, Phil. Trans. R. Soc. A, 373 (2015), 20140091, 18pp. doi: 10.1098/rsta.2014.0091.  Google Scholar

[6]

J. M. Amigó, R. Monetti, N. Tort-Colet and M. V. Sanchez-Vives, Infragranular layers lead information flow during slow oscillations according to information directionality indicators, J. Comput. Neurosci, 39 (2015), 53-62.   Google Scholar

[7]

J. M. Amigó, R. Monetti, B. Graff and G. Graff, Computing algebraic transfer entropy and coupling directions via transcripts, Chaos, 26 (2016), 113115, 12pp. doi: 10.1063/1.4967803.  Google Scholar

[8]

C. Bandt and B. Pompe, Permutation entropy: A natural complexity measure for time series, Phys. Rev. Lett., 88 (2002), 174102. Google Scholar

[9]

C. Cafaro, W. M. Lord, J. Sun and E. M. Bollt, Causation entropy from symbolic representations of dynamical systems, Chaos, 25 (2015), 043106, 10pp. doi: 10.1063/1.4916902.  Google Scholar

[10]

T. M. Cover and J. A. Thomas, Elements of Information Theory, (second ed.), John Wiley & Sons, Hoboken, 2006.  Google Scholar

[11]

European-Heart-Network, Cardiovascular Disease Statistics, 2017. Google Scholar

[12]

G. Graff, B. Graff, A. Kaczkowska, D. Makowiec, J. M. Amigó, J. Piskorski, K. Narkiewicz and P. Guzik, Ordinal pattern statistics for the assessment of heart rate variability, Eur. Phys. J. Special Topics, 222 (2013), 525-534.  doi: 10.1140/epjst/e2013-01857-4.  Google Scholar

[13]

B. Graff, G. Graff, D. Makowiec, A. Kaczkowska, D. Wejer, S. Budrejko, D. Kozƚ owski and K. Narkiewicz, Entropy measures in the assessment of heart rate variability in patients with cardiodepressive vasovagal syncope, Entropy, 17 (2015), 1007-1022.  doi: 10.3390/e17031007.  Google Scholar

[14]

C. W. J. Granger, Investigating causal relations by econometric models and cross-spectral methods, Econometrica, 37 (1969), 424-438.   Google Scholar

[15]

Y. Hirata, J. M. Amigó, Y. Matsuzaka, R. Yokota, H. Mushiake and K. Aihara, Detecting causality by combined use of multiple methods: Climate and brain examples, PLos One, 11 (2016), e0158572. doi: 10.1371/journal.pone.0158572.  Google Scholar

[16]

R. Monetti, W. Bunk, T. Aschenbrenner and F. Jamitzky, Characterizing synchronization in time series using information measures extracted from symbolic representations, Phys. Rev. E, 79 (2009), 046207. doi: 10.1103/PhysRevE.79.046207.  Google Scholar

[17]

R. Monetti, J. M. Amigó, T. Aschenbrenner and W. Bunk, Permutation complexity of interacting dynamical systems, Eur. Phys. J. Special Topics, 222 (2013), 421-436.  doi: 10.1140/epjst/e2013-01850-y.  Google Scholar

[18]

R. Monetti, W. Bunk, T. Aschenbrenner, S. Springer and J. M. Amigó, Information directionality in coupled time series using transcripts, Phys. Rev. E, 88 (2013), 022911. doi: 10.1103/PhysRevE.88.022911.  Google Scholar

[19]

U. Parlitz, S. Berg, S. Luther, A. Schirdewan, J. Kurths and N. Wessel, Classifying cardiac biosignals using ordinal pattern statistics and symbolic dynamics, Comput. Biol. Med., 42 (2012), 319-327.  doi: 10.1016/j.compbiomed.2011.03.017.  Google Scholar

[20]

A. Porta, A. Catai, A. Takahashi, V. Magagnin, T. Bassani, E. Tobaldini, P. van de Borne and N. Montano, Causal relationships between heart period and systolic arterial pressure during graded head-up tilt, Am. J. Physiol. Regul. Integr. Comp. Physiol, 300 (2011), R378–R386. doi: 10.1152/ajpregu.00553.2010.  Google Scholar

[21]

T. Schreiber, Measuring information transfer, Phys. Rev. Lett., 85 (2000), 461-464.  doi: 10.1103/PhysRevLett.85.461.  Google Scholar

[22]

D. Smirnov, Spurious causalities with transfer entropy, Phys. Rev. E, 87 (2013), 042917. doi: 10.1103/PhysRevE.87.042917.  Google Scholar

[23]

M. Staniek and K. Lehnertz, Symbolic transfer entropy, Phys. Rev. Lett. 100 (2008), 158101. doi: 10.1103/PhysRevLett.100.158101.  Google Scholar

[24]

N. Wiener, Modern Mathematics for Engineers, McGraw-Hill, New York, 1956.  Google Scholar

Figure 1.  Plots of $ \Delta AT_{\mathbf{\hat{Z}}\rightarrow \mathbf{\hat{Y}}}(\Lambda ) $ (solid line) and $ \Delta TI_{\mathbf{\hat{Z}}\rightarrow \mathbf{\hat{Y}}}(\Lambda ) $ (dash-dotted line) vs $ \Lambda $, $ 1\leq \Lambda \leq T-1 $, for $ T = 3,4,...,10 $
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 2.  Plots of $ \Delta AT_{\mathbf{\hat{Y}}\rightarrow \mathbf{\hat{X}}}(\Lambda ) $ (solid line) and $ \Delta TI_{\mathbf{\hat{Y}}\rightarrow \mathbf{\hat{X}}}(\Lambda ) $ (dash-dotted line) vs $ \Lambda $, $ 1\leq \Lambda \leq T-1 $, for $ T = 3,4,...,10 $
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 3.  Plots of $ \Delta AT_{\mathbf{\hat{Z}}\rightarrow \mathbf{\hat{X}}}(\Lambda ) $ (solid line) and $ \Delta TI_{\mathbf{\hat{Z}}\rightarrow \mathbf{\hat{X}}}(\Lambda ) $ (dash-dotted line) vs $ \Lambda $, $ 1\leq \Lambda \leq T-1 $, for $ T = 3,4,...,10 $
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 4.  Plots of $ \Delta AT_{\mathbf{\hat{D}}\rightarrow \mathbf{\hat{R}}}(\Lambda ) $ (continuous lines) and $ \Delta TI_{\mathbf{\hat{D}}\rightarrow \mathbf{\hat{R}}}(\Lambda ) $ (dash-dotted lines) for $ T = 8 $, $ 1\leq \Lambda \leq 7 $, and $ k_{zy} = 0.0 $ (first column), $ k_{zy} = 0.1 $ (second column), $ k_{zy} = 0.5 $ (third column), and $ k_{zy} = 0.9 $ (fourth column). Top row corresponds to the coupling direction $ \mathbf{\hat{Z}}\rightarrow \mathbf{\hat{Y}} $, middle row to $ \mathbf{\hat{Z}}\rightarrow \mathbf{\hat{X}} $, and bottom row to $ \mathbf{\hat{Y}}\rightarrow \mathbf{\hat{X}} $
Figure Options

Download full-size image

Download as PowerPoint slide

Figure 5.  The directionality indicators $ \Delta TE_{RR\rightarrow BP}(\Lambda ) $ (top row), $ \Delta STE_{RR\rightarrow BP}(\Lambda ) $ (middle row), and $ \Delta TMI_{RR\rightarrow BP}(\Lambda ) $ (bottom row) for the patient Groups Ⅰ (left column), Ⅱ (middle column), and ⅡB (right column) with $ T = 6 $ and $ 1\leq \Lambda \leq 5 $. For convenience, here $ TE $ stands for binned transfer entropy, $ STE $ for symbolic transfer entropy, and $ TMI $ for transcript mutual information. See the text for more details, the description of the patient Groups, and the data $ (RR_{n}) $ and $ (BP_{n}) $
Figure Options

Download full-size image

Download as PowerPoint slide

[1]

Wen-Guei Hu, Song-Sun Lin. On spatial entropy of multi-dimensional symbolic dynamical systems. Discrete & Continuous Dynamical Systems - A, 2016, 36 (7) : 3705-3717. doi: 10.3934/dcds.2016.36.3705
[2]

Y. T. Li, R. Wong. Integral and series representations of the dirac delta function. Communications on Pure & Applied Analysis, 2008, 7 (2) : 229-247. doi: 10.3934/cpaa.2008.7.229
[3]

Laura Luzzi, Ghaya Rekaya-Ben Othman, Jean-Claude Belfiore. Algebraic reduction for the Golden Code. Advances in Mathematics of Communications, 2012, 6 (1) : 1-26. doi: 10.3934/amc.2012.6.1
[4]

Lubomir Kostal, Shigeru Shinomoto. Efficient information transfer by Poisson neurons. Mathematical Biosciences & Engineering, 2016, 13 (3) : 509-520. doi: 10.3934/mbe.2016004
[5]

Frédéric Naud. Birkhoff cones, symbolic dynamics and spectrum of transfer operators. Discrete & Continuous Dynamical Systems - A, 2004, 11 (2&3) : 581-598. doi: 10.3934/dcds.2004.11.581
[6]

Geir Bogfjellmo. Algebraic structure of aromatic B-series. Journal of Computational Dynamics, 2019, 6 (2) : 199-222. doi: 10.3934/jcd.2019010
[7]

Asim Aziz, Wasim Jamshed, Yasir Ali, Moniba Shams. Heat transfer and entropy analysis of Maxwell hybrid nanofluid including effects of inclined magnetic field, Joule heating and thermal radiation. Discrete & Continuous Dynamical Systems - S, 2020, 13 (10) : 2667-2690. doi: 10.3934/dcdss.2020142
[8]

Annalisa Pascarella, Alberto Sorrentino, Cristina Campi, Michele Piana. Particle filtering, beamforming and multiple signal classification for the analysis of magnetoencephalography time series: a comparison of algorithms. Inverse Problems & Imaging, 2010, 4 (1) : 169-190. doi: 10.3934/ipi.2010.4.169
[9]

Fryderyk Falniowski, Marcin Kulczycki, Dominik Kwietniak, Jian Li. Two results on entropy, chaos and independence in symbolic dynamics. Discrete & Continuous Dynamical Systems - B, 2015, 20 (10) : 3487-3505. doi: 10.3934/dcdsb.2015.20.3487
[10]

Jean-François Babadjian, Francesca Prinari, Elvira Zappale. Dimensional reduction for supremal functionals. Discrete & Continuous Dynamical Systems - A, 2012, 32 (5) : 1503-1535. doi: 10.3934/dcds.2012.32.1503
[11]

Martin Frank, Cory D. Hauck, Edgar Olbrant. Perturbed, entropy-based closure for radiative transfer. Kinetic & Related Models, 2013, 6 (3) : 557-587. doi: 10.3934/krm.2013.6.557
[12]

Editorial Office. Retraction: Xiao-Qian Jiang and Lun-Chuan Zhang, Stock price fluctuation prediction method based on time series analysis. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 915-915. doi: 10.3934/dcdss.2019061
[13]

Peter Ashwin, Xin-Chu Fu. Symbolic analysis for some planar piecewise linear maps. Discrete & Continuous Dynamical Systems - A, 2003, 9 (6) : 1533-1548. doi: 10.3934/dcds.2003.9.1533
[14]

David Burguet. Examples of $\mathcal{C}^r$ interval map with large symbolic extension entropy. Discrete & Continuous Dynamical Systems - A, 2010, 26 (3) : 873-899. doi: 10.3934/dcds.2010.26.873
[15]

Mike Boyle, Tomasz Downarowicz. Symbolic extension entropy: $c^r$ examples, products and flows. Discrete & Continuous Dynamical Systems - A, 2006, 16 (2) : 329-341. doi: 10.3934/dcds.2006.16.329
[16]

Chuang Peng. Minimum degrees of polynomial models on time series. Conference Publications, 2005, 2005 (Special) : 720-729. doi: 10.3934/proc.2005.2005.720
[17]

Ruiqi Li, Yifan Chen, Xiang Zhao, Yanli Hu, Weidong Xiao. Time series based urban air quality predication. Big Data & Information Analytics, 2016, 1 (2&3) : 171-183. doi: 10.3934/bdia.2016003
[18]

Mark F. Demers, Hong-Kun Zhang. Spectral analysis of the transfer operator for the Lorentz gas. Journal of Modern Dynamics, 2011, 5 (4) : 665-709. doi: 10.3934/jmd.2011.5.665
[19]

Anke D. Pohl. Symbolic dynamics for the geodesic flow on two-dimensional hyperbolic good orbifolds. Discrete & Continuous Dynamical Systems - A, 2014, 34 (5) : 2173-2241. doi: 10.3934/dcds.2014.34.2173
[20]

Lorenzo Sella, Pieter Collins. Computation of symbolic dynamics for two-dimensional piecewise-affine maps. Discrete & Continuous Dynamical Systems - B, 2011, 15 (3) : 739-767. doi: 10.3934/dcdsb.2011.15.739

2019 Impact Factor: 1.27

Tools

Metrics

  • PDF downloads (92)
  • HTML views (397)
  • Cited by (0)

Other articles
by authors

[Back to Top]

Copyright © 2020 American Institute of Mathematical Sciences