2016, 21(7): 2337-2361. doi: 10.3934/dcdsb.2016050

Stochastic dynamics: Markov chains and random transformations

1. 

Department of Applied Mathematics, University of Washington, Seattle, WA 98195-3925, United States, United States, United States

Received  June 2015 Revised  February 2016 Published  August 2016

This article outlines an attempt to lay the groundwork for understanding stochastic dynamical descriptions of biological processes in terms of a discrete-state space, discrete-time random dynamical system (RDS), or random transformation approach. Such mathematics is not new for continuous systems, but the discrete state space formulation significantly reduces the technical requirements for its introduction to a much broader audiences. In particular, we establish some elementary contradistinctions between Markov chain (MC) and RDS descriptions of a stochastic dynamics. It is shown that a given MC is compatible with many possible RDS, and we study in particular the corresponding RDS with maximum metric entropy. Specifically, we show an emergent behavior of an MC with a unique absorbing and aperiodic communicating class, after all the trajectories of the RDS synchronizes. In biological modeling, it is now widely acknowledged that stochastic dynamics is a more complete description of biological reality than deterministic equations; here we further suggest that the RDS description could be a more refined description of stochastic dynamics than a Markov process. Possible applications of discrete-state RDS are systems with fluctuating law of motion, or environment, rather than inherent stochastic movements of individuals.
Citation: Felix X.-F. Ye, Yue Wang, Hong Qian. Stochastic dynamics: Markov chains and random transformations. Discrete & Continuous Dynamical Systems - B, 2016, 21 (7) : 2337-2361. doi: 10.3934/dcdsb.2016050
References:
[1]

L. J. S. Allen, An Introduction to Stochastic Processes with Applications to Biology,, $2^{nd}$ edition, (2011).

[2]

L. Arnold, Random Dynamical Systems,, Springer, (1998). doi: 10.1007/978-3-662-12878-7.

[3]

L. Arnold and H. Crauel, Random dynamical systems,, in Lyapunov Exponents, 1486 (2006), 1. doi: 10.1007/BFb0086654.

[4]

P. H. Baxendale, Statistical equilibrium and two-point motion for a stochastic flow of diffeomorphisms. Spatial stochastic processes,, Progr. Probab., 19 (1991), 189.

[5]

R. Bhattacharya and M. Majumdar, Random Dynamical Systems: Theory and Applications,, Cambridge Univ. Press, (2007). doi: 10.1017/CBO9780511618628.

[6]

G. Birkhoff, Three observations on linear algebra,, Univ. Nac. Tucumán. Revista A, 5 (1946), 147.

[7]

R. M Blumenthal and H. K. Corson, On continuous collections of measures,, Proc. 6th Berkeley Symp. on Math. Stat. and Prob., 2 (1972), 33.

[8]

K.-S. Chan and H. Tong, Chaos: A Statistical Perspective,, Springer, (2001). doi: 10.1007/978-1-4757-3464-5.

[9]

Y.-D. Chen, Asymmetry and external noise-induced free energy transduction,, Proc. Natl. Acad. Sci. U.S.A., 84 (1987), 729. doi: 10.1073/pnas.84.3.729.

[10]

T. Downarowicz, Entropy in Dynamical Systems,, Cambridge Univ. Press, (2011). doi: 10.1017/CBO9780511976155.

[11]

S. P. Ellner and J. Guckenheimer, Dynamic Models in Biology,, Princeton Univ. Press, (2006).

[12]

G. Froyland, Extracting dynamical behavior via Markov models,, Nonlinear dynamics and statistics (Cambridge, (2001), 281.

[13]

G. Gallavotti, Statistical Mechanics: A Short Treatise,, Springer, (1999). doi: 10.1007/978-3-662-03952-6.

[14]

H. Ge, M. Qian and H. Qian, Stochastic theory of nonequilibrium steady states (Part II): Applications in chemical biophysics,, Phys. Rep., 510 (2012), 87. doi: 10.1016/j.physrep.2011.09.001.

[15]

B. Hasselblatt and A. Katok, Principal structures,, in Handbook of Dynamical Systems, 1 (2002), 1. doi: 10.1016/S1874-575X(02)80003-0.

[16]

W. Horsthemke and R. Lefever, Noise-Induced Transitions: Theory and Applications in Physics, Chemistry, and Biology,, Springer, (1984).

[17]

E. Isaacson and H. B. Keller, Analysis of Numerical Methods,, Dover, (1966).

[18]

S. L. Kalpazidou, Cycle Representations of Markov Processes,, $2^{nd}$ edition, (2006).

[19]

M. Keane, Ergodic theory and subshifts of finite type,, in Ergodic theory, (1991), 35.

[20]

A. I. Khinchin, Mathematical Foundations of Information Theory,, Dover, (1957).

[21]

Yu. Kifer, Ergodic Theory of Random Transformations,, Birkhäuser, (1986). doi: 10.1007/978-1-4684-9175-3.

[22]

Yu. Kifer and P.-D. Liu, Random dynamics,, in Handbook of Dynamical Systems, 1 (2006), 379. doi: 10.1016/S1874-575X(06)80030-5.

[23]

A. Lasota and M. C. Mackey, Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics,, $2^{nd}$ edition, (1994). doi: 10.1007/978-1-4612-4286-4.

[24]

V. Lecomte, C. Appert-Rolland and F. van Wijland, Thermodynamic formalism for systems with Markov dynamics,, J. Stat. Phys., 127 (2007), 51. doi: 10.1007/s10955-006-9254-0.

[25]

T. Liggett, The coupling technique in interacting particle systems,, In Doeblin and Modern Probability, 149 (1993), 73. doi: 10.1090/conm/149/01271.

[26]

K. K. Lin, Stimulus-response reliability of biological networks,, in Nonautonomous and Random Dynamical Systems in Life Sciences (eds. P. Kloeden and C. Poetzsche), 2102 (2012), 135. doi: 10.1007/978-3-319-03080-7_4.

[27]

D. Lind and B. Marcus, An Introduction to Symbolic Dynamics and Coding,, Cambridge Univ. Press, (1995). doi: 10.1017/CBO9780511626302.

[28]

X. Mao and C. Yuan, Stochastic Differential Equations with Markovian Switching,, Imperial College Press, (2006). doi: 10.1142/p473.

[29]

M. Marcus, H. Minc and B. Moyls, Some results on non-negative matrices,, J. Res. Nat. Bur. Standards Sec. B, 65 (1961), 205. doi: 10.6028/jres.065B.019.

[30]

M. L. Mehta, Random Matrices,, $3^{rd}$ edition, (2004).

[31]

H. Minc, Nonnegative Matrices,, John Wiley & Sons, (1988).

[32]

J. A. Morrison and J. McKenna, Analysis of some stochastic ordinary differential equations,, in Stochastic Differential Equations, (1973), 97.

[33]

J. D. Murray, Mathematical Biology: I. An Introduction,, $3^{rd}$ edition, (2002).

[34]

D. S. Ornstein, Ergodic theory, randomness, and "chaos'',, Science, 243 (1989), 182. doi: 10.1126/science.243.4888.182.

[35]

Y. Peres, Analytic dependence of Lyapunov exponents on transition probabilities,, in Lyapunov Exponents, 1486 (1991), 64. doi: 10.1007/BFb0086658.

[36]

M. A. Pinsky, Lectures on Random Evolution,, World Scientific, (1991). doi: 10.1142/1328.

[37]

H. Qian and J. A. Schellman, Helix-coil theories: A comparative studies for finite length polypeptides,, J. Phys. Chem., 96 (1992), 3987. doi: 10.1021/j100189a015.

[38]

H. Qian, The mathematical theory of molecular motor movement and chemomechanical energy transduction,, J. Math. Chem., 27 (2000), 219. doi: 10.1023/A:1026428320489.

[39]

H. Qian, Cooperativity in cellular biochemical processes: Noise-enhanced sensitivity, fluctuating enzyme, bistability with nonlinear feedback, and other mechanisms for sigmoidal responses,, Annu. Rev. Biophys., 41 (2012), 179. doi: 10.1146/annurev-biophys-050511-102240.

[40]

H. Qian, S. Kjelstrup, A. B. Kolomeisky and D. Bedeaux, Entropy production in mesoscopic stochastic thermodynamics | Nonequilibrium steady state cycles driven by chemical potentials, temperatures, and mechanical forces,, J. Phys. Cond. Matt. 28 (2016), 28 (2016).

[41]

M. Qian, J.-S. Xie and S. Zhu, Smooth Ergodic Theory for Endomorphisms,, Lec. Notes Math. vol. 1978, (1978). doi: 10.1007/978-3-642-01954-8.

[42]

M. Qian and F.-X. Zhang, Entropy production rate of the coupled diffusion process,, J. Theor. Probab., 24 (2011), 729. doi: 10.1007/s10959-011-0352-9.

[43]

R. T. Rockafellar, Convex Analysis,, Princeton Univ. Press, (1970).

[44]

M. Santillán and H. Qian, Irreversible thermodynamics in multiscale stochastic dynamical systems,, Phys. Rev. E, 83 (2011).

[45]

S. H. Strogatz, Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering,, Westview Press, (2001).

[46]

A. Swishchuk and S. Islam, Random Dynamical Systems in Finance,, Chapman & Hall/CRC, (2013). doi: 10.1201/b14989.

[47]

H. Thorisson, Coupling and shift-coupling random sequences,, Doeblin and Modern Probability, 149 (1993), 85. doi: 10.1090/conm/149/01280.

[48]

J. M. van Campenhout and T. M. Cover, Maximum entropy and conditional probability,, IEEE Infor. Th., IT-27 (1981), 483. doi: 10.1109/TIT.1981.1056374.

[49]

J. van Neumann, The general and logical theory of automata,, Cerebral Mechanisms in Behavior, (1951), 1.

[50]

P. Walters, An Introduction to Ergodic Theory,, Spinger, (1982).

[51]

S. Wolfram, A New Kind of Science,, Wolfram media, (2002).

[52]

G. G. Yin and C. Zhu, Hybrid Switching Diffusions: Properties and Applications,, Springer, (2010). doi: 10.1007/978-1-4419-1105-6.

[53]

X.-J. Zhang, H. Qian and M. Qian, Stochastic theory of nonequilibrium steady states and its applications (Part I),, Phys. Rep., 510 (2012), 1.

[54]

X.-J. Zhang, M. Qian and H. Qian, Stochastic dynamics of electrical membrane with voltage-dependent ion channel fluctuations,, Europhys. Lett., 106 (2014).

show all references

References:
[1]

L. J. S. Allen, An Introduction to Stochastic Processes with Applications to Biology,, $2^{nd}$ edition, (2011).

[2]

L. Arnold, Random Dynamical Systems,, Springer, (1998). doi: 10.1007/978-3-662-12878-7.

[3]

L. Arnold and H. Crauel, Random dynamical systems,, in Lyapunov Exponents, 1486 (2006), 1. doi: 10.1007/BFb0086654.

[4]

P. H. Baxendale, Statistical equilibrium and two-point motion for a stochastic flow of diffeomorphisms. Spatial stochastic processes,, Progr. Probab., 19 (1991), 189.

[5]

R. Bhattacharya and M. Majumdar, Random Dynamical Systems: Theory and Applications,, Cambridge Univ. Press, (2007). doi: 10.1017/CBO9780511618628.

[6]

G. Birkhoff, Three observations on linear algebra,, Univ. Nac. Tucumán. Revista A, 5 (1946), 147.

[7]

R. M Blumenthal and H. K. Corson, On continuous collections of measures,, Proc. 6th Berkeley Symp. on Math. Stat. and Prob., 2 (1972), 33.

[8]

K.-S. Chan and H. Tong, Chaos: A Statistical Perspective,, Springer, (2001). doi: 10.1007/978-1-4757-3464-5.

[9]

Y.-D. Chen, Asymmetry and external noise-induced free energy transduction,, Proc. Natl. Acad. Sci. U.S.A., 84 (1987), 729. doi: 10.1073/pnas.84.3.729.

[10]

T. Downarowicz, Entropy in Dynamical Systems,, Cambridge Univ. Press, (2011). doi: 10.1017/CBO9780511976155.

[11]

S. P. Ellner and J. Guckenheimer, Dynamic Models in Biology,, Princeton Univ. Press, (2006).

[12]

G. Froyland, Extracting dynamical behavior via Markov models,, Nonlinear dynamics and statistics (Cambridge, (2001), 281.

[13]

G. Gallavotti, Statistical Mechanics: A Short Treatise,, Springer, (1999). doi: 10.1007/978-3-662-03952-6.

[14]

H. Ge, M. Qian and H. Qian, Stochastic theory of nonequilibrium steady states (Part II): Applications in chemical biophysics,, Phys. Rep., 510 (2012), 87. doi: 10.1016/j.physrep.2011.09.001.

[15]

B. Hasselblatt and A. Katok, Principal structures,, in Handbook of Dynamical Systems, 1 (2002), 1. doi: 10.1016/S1874-575X(02)80003-0.

[16]

W. Horsthemke and R. Lefever, Noise-Induced Transitions: Theory and Applications in Physics, Chemistry, and Biology,, Springer, (1984).

[17]

E. Isaacson and H. B. Keller, Analysis of Numerical Methods,, Dover, (1966).

[18]

S. L. Kalpazidou, Cycle Representations of Markov Processes,, $2^{nd}$ edition, (2006).

[19]

M. Keane, Ergodic theory and subshifts of finite type,, in Ergodic theory, (1991), 35.

[20]

A. I. Khinchin, Mathematical Foundations of Information Theory,, Dover, (1957).

[21]

Yu. Kifer, Ergodic Theory of Random Transformations,, Birkhäuser, (1986). doi: 10.1007/978-1-4684-9175-3.

[22]

Yu. Kifer and P.-D. Liu, Random dynamics,, in Handbook of Dynamical Systems, 1 (2006), 379. doi: 10.1016/S1874-575X(06)80030-5.

[23]

A. Lasota and M. C. Mackey, Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics,, $2^{nd}$ edition, (1994). doi: 10.1007/978-1-4612-4286-4.

[24]

V. Lecomte, C. Appert-Rolland and F. van Wijland, Thermodynamic formalism for systems with Markov dynamics,, J. Stat. Phys., 127 (2007), 51. doi: 10.1007/s10955-006-9254-0.

[25]

T. Liggett, The coupling technique in interacting particle systems,, In Doeblin and Modern Probability, 149 (1993), 73. doi: 10.1090/conm/149/01271.

[26]

K. K. Lin, Stimulus-response reliability of biological networks,, in Nonautonomous and Random Dynamical Systems in Life Sciences (eds. P. Kloeden and C. Poetzsche), 2102 (2012), 135. doi: 10.1007/978-3-319-03080-7_4.

[27]

D. Lind and B. Marcus, An Introduction to Symbolic Dynamics and Coding,, Cambridge Univ. Press, (1995). doi: 10.1017/CBO9780511626302.

[28]

X. Mao and C. Yuan, Stochastic Differential Equations with Markovian Switching,, Imperial College Press, (2006). doi: 10.1142/p473.

[29]

M. Marcus, H. Minc and B. Moyls, Some results on non-negative matrices,, J. Res. Nat. Bur. Standards Sec. B, 65 (1961), 205. doi: 10.6028/jres.065B.019.

[30]

M. L. Mehta, Random Matrices,, $3^{rd}$ edition, (2004).

[31]

H. Minc, Nonnegative Matrices,, John Wiley & Sons, (1988).

[32]

J. A. Morrison and J. McKenna, Analysis of some stochastic ordinary differential equations,, in Stochastic Differential Equations, (1973), 97.

[33]

J. D. Murray, Mathematical Biology: I. An Introduction,, $3^{rd}$ edition, (2002).

[34]

D. S. Ornstein, Ergodic theory, randomness, and "chaos'',, Science, 243 (1989), 182. doi: 10.1126/science.243.4888.182.

[35]

Y. Peres, Analytic dependence of Lyapunov exponents on transition probabilities,, in Lyapunov Exponents, 1486 (1991), 64. doi: 10.1007/BFb0086658.

[36]

M. A. Pinsky, Lectures on Random Evolution,, World Scientific, (1991). doi: 10.1142/1328.

[37]

H. Qian and J. A. Schellman, Helix-coil theories: A comparative studies for finite length polypeptides,, J. Phys. Chem., 96 (1992), 3987. doi: 10.1021/j100189a015.

[38]

H. Qian, The mathematical theory of molecular motor movement and chemomechanical energy transduction,, J. Math. Chem., 27 (2000), 219. doi: 10.1023/A:1026428320489.

[39]

H. Qian, Cooperativity in cellular biochemical processes: Noise-enhanced sensitivity, fluctuating enzyme, bistability with nonlinear feedback, and other mechanisms for sigmoidal responses,, Annu. Rev. Biophys., 41 (2012), 179. doi: 10.1146/annurev-biophys-050511-102240.

[40]

H. Qian, S. Kjelstrup, A. B. Kolomeisky and D. Bedeaux, Entropy production in mesoscopic stochastic thermodynamics | Nonequilibrium steady state cycles driven by chemical potentials, temperatures, and mechanical forces,, J. Phys. Cond. Matt. 28 (2016), 28 (2016).

[41]

M. Qian, J.-S. Xie and S. Zhu, Smooth Ergodic Theory for Endomorphisms,, Lec. Notes Math. vol. 1978, (1978). doi: 10.1007/978-3-642-01954-8.

[42]

M. Qian and F.-X. Zhang, Entropy production rate of the coupled diffusion process,, J. Theor. Probab., 24 (2011), 729. doi: 10.1007/s10959-011-0352-9.

[43]

R. T. Rockafellar, Convex Analysis,, Princeton Univ. Press, (1970).

[44]

M. Santillán and H. Qian, Irreversible thermodynamics in multiscale stochastic dynamical systems,, Phys. Rev. E, 83 (2011).

[45]

S. H. Strogatz, Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering,, Westview Press, (2001).

[46]

A. Swishchuk and S. Islam, Random Dynamical Systems in Finance,, Chapman & Hall/CRC, (2013). doi: 10.1201/b14989.

[47]

H. Thorisson, Coupling and shift-coupling random sequences,, Doeblin and Modern Probability, 149 (1993), 85. doi: 10.1090/conm/149/01280.

[48]

J. M. van Campenhout and T. M. Cover, Maximum entropy and conditional probability,, IEEE Infor. Th., IT-27 (1981), 483. doi: 10.1109/TIT.1981.1056374.

[49]

J. van Neumann, The general and logical theory of automata,, Cerebral Mechanisms in Behavior, (1951), 1.

[50]

P. Walters, An Introduction to Ergodic Theory,, Spinger, (1982).

[51]

S. Wolfram, A New Kind of Science,, Wolfram media, (2002).

[52]

G. G. Yin and C. Zhu, Hybrid Switching Diffusions: Properties and Applications,, Springer, (2010). doi: 10.1007/978-1-4419-1105-6.

[53]

X.-J. Zhang, H. Qian and M. Qian, Stochastic theory of nonequilibrium steady states and its applications (Part I),, Phys. Rep., 510 (2012), 1.

[54]

X.-J. Zhang, M. Qian and H. Qian, Stochastic dynamics of electrical membrane with voltage-dependent ion channel fluctuations,, Europhys. Lett., 106 (2014).

[1]

Yujun Zhu. Preimage entropy for random dynamical systems. Discrete & Continuous Dynamical Systems - A, 2007, 18 (4) : 829-851. doi: 10.3934/dcds.2007.18.829

[2]

Deena Schmidt, Janet Best, Mark S. Blumberg. Random graph and stochastic process contributions to network dynamics. Conference Publications, 2011, 2011 (Special) : 1279-1288. doi: 10.3934/proc.2011.2011.1279

[3]

Daoyi Xu, Yumei Huang, Zhiguo Yang. Existence theorems for periodic Markov process and stochastic functional differential equations. Discrete & Continuous Dynamical Systems - A, 2009, 24 (3) : 1005-1023. doi: 10.3934/dcds.2009.24.1005

[4]

Ralf Banisch, Carsten Hartmann. A sparse Markov chain approximation of LQ-type stochastic control problems. Mathematical Control & Related Fields, 2016, 6 (3) : 363-389. doi: 10.3934/mcrf.2016007

[5]

Xiangnan He, Wenlian Lu, Tianping Chen. On transverse stability of random dynamical system. Discrete & Continuous Dynamical Systems - A, 2013, 33 (2) : 701-721. doi: 10.3934/dcds.2013.33.701

[6]

Karsten Keller, Sergiy Maksymenko, Inga Stolz. Entropy determination based on the ordinal structure of a dynamical system. Discrete & Continuous Dynamical Systems - B, 2015, 20 (10) : 3507-3524. doi: 10.3934/dcdsb.2015.20.3507

[7]

Ralf Banisch, Carsten Hartmann. Addendum to "A sparse Markov chain approximation of LQ-type stochastic control problems". Mathematical Control & Related Fields, 2017, 7 (4) : 623-623. doi: 10.3934/mcrf.2017023

[8]

Zhiming Li, Lin Shu. The metric entropy of random dynamical systems in a Hilbert space: Characterization of invariant measures satisfying Pesin's entropy formula. Discrete & Continuous Dynamical Systems - A, 2013, 33 (9) : 4123-4155. doi: 10.3934/dcds.2013.33.4123

[9]

Junyi Tu, Yuncheng You. Random attractor of stochastic Brusselator system with multiplicative noise. Discrete & Continuous Dynamical Systems - A, 2016, 36 (5) : 2757-2779. doi: 10.3934/dcds.2016.36.2757

[10]

Samuel N. Cohen, Lukasz Szpruch. On Markovian solutions to Markov Chain BSDEs. Numerical Algebra, Control & Optimization, 2012, 2 (2) : 257-269. doi: 10.3934/naco.2012.2.257

[11]

Bixiang Wang. Stochastic bifurcation of pathwise random almost periodic and almost automorphic solutions for random dynamical systems. Discrete & Continuous Dynamical Systems - A, 2015, 35 (8) : 3745-3769. doi: 10.3934/dcds.2015.35.3745

[12]

Mario Roy, Mariusz Urbański. Random graph directed Markov systems. Discrete & Continuous Dynamical Systems - A, 2011, 30 (1) : 261-298. doi: 10.3934/dcds.2011.30.261

[13]

María J. Garrido–Atienza, Kening Lu, Björn Schmalfuss. Random dynamical systems for stochastic partial differential equations driven by a fractional Brownian motion. Discrete & Continuous Dynamical Systems - B, 2010, 14 (2) : 473-493. doi: 10.3934/dcdsb.2010.14.473

[14]

Paulina Grzegorek, Michal Kupsa. Exponential return times in a zero-entropy process. Communications on Pure & Applied Analysis, 2012, 11 (3) : 1339-1361. doi: 10.3934/cpaa.2012.11.1339

[15]

Lianfa He, Hongwen Zheng, Yujun Zhu. Shadowing in random dynamical systems. Discrete & Continuous Dynamical Systems - A, 2005, 12 (2) : 355-362. doi: 10.3934/dcds.2005.12.355

[16]

Philippe Marie, Jérôme Rousseau. Recurrence for random dynamical systems. Discrete & Continuous Dynamical Systems - A, 2011, 30 (1) : 1-16. doi: 10.3934/dcds.2011.30.1

[17]

Johnathan M. Bardsley. Gaussian Markov random field priors for inverse problems. Inverse Problems & Imaging, 2013, 7 (2) : 397-416. doi: 10.3934/ipi.2013.7.397

[18]

Manfred Denker, Yuri Kifer, Manuel Stadlbauer. Thermodynamic formalism for random countable Markov shifts. Discrete & Continuous Dynamical Systems - A, 2008, 22 (1&2) : 131-164. doi: 10.3934/dcds.2008.22.131

[19]

Manfred Denker, Yuri Kifer, Manuel Stadlbauer. Corrigendum to: Thermodynamic formalism for random countable Markov shifts. Discrete & Continuous Dynamical Systems - A, 2015, 35 (1) : 593-594. doi: 10.3934/dcds.2015.35.593

[20]

Xiaoyue Li, Xuerong Mao. Population dynamical behavior of non-autonomous Lotka-Volterra competitive system with random perturbation. Discrete & Continuous Dynamical Systems - A, 2009, 24 (2) : 523-545. doi: 10.3934/dcds.2009.24.523

2017 Impact Factor: 0.972

Metrics

  • PDF downloads (30)
  • HTML views (0)
  • Cited by (2)

Other articles
by authors

[Back to Top]