Stochastic dynamics: Markov chains and random transformations

Felix X.-F.  Ye; Yue  Wang; Hong  Qian

doi:10.3934/dcdsb.2016050

Article Contents

2016, Volume 21, Issue 7: 2337-2361. Doi: 10.3934/dcdsb.2016050

This issue Previous Article Strong Allee effect in a stochastic logistic model with mate limitation and stochastic immigration Next Article Stationary distribution of stochastic SIRS epidemic model with standard incidence

Stochastic dynamics: Markov chains and random transformations

1.
Department of Applied Mathematics, University of Washington, Seattle, WA 98195-3925

Received: May 31, 2015

Revised: January 31, 2016

Published: August 2016

Abstract Related Papers Cited by

Abstract

This article outlines an attempt to lay the groundwork for understanding stochastic dynamical descriptions of biological processes in terms of a discrete-state space, discrete-time random dynamical system (RDS), or random transformation approach. Such mathematics is not new for continuous systems, but the discrete state space formulation significantly reduces the technical requirements for its introduction to a much broader audiences. In particular, we establish some elementary contradistinctions between Markov chain (MC) and RDS descriptions of a stochastic dynamics. It is shown that a given MC is compatible with many possible RDS, and we study in particular the corresponding RDS with maximum metric entropy. Specifically, we show an emergent behavior of an MC with a unique absorbing and aperiodic communicating class, after all the trajectories of the RDS synchronizes. In biological modeling, it is now widely acknowledged that stochastic dynamics is a more complete description of biological reality than deterministic equations; here we further suggest that the RDS description could be a more refined description of stochastic dynamics than a Markov process. Possible applications of discrete-state RDS are systems with fluctuating law of motion, or environment, rather than inherent stochastic movements of individuals.

Keywords:

Mathematics Subject Classification: Primary: 60G10, 92B05; Secondary: 34F05.

Citation:

Related Papers

Cited by

References

[1]	L. J. S. Allen, An Introduction to Stochastic Processes with Applications to Biology, $2^{nd}$ edition, CRC Press, Boca Raton, FL, 2011.
[2]	L. Arnold, Random Dynamical Systems, Springer, New York, 1998.doi: 10.1007/978-3-662-12878-7.
[3]	L. Arnold and H. Crauel, Random dynamical systems, in Lyapunov Exponents, (eds. L. Arnold, H. Crauel and J.-P. Eckmann), Springer, Berlin, 1486 (2006), 1-22.doi: 10.1007/BFb0086654.
[4]	P. H. Baxendale, Statistical equilibrium and two-point motion for a stochastic flow of diffeomorphisms. Spatial stochastic processes, Progr. Probab., 19 (1991), 189-218.
[5]	R. Bhattacharya and M. Majumdar, Random Dynamical Systems: Theory and Applications, Cambridge Univ. Press, U.K., 2007.doi: 10.1017/CBO9780511618628.
[6]	G. Birkhoff, Three observations on linear algebra, Univ. Nac. Tucumán. Revista A, 5 (1946), 147-151.
[7]	R. M Blumenthal and H. K. Corson, On continuous collections of measures, Proc. 6th Berkeley Symp. on Math. Stat. and Prob., 2 (1972), 33-40.
[8]	K.-S. Chan and H. Tong, Chaos: A Statistical Perspective, Springer, New York, 2001.doi: 10.1007/978-1-4757-3464-5.
[9]	Y.-D. Chen, Asymmetry and external noise-induced free energy transduction, Proc. Natl. Acad. Sci. U.S.A., 84 (1987), 729-733.doi: 10.1073/pnas.84.3.729.
[10]	T. Downarowicz, Entropy in Dynamical Systems, Cambridge Univ. Press, UK, 2011.doi: 10.1017/CBO9780511976155.
[11]	S. P. Ellner and J. Guckenheimer, Dynamic Models in Biology, Princeton Univ. Press, NJ, 2006.
[12]	G. Froyland, Extracting dynamical behavior via Markov models, Nonlinear dynamics and statistics (Cambridge, 1998), Birkhäuser Boston, Boston, MA, (2001), 281-321.
[13]	G. Gallavotti, Statistical Mechanics: A Short Treatise, Springer, New York, 1999.doi: 10.1007/978-3-662-03952-6.
[14]	H. Ge, M. Qian and H. Qian, Stochastic theory of nonequilibrium steady states (Part II): Applications in chemical biophysics, Phys. Rep., 510 (2012), 87-118.doi: 10.1016/j.physrep.2011.09.001.
[15]	B. Hasselblatt and A. Katok, Principal structures, in Handbook of Dynamical Systems, vol. 1A (eds. B. Hasselblatt and A. Katok), Elsevier, Amsterdam, 1 (2002), 1-203.doi: 10.1016/S1874-575X(02)80003-0.
[16]	W. Horsthemke and R. Lefever, Noise-Induced Transitions: Theory and Applications in Physics, Chemistry, and Biology, Springer, New York, 1984.
[17]	E. Isaacson and H. B. Keller, Analysis of Numerical Methods, Dover, New York, 1966.
[18]	S. L. Kalpazidou, Cycle Representations of Markov Processes, $2^{nd}$ edition, Springer, New York, 2006.
[19]	M. Keane, Ergodic theory and subshifts of finite type, in Ergodic theory, Symbolic Dynamics and Hyperbolic Spaces (eds. T. Bedford, M. Keane and C. Series), Oxford, (1991), 35-70.
[20]	A. I. Khinchin, Mathematical Foundations of Information Theory, Dover, New York, 1957.
[21]	Yu. Kifer, Ergodic Theory of Random Transformations, Birkhäuser, Basel, 1986.doi: 10.1007/978-1-4684-9175-3.
[22]	Yu. Kifer and P.-D. Liu, Random dynamics, in Handbook of Dynamical Systems, (eds. B. Hasselblatt and A. Katok), Elsevier, Amsterdam, 1 (2006), 379-499.doi: 10.1016/S1874-575X(06)80030-5.
[23]	A. Lasota and M. C. Mackey, Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics, $2^{nd}$ edition, Springer-Verlag, New York, 1994.doi: 10.1007/978-1-4612-4286-4.
[24]	V. Lecomte, C. Appert-Rolland and F. van Wijland, Thermodynamic formalism for systems with Markov dynamics, J. Stat. Phys., 127 (2007), 51-106.doi: 10.1007/s10955-006-9254-0.
[25]	T. Liggett, The coupling technique in interacting particle systems, In Doeblin and Modern Probability, (ed. H. Cohn), AMS, Providence, 149 (1993), 73-83.doi: 10.1090/conm/149/01271.
[26]	K. K. Lin, Stimulus-response reliability of biological networks, in Nonautonomous and Random Dynamical Systems in Life Sciences (eds. P. Kloeden and C. Poetzsche), Springer, New York, 2102 (2012), 135-161.doi: 10.1007/978-3-319-03080-7_4.
[27]	D. Lind and B. Marcus, An Introduction to Symbolic Dynamics and Coding, Cambridge Univ. Press, U.K., 1995.doi: 10.1017/CBO9780511626302.
[28]	X. Mao and C. Yuan, Stochastic Differential Equations with Markovian Switching, Imperial College Press, UK, 2006.doi: 10.1142/p473.
[29]	M. Marcus, H. Minc and B. Moyls, Some results on non-negative matrices, J. Res. Nat. Bur. Standards Sec. B, 65 (1961), 205-209.doi: 10.6028/jres.065B.019.
[30]	M. L. Mehta, Random Matrices, $3^{rd}$ edition, Academic Press, New York, 2004.
[31]	H. Minc, Nonnegative Matrices, John Wiley & Sons, New York, 1988.
[32]	J. A. Morrison and J. McKenna, Analysis of some stochastic ordinary differential equations, in Stochastic Differential Equations, SIAM-AMS Proc., Vol. 6 (eds. J. B. Keller and H. P. McKean), Amer. Math. Soc., Providence, R.I., (1973), 97-161.
[33]	J. D. Murray, Mathematical Biology: I. An Introduction, $3^{rd}$ edition, Springer, New York, 2002.
[34]	D. S. Ornstein, Ergodic theory, randomness, and "chaos'', Science, 243 (1989), 182-187.doi: 10.1126/science.243.4888.182.
[35]	Y. Peres, Analytic dependence of Lyapunov exponents on transition probabilities, in Lyapunov Exponents, (eds. L. Arnold, H. Crauel and J.-P. Eckmann), Springer, Berlin, 1486 (1991), 64-80.doi: 10.1007/BFb0086658.
[36]	M. A. Pinsky, Lectures on Random Evolution, World Scientific, Singapore, 1991.doi: 10.1142/1328.
[37]	H. Qian and J. A. Schellman, Helix-coil theories: A comparative studies for finite length polypeptides, J. Phys. Chem., 96 (1992), 3987-3994.doi: 10.1021/j100189a015.
[38]	H. Qian, The mathematical theory of molecular motor movement and chemomechanical energy transduction, J. Math. Chem., 27 (2000), 219-234.doi: 10.1023/A:1026428320489.
[39]	H. Qian, Cooperativity in cellular biochemical processes: Noise-enhanced sensitivity, fluctuating enzyme, bistability with nonlinear feedback, and other mechanisms for sigmoidal responses, Annu. Rev. Biophys., 41 (2012), 179-204.doi: 10.1146/annurev-biophys-050511-102240.
[40]	H. Qian, S. Kjelstrup, A. B. Kolomeisky and D. Bedeaux, Entropy production in mesoscopic stochastic thermodynamics \| Nonequilibrium steady state cycles driven by chemical potentials, temperatures, and mechanical forces, J. Phys. Cond. Matt. 28 (2016), 153004. arXiv:1601.04018
[41]	M. Qian, J.-S. Xie and S. Zhu, Smooth Ergodic Theory for Endomorphisms, Lec. Notes Math. vol. 1978, Springer, New York, 2009.doi: 10.1007/978-3-642-01954-8.
[42]	M. Qian and F.-X. Zhang, Entropy production rate of the coupled diffusion process, J. Theor. Probab., 24 (2011), 729-745.doi: 10.1007/s10959-011-0352-9.
[43]	R. T. Rockafellar, Convex Analysis, Princeton Univ. Press, NJ, 1970.
[44]	M. Santillán and H. Qian, Irreversible thermodynamics in multiscale stochastic dynamical systems, Phys. Rev. E, 83 (2011), 041130. arXiv:1003.3513
[45]	S. H. Strogatz, Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering, Westview Press, Boulder, 2001.
[46]	A. Swishchuk and S. Islam, Random Dynamical Systems in Finance, Chapman & Hall/CRC, New York, 2013.doi: 10.1201/b14989.
[47]	H. Thorisson, Coupling and shift-coupling random sequences, Doeblin and Modern Probability, (ed. H. Cohn), AMS, Providence, 149 (1993), 85-95.doi: 10.1090/conm/149/01280.
[48]	J. M. van Campenhout and T. M. Cover, Maximum entropy and conditional probability, IEEE Infor. Th., IT-27 (1981), 483-489.doi: 10.1109/TIT.1981.1056374.
[49]	J. van Neumann, The general and logical theory of automata, Cerebral Mechanisms in Behavior, The Hixon Symposium, pp. 1-31; discussion, pp. 32-41. John Wiley & Sons, Inc., New York, N. Y.; Chapman & Hall, Ltd., London, 1951.
[50]	P. Walters, An Introduction to Ergodic Theory, Spinger, New York, 1982.
[51]	S. Wolfram, A New Kind of Science, Wolfram media, Champaign, 2002.
[52]	G. G. Yin and C. Zhu, Hybrid Switching Diffusions: Properties and Applications, Springer, New York, 2010.doi: 10.1007/978-1-4419-1105-6.
[53]	X.-J. Zhang, H. Qian and M. Qian, Stochastic theory of nonequilibrium steady states and its applications (Part I), Phys. Rep., 510 (2012), 1-86.
[54]	X.-J. Zhang, M. Qian and H. Qian, Stochastic dynamics of electrical membrane with voltage-dependent ion channel fluctuations, Europhys. Lett., 106 (2014), 10002. arXiv:1404.1548