- Previous Article
- MBE Home
- This Issue
-
Next Article
Structural phase transitions in neural networks
Network inference with hidden units
1. | Department of Mathematics, Stockholm University, Kräftriket, S-106 91 Stockholm, Sweden |
2. | Nordita, Stockholm University and KTH, Roslagstullsbacken 23, S-106 91 Stockholm, Sweden |
References:
[1] |
D. Ackley, G. E. Hinton and T. J. Sejnowski, A learning algorithm for Boltzmann machines, Cogn. Sci., 9 (1985), 147-169. |
[2] |
H. Akaike, A new look at the statistical model identification. System identification and time-series analysis, IEE Transactions on Automatic Control, AC-19 (1974), 716-723. |
[3] |
D. Barber, "Bayesian Reasoning and Machine Learning," chapter 11, Cambridge Univ. Press, 2012. |
[4] |
A. P. Dempster, N. M. Laird and D. B. Rubin, Maximum likelihood from incomplete data via the EM algorithm. With discussion, J. Roy. Stat. Soc. B, 39 (1977), 1-38. |
[5] |
B. Dunn and Y. Roudi, Learning and inference in a nonequilibrium Ising model with hidden nodes, Phys. Rev. E, 87 (2013), 022127.
doi: 10.1103/PhysRevE.87.022127. |
[6] |
R. J. Glauber, Time-dependent statistics of the Ising model, J. Math. Phys., 4 (1963), 294-307.
doi: 10.1063/1.1703954. |
[7] |
J. Hertz, Y. Roudi and J. Tyrcha, Ising models for inferring network structure from spike data, in "Principles of Neural Coding" (eds. S. Panzeri and R. R. Quiroga), CRC Press, (2013), 527-546.
doi: 10.1201/b14756-31. |
[8] |
M. Mézard, G. Parisi and M. Virasoro, "Spin Glass Theory and Beyond," chapter 2, World Scientific Lecture Notes in Physics, 9, World Scientific Publishing Co., Inc., Teaneck, NJ, 1987. |
[9] |
B. A. Pearlmutter, Learning state space trajectories in recurrent neural networks, Neural Computation, 1 (1989), 263-269. |
[10] |
P. Peretto, Collective properties of neural networks: A statistical physics approach, Biol. Cybern., 50 (1984), 51-62.
doi: 10.1007/BF00317939. |
[11] |
F. J. Pineda, Generalization of back-propagation to recurrent neural networks, Phys. Rev. Lett., 59 (1987), 2229-2232.
doi: 10.1103/PhysRevLett.59.2229. |
[12] |
Y. Roudi and J. Hertz, Mean-field theory for nonequilibrium network reconstruction, Phys. Rev. Lett., 106 (2011), 048702.
doi: 10.1103/PhysRevLett.106.048702. |
[13] |
Y. Roudi, J. Tyrcha and J. Hertz, The Ising model for neural data: Model quality and approximate methods for extracting functional connectivity, Phys. Rev. E, 79 (2009), 051915.
doi: 10.1103/PhysRevE.79.051915. |
[14] |
D. E. Rumelhart, G. E. Hinton and R. J. Williams, Learning Internal Representations by Error Propagation, in "Parallel Distributed Processing" (eds. D. E. Rumelhart and J. L. McClelland), Vol. 1, Chapter 8, MIT Press, 1986. |
[15] |
L. K. Saul, T. Jaakkola and M. I. Jordan, Mean field theory for sigmoid belief networks, J. Art. Intel. Res., 4 (1996), 61-76. |
[16] |
E. Schneidman, M. J. Berry, R. Segev and W. Bialek, Weak pairwise correlations imply strongly correlated network states in a neural population, Nature, 440 (2006), 1007-1012.
doi: 10.1038/nature04701. |
[17] |
G. E. Schwarz, Estimating the dimension of a model, Annals of Statistics, 6 (1978), 461-464.
doi: 10.1214/aos/1176344136. |
[18] |
R. Sundberg, Maximum likelihood theory for incomplete data from an exponential family, Scand. J. Statistics, 1 (1974), 49-58. |
[19] |
D. Sherrington and S. Kirkpatrick, Solvable model of a spin-glass, Phys. Rev. Lett., 35 (1975), 1792-1796.
doi: 10.1103/PhysRevLett.35.1792. |
[20] |
D. J. Thouless, P. W. Anderson and R. G. Palmer, Solution of "soluble model of a spin glass,'' Philos. Mag., 92 (1974), 272-279. |
[21] |
R. J. Williams and D. Zipser, A learning algorithm for continually running fully recurrent networks, Neural Comp., 1 (1989), 270-280.
doi: 10.1162/neco.1989.1.2.270. |
show all references
References:
[1] |
D. Ackley, G. E. Hinton and T. J. Sejnowski, A learning algorithm for Boltzmann machines, Cogn. Sci., 9 (1985), 147-169. |
[2] |
H. Akaike, A new look at the statistical model identification. System identification and time-series analysis, IEE Transactions on Automatic Control, AC-19 (1974), 716-723. |
[3] |
D. Barber, "Bayesian Reasoning and Machine Learning," chapter 11, Cambridge Univ. Press, 2012. |
[4] |
A. P. Dempster, N. M. Laird and D. B. Rubin, Maximum likelihood from incomplete data via the EM algorithm. With discussion, J. Roy. Stat. Soc. B, 39 (1977), 1-38. |
[5] |
B. Dunn and Y. Roudi, Learning and inference in a nonequilibrium Ising model with hidden nodes, Phys. Rev. E, 87 (2013), 022127.
doi: 10.1103/PhysRevE.87.022127. |
[6] |
R. J. Glauber, Time-dependent statistics of the Ising model, J. Math. Phys., 4 (1963), 294-307.
doi: 10.1063/1.1703954. |
[7] |
J. Hertz, Y. Roudi and J. Tyrcha, Ising models for inferring network structure from spike data, in "Principles of Neural Coding" (eds. S. Panzeri and R. R. Quiroga), CRC Press, (2013), 527-546.
doi: 10.1201/b14756-31. |
[8] |
M. Mézard, G. Parisi and M. Virasoro, "Spin Glass Theory and Beyond," chapter 2, World Scientific Lecture Notes in Physics, 9, World Scientific Publishing Co., Inc., Teaneck, NJ, 1987. |
[9] |
B. A. Pearlmutter, Learning state space trajectories in recurrent neural networks, Neural Computation, 1 (1989), 263-269. |
[10] |
P. Peretto, Collective properties of neural networks: A statistical physics approach, Biol. Cybern., 50 (1984), 51-62.
doi: 10.1007/BF00317939. |
[11] |
F. J. Pineda, Generalization of back-propagation to recurrent neural networks, Phys. Rev. Lett., 59 (1987), 2229-2232.
doi: 10.1103/PhysRevLett.59.2229. |
[12] |
Y. Roudi and J. Hertz, Mean-field theory for nonequilibrium network reconstruction, Phys. Rev. Lett., 106 (2011), 048702.
doi: 10.1103/PhysRevLett.106.048702. |
[13] |
Y. Roudi, J. Tyrcha and J. Hertz, The Ising model for neural data: Model quality and approximate methods for extracting functional connectivity, Phys. Rev. E, 79 (2009), 051915.
doi: 10.1103/PhysRevE.79.051915. |
[14] |
D. E. Rumelhart, G. E. Hinton and R. J. Williams, Learning Internal Representations by Error Propagation, in "Parallel Distributed Processing" (eds. D. E. Rumelhart and J. L. McClelland), Vol. 1, Chapter 8, MIT Press, 1986. |
[15] |
L. K. Saul, T. Jaakkola and M. I. Jordan, Mean field theory for sigmoid belief networks, J. Art. Intel. Res., 4 (1996), 61-76. |
[16] |
E. Schneidman, M. J. Berry, R. Segev and W. Bialek, Weak pairwise correlations imply strongly correlated network states in a neural population, Nature, 440 (2006), 1007-1012.
doi: 10.1038/nature04701. |
[17] |
G. E. Schwarz, Estimating the dimension of a model, Annals of Statistics, 6 (1978), 461-464.
doi: 10.1214/aos/1176344136. |
[18] |
R. Sundberg, Maximum likelihood theory for incomplete data from an exponential family, Scand. J. Statistics, 1 (1974), 49-58. |
[19] |
D. Sherrington and S. Kirkpatrick, Solvable model of a spin-glass, Phys. Rev. Lett., 35 (1975), 1792-1796.
doi: 10.1103/PhysRevLett.35.1792. |
[20] |
D. J. Thouless, P. W. Anderson and R. G. Palmer, Solution of "soluble model of a spin glass,'' Philos. Mag., 92 (1974), 272-279. |
[21] |
R. J. Williams and D. Zipser, A learning algorithm for continually running fully recurrent networks, Neural Comp., 1 (1989), 270-280.
doi: 10.1162/neco.1989.1.2.270. |
[1] |
Deng Lu, Maria De Iorio, Ajay Jasra, Gary L. Rosner. Bayesian inference for latent chain graphs. Foundations of Data Science, 2020, 2 (1) : 35-54. doi: 10.3934/fods.2020003 |
[2] |
Michael Herty, Lorenzo Pareschi, Giuseppe Visconti. Mean field models for large data–clustering problems. Networks and Heterogeneous Media, 2020, 15 (3) : 463-487. doi: 10.3934/nhm.2020027 |
[3] |
Roberto Natalini, Thierry Paul. On the mean field limit for Cucker-Smale models. Discrete and Continuous Dynamical Systems - B, 2022, 27 (5) : 2873-2889. doi: 10.3934/dcdsb.2021164 |
[4] |
Gerasimenko Viktor. Heisenberg picture of quantum kinetic evolution in mean-field limit. Kinetic and Related Models, 2011, 4 (1) : 385-399. doi: 10.3934/krm.2011.4.385 |
[5] |
Michael Herty, Torsten Trimborn, Giuseppe Visconti. Mean-field and kinetic descriptions of neural differential equations. Foundations of Data Science, 2022, 4 (2) : 271-298. doi: 10.3934/fods.2022007 |
[6] |
Xiaoyu Zheng, Peter Palffy-Muhoray. One order parameter tensor mean field theory for biaxial liquid crystals. Discrete and Continuous Dynamical Systems - B, 2011, 15 (2) : 475-490. doi: 10.3934/dcdsb.2011.15.475 |
[7] |
Rong Yang, Li Chen. Mean-field limit for a collision-avoiding flocking system and the time-asymptotic flocking dynamics for the kinetic equation. Kinetic and Related Models, 2014, 7 (2) : 381-400. doi: 10.3934/krm.2014.7.381 |
[8] |
Joachim Crevat. Mean-field limit of a spatially-extended FitzHugh-Nagumo neural network. Kinetic and Related Models, 2019, 12 (6) : 1329-1358. doi: 10.3934/krm.2019052 |
[9] |
Fabio Bagagiolo, Rosario Maggistro, Raffaele Pesenti. Origin-to-destination network flow with path preferences and velocity controls: A mean field game-like approach. Journal of Dynamics and Games, 2021, 8 (4) : 359-380. doi: 10.3934/jdg.2021007 |
[10] |
Evangelos Evangelou. Approximate Bayesian inference for geostatistical generalised linear models. Foundations of Data Science, 2019, 1 (1) : 39-60. doi: 10.3934/fods.2019002 |
[11] |
Oliver J. Maclaren, Helen M. Byrne, Alexander G. Fletcher, Philip K. Maini. Models, measurement and inference in epithelial tissue dynamics. Mathematical Biosciences & Engineering, 2015, 12 (6) : 1321-1340. doi: 10.3934/mbe.2015.12.1321 |
[12] |
Chjan C. Lim. Extremal free energy in a simple mean field theory for a coupled Barotropic fluid - rotating sphere system. Discrete and Continuous Dynamical Systems, 2007, 19 (2) : 361-386. doi: 10.3934/dcds.2007.19.361 |
[13] |
Samuel N. Cohen. Uncertainty and filtering of hidden Markov models in discrete time. Probability, Uncertainty and Quantitative Risk, 2020, 5 (0) : 4-. doi: 10.1186/s41546-020-00046-x |
[14] |
William Chad Young, Adrian E. Raftery, Ka Yee Yeung. A posterior probability approach for gene regulatory network inference in genetic perturbation data. Mathematical Biosciences & Engineering, 2016, 13 (6) : 1241-1251. doi: 10.3934/mbe.2016041 |
[15] |
Franco Flandoli, Matti Leimbach. Mean field limit with proliferation. Discrete and Continuous Dynamical Systems - B, 2016, 21 (9) : 3029-3052. doi: 10.3934/dcdsb.2016086 |
[16] |
Yu Yang, Dongmei Xiao. Influence of latent period and nonlinear incidence rate on the dynamics of SIRS epidemiological models. Discrete and Continuous Dynamical Systems - B, 2010, 13 (1) : 195-211. doi: 10.3934/dcdsb.2010.13.195 |
[17] |
Napoleon Bame, Samuel Bowong, Josepha Mbang, Gauthier Sallet, Jean-Jules Tewa. Global stability analysis for SEIS models with n latent classes. Mathematical Biosciences & Engineering, 2008, 5 (1) : 20-33. doi: 10.3934/mbe.2008.5.20 |
[18] |
Darryl D. Holm, Vakhtang Putkaradze, Cesare Tronci. Collisionless kinetic theory of rolling molecules. Kinetic and Related Models, 2013, 6 (2) : 429-458. doi: 10.3934/krm.2013.6.429 |
[19] |
Emmanuel Frénod, Mathieu Lutz. On the Geometrical Gyro-Kinetic theory. Kinetic and Related Models, 2014, 7 (4) : 621-659. doi: 10.3934/krm.2014.7.621 |
[20] |
Dong-Mei Zhu, Wai-Ki Ching, Robert J. Elliott, Tak-Kuen Siu, Lianmin Zhang. Hidden Markov models with threshold effects and their applications to oil price forecasting. Journal of Industrial and Management Optimization, 2017, 13 (2) : 757-773. doi: 10.3934/jimo.2016045 |
2018 Impact Factor: 1.313
Tools
Metrics
Other articles
by authors
[Back to Top]