2014, 11(2): 233-256. doi: 10.3934/mbe.2014.11.233

Synaptic energy drives the information processing mechanisms in spiking neural networks

1. 

Faculty of Mathematics and Computer Science, Dept. of Computer Engineering, Leipzig University, Germany, Germany

Received  November 2012 Revised  March 2013 Published  October 2013

Flow of energy and free energy minimization underpins almost every aspect of naturally occurring physical mechanisms. Inspired by this fact this work establishes an energy-based framework that spans the multi-scale range of biological neural systems and integrates synaptic dynamic, synchronous spiking activity and neural states into one consistent working paradigm. Following a bottom-up approach, a hypothetical energy function is proposed for dynamic synaptic models based on the theoretical thermodynamic principles and the Hopfield networks. We show that a synapse exposes stable operating points in terms of its excitatory postsynaptic potential as a function of its synaptic strength. We postulate that synapses in a network operating at these stable points can drive this network to an internal state of synchronous firing. The presented analysis is related to the widely investigated temporal coherent activities (cell assemblies) over a certain range of time scales (binding-by-synchrony). This introduces a novel explanation of the observed (poly)synchronous activities within networks regarding the synaptic (coupling) functionality. On a network level the transitions from one firing scheme to the other express discrete sets of neural states. The neural states exist as long as the network sustains the internal synaptic energy.
Citation: Karim El Laithy, Martin Bogdan. Synaptic energy drives the information processing mechanisms in spiking neural networks. Mathematical Biosciences & Engineering, 2014, 11 (2) : 233-256. doi: 10.3934/mbe.2014.11.233
References:
[1]

A. B. Barrett, G. O. Billings, R. G. M. Morris and M. C. W. van Rossum, State based model of long-term potentiation and synaptic tagging and capture,, PLoS Comput. Biol., 5 (2009). doi: 10.1371/journal.pcbi.1000259. Google Scholar

[2]

K. El-laithy, Towards a Brain-Inspired Information Processing System: Modeling and Analysis of Synaptic Dynamics,, Ph.D thesis, (2011). Google Scholar

[3]

K. El-Laithy and M. Bogdan, Synchrony state generation in artificial neural networks with stochastic synapses,, in Artificial Neural Networks - ICANN 2009, (2009), 181. Google Scholar

[4]

K. El-Laithy and M. Bogdan, Predicting spike-timing of a thalamic neuron using a stochastic synaptic model,, in ESANN Proceedings, (2010), 357. Google Scholar

[5]

K. El-Laithy and M. Bogdan, A hypothetical free synaptic energy function and related states of synchrony,, in Artificial Neural Networks and Machine Learning - ICANN 2011, (2011), 40. Google Scholar

[6]

K. El-Laithy and M. Bogdan, On the capacity of transient internal states in liquid-state machines,, in Artificial Neural Networks and Machine Learning - ICANN 2011, (2011), 56. Google Scholar

[7]

K. El-laithy and M. Bogdan, Synchrony state generation: An approach using stochastic synapses,, J. of Artificial Intelligence and Soft Computing Research, 1 (2011), 17. Google Scholar

[8]

K. El-Laithy and M. Bogdan, Temporal finite-state machines: A novel framework for the general class of dynamic networks,, in ICONIP 2012, (2012), 425. Google Scholar

[9]

C. Eliasmith and C. H. Anderson, Neural Engineering. Computation, Representation, and Dynamics in Neurobiological Systems,, Computational Neuroscience, (2003). Google Scholar

[10]

A. K. Engel, P. Fries, P. Knig, M. Brecht and W. Singer, Temporal binding, binocular rivalry, and consciousness,, Consciousness and Cognition, 8 (1999), 128. Google Scholar

[11]

A. K. Engel and W. Singer, Temporal binding and the neural correlates of sensory awareness,, Trends in Cognitive Sciences, 5 (2001), 16. Google Scholar

[12]

K. Friston, The free-energy principle: A rough guide to the brain?,, Trends in Cognitive Sciences, 13 (2009), 293. Google Scholar

[13]

K. Friston, The free-energy principle: A unified brain theory, Nature Reviews Neuroscience, 11 (2010), 127. Google Scholar

[14]

H. M. Fuchs, Neural Networks with Dynamic Synapses,, Ph.D thesis, (1998). Google Scholar

[15]

J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities,, Proc. Nat. Acad. Sci. U.S.A., 79 (1982), 2554. doi: 10.1073/pnas.79.8.2554. Google Scholar

[16]

Y. Ikegaya, G. Aaron, R. Cossart, D. Aronov, I. Lampl, D. Ferster and R. Yuste, Synfire chains and cortical songs: Temporal modules of cortical activity,, Science, 304 (2004), 559. doi: 10.1126/science.1093173. Google Scholar

[17]

E. Izhikevich, Which model to use for cortical spiking neurons?, IEEE Transactions on Neural Networks, 15 (2004), 1063. doi: 10.1109/TNN.2004.832719. Google Scholar

[18]

E. M. Izhikevich, Polychronization: Computation with spikes,, Neural Comput., 18 (2006), 245. doi: 10.1162/089976606775093882. Google Scholar

[19]

A. Levina, J. M. Herrmann and T. Geisel, Phase transitions towards criticality in a neural system with adaptive interactions,, Physical Review Letters, 102 (2009). doi: 10.1103/PhysRevLett.102.118110. Google Scholar

[20]

J. M. Montgomery and D. V. Madison, State-dependent heterogeneity in synaptic depression between pyramidal cell pairs,, Neuron, 33 (2002), 765. doi: 10.1016/S0896-6273(02)00606-2. Google Scholar

[21]

J. M. Montgomery and D. V. Madison, Discrete synaptic states define a major mechanism of synapse plasticity,, Trends in Neurosciences, 27 (2004), 744. doi: 10.1016/j.tins.2004.10.006. Google Scholar

[22]

A. Morrison, M. Diesmann and W. Gerstner, Phenomenological models of synaptic plasticity based on spike timing,, Biol. Cybernet., 98 (2008), 459. doi: 10.1007/s00422-008-0233-1. Google Scholar

[23]

P. S. Neelakanta and D. DeGroff, Neural Network Modeling: Statistical Mechanics and Cybernetics Perspectives,, CRC Press, (1994). Google Scholar

[24]

A. Revonsuo and J. Newman, Binding and consciousness,, Consciousness and Cognition, 8 (1999), 123. doi: 10.1006/ccog.1999.0393. Google Scholar

[25]

C. Sarasola, A. d'Anjou, F. J. Torrealdea and M. Graña, Minimization of the energy flow in the synchronization of nonidentical chaotic systems,, Phys. Rev. E, 72 (2005). doi: 10.1103/PhysRevE.72.026223. Google Scholar

[26]

A. P. Shon and R. P. N. Rao, Temporal Sequence Learning with Dynamic Synapses,, Technical report, (2003). Google Scholar

[27]

W. Singer, Understanding the brain,, EMBO Reports, 8 (2007). doi: 10.1038/sj.embor.7400994. Google Scholar

[28]

S. Stringer, E. Rolls and T. Trappenberg, Self-organizing continuous attractor network models of hippocampal spatial view cells,, Neurobiology of Learning and Memory, 83 (2005), 79. doi: 10.1016/j.nlm.2004.08.003. Google Scholar

[29]

F. J. Torrealdea, A. d'Anjou, M. Graña and C. Sarasola, Energy aspects of the synchronization of model neurons,, Physical Review E, 74 (2006). doi: 10.1103/PhysRevE.74.011905. Google Scholar

[30]

C. von der Malsburg, The what and why of binding: The modeler's perspective,, Neuron, 24 (1999), 95. Google Scholar

show all references

References:
[1]

A. B. Barrett, G. O. Billings, R. G. M. Morris and M. C. W. van Rossum, State based model of long-term potentiation and synaptic tagging and capture,, PLoS Comput. Biol., 5 (2009). doi: 10.1371/journal.pcbi.1000259. Google Scholar

[2]

K. El-laithy, Towards a Brain-Inspired Information Processing System: Modeling and Analysis of Synaptic Dynamics,, Ph.D thesis, (2011). Google Scholar

[3]

K. El-Laithy and M. Bogdan, Synchrony state generation in artificial neural networks with stochastic synapses,, in Artificial Neural Networks - ICANN 2009, (2009), 181. Google Scholar

[4]

K. El-Laithy and M. Bogdan, Predicting spike-timing of a thalamic neuron using a stochastic synaptic model,, in ESANN Proceedings, (2010), 357. Google Scholar

[5]

K. El-Laithy and M. Bogdan, A hypothetical free synaptic energy function and related states of synchrony,, in Artificial Neural Networks and Machine Learning - ICANN 2011, (2011), 40. Google Scholar

[6]

K. El-Laithy and M. Bogdan, On the capacity of transient internal states in liquid-state machines,, in Artificial Neural Networks and Machine Learning - ICANN 2011, (2011), 56. Google Scholar

[7]

K. El-laithy and M. Bogdan, Synchrony state generation: An approach using stochastic synapses,, J. of Artificial Intelligence and Soft Computing Research, 1 (2011), 17. Google Scholar

[8]

K. El-Laithy and M. Bogdan, Temporal finite-state machines: A novel framework for the general class of dynamic networks,, in ICONIP 2012, (2012), 425. Google Scholar

[9]

C. Eliasmith and C. H. Anderson, Neural Engineering. Computation, Representation, and Dynamics in Neurobiological Systems,, Computational Neuroscience, (2003). Google Scholar

[10]

A. K. Engel, P. Fries, P. Knig, M. Brecht and W. Singer, Temporal binding, binocular rivalry, and consciousness,, Consciousness and Cognition, 8 (1999), 128. Google Scholar

[11]

A. K. Engel and W. Singer, Temporal binding and the neural correlates of sensory awareness,, Trends in Cognitive Sciences, 5 (2001), 16. Google Scholar

[12]

K. Friston, The free-energy principle: A rough guide to the brain?,, Trends in Cognitive Sciences, 13 (2009), 293. Google Scholar

[13]

K. Friston, The free-energy principle: A unified brain theory, Nature Reviews Neuroscience, 11 (2010), 127. Google Scholar

[14]

H. M. Fuchs, Neural Networks with Dynamic Synapses,, Ph.D thesis, (1998). Google Scholar

[15]

J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities,, Proc. Nat. Acad. Sci. U.S.A., 79 (1982), 2554. doi: 10.1073/pnas.79.8.2554. Google Scholar

[16]

Y. Ikegaya, G. Aaron, R. Cossart, D. Aronov, I. Lampl, D. Ferster and R. Yuste, Synfire chains and cortical songs: Temporal modules of cortical activity,, Science, 304 (2004), 559. doi: 10.1126/science.1093173. Google Scholar

[17]

E. Izhikevich, Which model to use for cortical spiking neurons?, IEEE Transactions on Neural Networks, 15 (2004), 1063. doi: 10.1109/TNN.2004.832719. Google Scholar

[18]

E. M. Izhikevich, Polychronization: Computation with spikes,, Neural Comput., 18 (2006), 245. doi: 10.1162/089976606775093882. Google Scholar

[19]

A. Levina, J. M. Herrmann and T. Geisel, Phase transitions towards criticality in a neural system with adaptive interactions,, Physical Review Letters, 102 (2009). doi: 10.1103/PhysRevLett.102.118110. Google Scholar

[20]

J. M. Montgomery and D. V. Madison, State-dependent heterogeneity in synaptic depression between pyramidal cell pairs,, Neuron, 33 (2002), 765. doi: 10.1016/S0896-6273(02)00606-2. Google Scholar

[21]

J. M. Montgomery and D. V. Madison, Discrete synaptic states define a major mechanism of synapse plasticity,, Trends in Neurosciences, 27 (2004), 744. doi: 10.1016/j.tins.2004.10.006. Google Scholar

[22]

A. Morrison, M. Diesmann and W. Gerstner, Phenomenological models of synaptic plasticity based on spike timing,, Biol. Cybernet., 98 (2008), 459. doi: 10.1007/s00422-008-0233-1. Google Scholar

[23]

P. S. Neelakanta and D. DeGroff, Neural Network Modeling: Statistical Mechanics and Cybernetics Perspectives,, CRC Press, (1994). Google Scholar

[24]

A. Revonsuo and J. Newman, Binding and consciousness,, Consciousness and Cognition, 8 (1999), 123. doi: 10.1006/ccog.1999.0393. Google Scholar

[25]

C. Sarasola, A. d'Anjou, F. J. Torrealdea and M. Graña, Minimization of the energy flow in the synchronization of nonidentical chaotic systems,, Phys. Rev. E, 72 (2005). doi: 10.1103/PhysRevE.72.026223. Google Scholar

[26]

A. P. Shon and R. P. N. Rao, Temporal Sequence Learning with Dynamic Synapses,, Technical report, (2003). Google Scholar

[27]

W. Singer, Understanding the brain,, EMBO Reports, 8 (2007). doi: 10.1038/sj.embor.7400994. Google Scholar

[28]

S. Stringer, E. Rolls and T. Trappenberg, Self-organizing continuous attractor network models of hippocampal spatial view cells,, Neurobiology of Learning and Memory, 83 (2005), 79. doi: 10.1016/j.nlm.2004.08.003. Google Scholar

[29]

F. J. Torrealdea, A. d'Anjou, M. Graña and C. Sarasola, Energy aspects of the synchronization of model neurons,, Physical Review E, 74 (2006). doi: 10.1103/PhysRevE.74.011905. Google Scholar

[30]

C. von der Malsburg, The what and why of binding: The modeler's perspective,, Neuron, 24 (1999), 95. Google Scholar

[1]

Xilin Fu, Zhang Chen. New discrete analogue of neural networks with nonlinear amplification function and its periodic dynamic analysis. Conference Publications, 2007, 2007 (Special) : 391-398. doi: 10.3934/proc.2007.2007.391

[2]

Alessia Marigo, Benedetto Piccoli. A model for biological dynamic networks. Networks & Heterogeneous Media, 2011, 6 (4) : 647-663. doi: 10.3934/nhm.2011.6.647

[3]

K Najarian. On stochastic stability of dynamic neural models in presence of noise. Conference Publications, 2003, 2003 (Special) : 656-663. doi: 10.3934/proc.2003.2003.656

[4]

Joseph D. Skufca, Erik M. Bollt. Communication and Synchronization in Disconnected Networks with Dynamic Topology: Moving Neighborhood Networks. Mathematical Biosciences & Engineering, 2004, 1 (2) : 347-359. doi: 10.3934/mbe.2004.1.347

[5]

Cristina De Ambrosi, Annalisa Barla, Lorenzo Tortolina, Nicoletta Castagnino, Raffaele Pesenti, Alessandro Verri, Alberto Ballestrero, Franco Patrone, Silvio Parodi. Parameter space exploration within dynamic simulations of signaling networks. Mathematical Biosciences & Engineering, 2013, 10 (1) : 103-120. doi: 10.3934/mbe.2013.10.103

[6]

Ying Sue Huang. Resynchronization of delayed neural networks. Discrete & Continuous Dynamical Systems - A, 2001, 7 (2) : 397-401. doi: 10.3934/dcds.2001.7.397

[7]

James Walsh. Diffusive heat transport in Budyko's energy balance climate model with a dynamic ice line. Discrete & Continuous Dynamical Systems - B, 2017, 22 (7) : 2687-2715. doi: 10.3934/dcdsb.2017131

[8]

Martin Gugat, Falk M. Hante, Markus Hirsch-Dick, Günter Leugering. Stationary states in gas networks. Networks & Heterogeneous Media, 2015, 10 (2) : 295-320. doi: 10.3934/nhm.2015.10.295

[9]

Jin Soo Park, Kyung Jae Kim, Yun Han Bae, Bong Dae Choi. Admission control by dynamic bandwidth reservation using road layout and bidirectional navigator in wireless multimedia networks. Numerical Algebra, Control & Optimization, 2011, 1 (4) : 627-638. doi: 10.3934/naco.2011.1.627

[10]

Hyeon Je Cho, Ganguk Hwang. Optimal design for dynamic spectrum access in cognitive radio networks under Rayleigh fading. Journal of Industrial & Management Optimization, 2012, 8 (4) : 821-840. doi: 10.3934/jimo.2012.8.821

[11]

Tatyana S. Turova. Structural phase transitions in neural networks. Mathematical Biosciences & Engineering, 2014, 11 (1) : 139-148. doi: 10.3934/mbe.2014.11.139

[12]

Tone Arnold, Myrna Wooders. Dynamic club formation with coordination. Journal of Dynamics & Games, 2015, 2 (3&4) : 341-361. doi: 10.3934/jdg.2015010

[13]

Ricai Luo, Honglei Xu, Wu-Sheng Wang, Jie Sun, Wei Xu. A weak condition for global stability of delayed neural networks. Journal of Industrial & Management Optimization, 2016, 12 (2) : 505-514. doi: 10.3934/jimo.2016.12.505

[14]

Benedetta Lisena. Average criteria for periodic neural networks with delay. Discrete & Continuous Dynamical Systems - B, 2014, 19 (3) : 761-773. doi: 10.3934/dcdsb.2014.19.761

[15]

Larry Turyn. Cellular neural networks: asymmetric templates and spatial chaos. Conference Publications, 2003, 2003 (Special) : 864-871. doi: 10.3934/proc.2003.2003.864

[16]

Andrzej Nowakowski, Jan Sokolowski. On dual dynamic programming in shape control. Communications on Pure & Applied Analysis, 2012, 11 (6) : 2473-2485. doi: 10.3934/cpaa.2012.11.2473

[17]

Jonathan H. Tu, Clarence W. Rowley, Dirk M. Luchtenburg, Steven L. Brunton, J. Nathan Kutz. On dynamic mode decomposition: Theory and applications. Journal of Computational Dynamics, 2014, 1 (2) : 391-421. doi: 10.3934/jcd.2014.1.391

[18]

Steven L. Brunton, Joshua L. Proctor, Jonathan H. Tu, J. Nathan Kutz. Compressed sensing and dynamic mode decomposition. Journal of Computational Dynamics, 2015, 2 (2) : 165-191. doi: 10.3934/jcd.2015002

[19]

Tao Li, Suresh P. Sethi. A review of dynamic Stackelberg game models. Discrete & Continuous Dynamical Systems - B, 2017, 22 (1) : 125-159. doi: 10.3934/dcdsb.2017007

[20]

Tai-Ping Liu, Shih-Hsien Yu. Hyperbolic conservation laws and dynamic systems. Discrete & Continuous Dynamical Systems - A, 2000, 6 (1) : 143-145. doi: 10.3934/dcds.2000.6.143

2018 Impact Factor: 1.313

Metrics

  • PDF downloads (5)
  • HTML views (0)
  • Cited by (0)

Other articles
by authors

[Back to Top]