2014, 11(2): 233-256. doi: 10.3934/mbe.2014.11.233

Synaptic energy drives the information processing mechanisms in spiking neural networks

1. 

Faculty of Mathematics and Computer Science, Dept. of Computer Engineering, Leipzig University, Germany, Germany

Received  November 2012 Revised  March 2013 Published  October 2013

Flow of energy and free energy minimization underpins almost every aspect of naturally occurring physical mechanisms. Inspired by this fact this work establishes an energy-based framework that spans the multi-scale range of biological neural systems and integrates synaptic dynamic, synchronous spiking activity and neural states into one consistent working paradigm. Following a bottom-up approach, a hypothetical energy function is proposed for dynamic synaptic models based on the theoretical thermodynamic principles and the Hopfield networks. We show that a synapse exposes stable operating points in terms of its excitatory postsynaptic potential as a function of its synaptic strength. We postulate that synapses in a network operating at these stable points can drive this network to an internal state of synchronous firing. The presented analysis is related to the widely investigated temporal coherent activities (cell assemblies) over a certain range of time scales (binding-by-synchrony). This introduces a novel explanation of the observed (poly)synchronous activities within networks regarding the synaptic (coupling) functionality. On a network level the transitions from one firing scheme to the other express discrete sets of neural states. The neural states exist as long as the network sustains the internal synaptic energy.
Citation: Karim El Laithy, Martin Bogdan. Synaptic energy drives the information processing mechanisms in spiking neural networks. Mathematical Biosciences & Engineering, 2014, 11 (2) : 233-256. doi: 10.3934/mbe.2014.11.233
References:
[1]

A. B. Barrett, G. O. Billings, R. G. M. Morris and M. C. W. van Rossum, State based model of long-term potentiation and synaptic tagging and capture,, PLoS Comput. Biol., 5 (2009). doi: 10.1371/journal.pcbi.1000259.

[2]

K. El-laithy, Towards a Brain-Inspired Information Processing System: Modeling and Analysis of Synaptic Dynamics,, Ph.D thesis, (2011).

[3]

K. El-Laithy and M. Bogdan, Synchrony state generation in artificial neural networks with stochastic synapses,, in Artificial Neural Networks - ICANN 2009, (2009), 181.

[4]

K. El-Laithy and M. Bogdan, Predicting spike-timing of a thalamic neuron using a stochastic synaptic model,, in ESANN Proceedings, (2010), 357.

[5]

K. El-Laithy and M. Bogdan, A hypothetical free synaptic energy function and related states of synchrony,, in Artificial Neural Networks and Machine Learning - ICANN 2011, (2011), 40.

[6]

K. El-Laithy and M. Bogdan, On the capacity of transient internal states in liquid-state machines,, in Artificial Neural Networks and Machine Learning - ICANN 2011, (2011), 56.

[7]

K. El-laithy and M. Bogdan, Synchrony state generation: An approach using stochastic synapses,, J. of Artificial Intelligence and Soft Computing Research, 1 (2011), 17.

[8]

K. El-Laithy and M. Bogdan, Temporal finite-state machines: A novel framework for the general class of dynamic networks,, in ICONIP 2012, (2012), 425.

[9]

C. Eliasmith and C. H. Anderson, Neural Engineering. Computation, Representation, and Dynamics in Neurobiological Systems,, Computational Neuroscience, (2003).

[10]

A. K. Engel, P. Fries, P. Knig, M. Brecht and W. Singer, Temporal binding, binocular rivalry, and consciousness,, Consciousness and Cognition, 8 (1999), 128.

[11]

A. K. Engel and W. Singer, Temporal binding and the neural correlates of sensory awareness,, Trends in Cognitive Sciences, 5 (2001), 16.

[12]

K. Friston, The free-energy principle: A rough guide to the brain?,, Trends in Cognitive Sciences, 13 (2009), 293.

[13]

K. Friston, The free-energy principle: A unified brain theory, Nature Reviews Neuroscience, 11 (2010), 127.

[14]

H. M. Fuchs, Neural Networks with Dynamic Synapses,, Ph.D thesis, (1998).

[15]

J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities,, Proc. Nat. Acad. Sci. U.S.A., 79 (1982), 2554. doi: 10.1073/pnas.79.8.2554.

[16]

Y. Ikegaya, G. Aaron, R. Cossart, D. Aronov, I. Lampl, D. Ferster and R. Yuste, Synfire chains and cortical songs: Temporal modules of cortical activity,, Science, 304 (2004), 559. doi: 10.1126/science.1093173.

[17]

E. Izhikevich, Which model to use for cortical spiking neurons?, IEEE Transactions on Neural Networks, 15 (2004), 1063. doi: 10.1109/TNN.2004.832719.

[18]

E. M. Izhikevich, Polychronization: Computation with spikes,, Neural Comput., 18 (2006), 245. doi: 10.1162/089976606775093882.

[19]

A. Levina, J. M. Herrmann and T. Geisel, Phase transitions towards criticality in a neural system with adaptive interactions,, Physical Review Letters, 102 (2009). doi: 10.1103/PhysRevLett.102.118110.

[20]

J. M. Montgomery and D. V. Madison, State-dependent heterogeneity in synaptic depression between pyramidal cell pairs,, Neuron, 33 (2002), 765. doi: 10.1016/S0896-6273(02)00606-2.

[21]

J. M. Montgomery and D. V. Madison, Discrete synaptic states define a major mechanism of synapse plasticity,, Trends in Neurosciences, 27 (2004), 744. doi: 10.1016/j.tins.2004.10.006.

[22]

A. Morrison, M. Diesmann and W. Gerstner, Phenomenological models of synaptic plasticity based on spike timing,, Biol. Cybernet., 98 (2008), 459. doi: 10.1007/s00422-008-0233-1.

[23]

P. S. Neelakanta and D. DeGroff, Neural Network Modeling: Statistical Mechanics and Cybernetics Perspectives,, CRC Press, (1994).

[24]

A. Revonsuo and J. Newman, Binding and consciousness,, Consciousness and Cognition, 8 (1999), 123. doi: 10.1006/ccog.1999.0393.

[25]

C. Sarasola, A. d'Anjou, F. J. Torrealdea and M. Graña, Minimization of the energy flow in the synchronization of nonidentical chaotic systems,, Phys. Rev. E, 72 (2005). doi: 10.1103/PhysRevE.72.026223.

[26]

A. P. Shon and R. P. N. Rao, Temporal Sequence Learning with Dynamic Synapses,, Technical report, (2003).

[27]

W. Singer, Understanding the brain,, EMBO Reports, 8 (2007). doi: 10.1038/sj.embor.7400994.

[28]

S. Stringer, E. Rolls and T. Trappenberg, Self-organizing continuous attractor network models of hippocampal spatial view cells,, Neurobiology of Learning and Memory, 83 (2005), 79. doi: 10.1016/j.nlm.2004.08.003.

[29]

F. J. Torrealdea, A. d'Anjou, M. Graña and C. Sarasola, Energy aspects of the synchronization of model neurons,, Physical Review E, 74 (2006). doi: 10.1103/PhysRevE.74.011905.

[30]

C. von der Malsburg, The what and why of binding: The modeler's perspective,, Neuron, 24 (1999), 95.

show all references

References:
[1]

A. B. Barrett, G. O. Billings, R. G. M. Morris and M. C. W. van Rossum, State based model of long-term potentiation and synaptic tagging and capture,, PLoS Comput. Biol., 5 (2009). doi: 10.1371/journal.pcbi.1000259.

[2]

K. El-laithy, Towards a Brain-Inspired Information Processing System: Modeling and Analysis of Synaptic Dynamics,, Ph.D thesis, (2011).

[3]

K. El-Laithy and M. Bogdan, Synchrony state generation in artificial neural networks with stochastic synapses,, in Artificial Neural Networks - ICANN 2009, (2009), 181.

[4]

K. El-Laithy and M. Bogdan, Predicting spike-timing of a thalamic neuron using a stochastic synaptic model,, in ESANN Proceedings, (2010), 357.

[5]

K. El-Laithy and M. Bogdan, A hypothetical free synaptic energy function and related states of synchrony,, in Artificial Neural Networks and Machine Learning - ICANN 2011, (2011), 40.

[6]

K. El-Laithy and M. Bogdan, On the capacity of transient internal states in liquid-state machines,, in Artificial Neural Networks and Machine Learning - ICANN 2011, (2011), 56.

[7]

K. El-laithy and M. Bogdan, Synchrony state generation: An approach using stochastic synapses,, J. of Artificial Intelligence and Soft Computing Research, 1 (2011), 17.

[8]

K. El-Laithy and M. Bogdan, Temporal finite-state machines: A novel framework for the general class of dynamic networks,, in ICONIP 2012, (2012), 425.

[9]

C. Eliasmith and C. H. Anderson, Neural Engineering. Computation, Representation, and Dynamics in Neurobiological Systems,, Computational Neuroscience, (2003).

[10]

A. K. Engel, P. Fries, P. Knig, M. Brecht and W. Singer, Temporal binding, binocular rivalry, and consciousness,, Consciousness and Cognition, 8 (1999), 128.

[11]

A. K. Engel and W. Singer, Temporal binding and the neural correlates of sensory awareness,, Trends in Cognitive Sciences, 5 (2001), 16.

[12]

K. Friston, The free-energy principle: A rough guide to the brain?,, Trends in Cognitive Sciences, 13 (2009), 293.

[13]

K. Friston, The free-energy principle: A unified brain theory, Nature Reviews Neuroscience, 11 (2010), 127.

[14]

H. M. Fuchs, Neural Networks with Dynamic Synapses,, Ph.D thesis, (1998).

[15]

J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities,, Proc. Nat. Acad. Sci. U.S.A., 79 (1982), 2554. doi: 10.1073/pnas.79.8.2554.

[16]

Y. Ikegaya, G. Aaron, R. Cossart, D. Aronov, I. Lampl, D. Ferster and R. Yuste, Synfire chains and cortical songs: Temporal modules of cortical activity,, Science, 304 (2004), 559. doi: 10.1126/science.1093173.

[17]

E. Izhikevich, Which model to use for cortical spiking neurons?, IEEE Transactions on Neural Networks, 15 (2004), 1063. doi: 10.1109/TNN.2004.832719.

[18]

E. M. Izhikevich, Polychronization: Computation with spikes,, Neural Comput., 18 (2006), 245. doi: 10.1162/089976606775093882.

[19]

A. Levina, J. M. Herrmann and T. Geisel, Phase transitions towards criticality in a neural system with adaptive interactions,, Physical Review Letters, 102 (2009). doi: 10.1103/PhysRevLett.102.118110.

[20]

J. M. Montgomery and D. V. Madison, State-dependent heterogeneity in synaptic depression between pyramidal cell pairs,, Neuron, 33 (2002), 765. doi: 10.1016/S0896-6273(02)00606-2.

[21]

J. M. Montgomery and D. V. Madison, Discrete synaptic states define a major mechanism of synapse plasticity,, Trends in Neurosciences, 27 (2004), 744. doi: 10.1016/j.tins.2004.10.006.

[22]

A. Morrison, M. Diesmann and W. Gerstner, Phenomenological models of synaptic plasticity based on spike timing,, Biol. Cybernet., 98 (2008), 459. doi: 10.1007/s00422-008-0233-1.

[23]

P. S. Neelakanta and D. DeGroff, Neural Network Modeling: Statistical Mechanics and Cybernetics Perspectives,, CRC Press, (1994).

[24]

A. Revonsuo and J. Newman, Binding and consciousness,, Consciousness and Cognition, 8 (1999), 123. doi: 10.1006/ccog.1999.0393.

[25]

C. Sarasola, A. d'Anjou, F. J. Torrealdea and M. Graña, Minimization of the energy flow in the synchronization of nonidentical chaotic systems,, Phys. Rev. E, 72 (2005). doi: 10.1103/PhysRevE.72.026223.

[26]

A. P. Shon and R. P. N. Rao, Temporal Sequence Learning with Dynamic Synapses,, Technical report, (2003).

[27]

W. Singer, Understanding the brain,, EMBO Reports, 8 (2007). doi: 10.1038/sj.embor.7400994.

[28]

S. Stringer, E. Rolls and T. Trappenberg, Self-organizing continuous attractor network models of hippocampal spatial view cells,, Neurobiology of Learning and Memory, 83 (2005), 79. doi: 10.1016/j.nlm.2004.08.003.

[29]

F. J. Torrealdea, A. d'Anjou, M. Graña and C. Sarasola, Energy aspects of the synchronization of model neurons,, Physical Review E, 74 (2006). doi: 10.1103/PhysRevE.74.011905.

[30]

C. von der Malsburg, The what and why of binding: The modeler's perspective,, Neuron, 24 (1999), 95.

[1]

Xilin Fu, Zhang Chen. New discrete analogue of neural networks with nonlinear amplification function and its periodic dynamic analysis. Conference Publications, 2007, 2007 (Special) : 391-398. doi: 10.3934/proc.2007.2007.391

[2]

Alessia Marigo, Benedetto Piccoli. A model for biological dynamic networks. Networks & Heterogeneous Media, 2011, 6 (4) : 647-663. doi: 10.3934/nhm.2011.6.647

[3]

K Najarian. On stochastic stability of dynamic neural models in presence of noise. Conference Publications, 2003, 2003 (Special) : 656-663. doi: 10.3934/proc.2003.2003.656

[4]

Joseph D. Skufca, Erik M. Bollt. Communication and Synchronization in Disconnected Networks with Dynamic Topology: Moving Neighborhood Networks. Mathematical Biosciences & Engineering, 2004, 1 (2) : 347-359. doi: 10.3934/mbe.2004.1.347

[5]

Cristina De Ambrosi, Annalisa Barla, Lorenzo Tortolina, Nicoletta Castagnino, Raffaele Pesenti, Alessandro Verri, Alberto Ballestrero, Franco Patrone, Silvio Parodi. Parameter space exploration within dynamic simulations of signaling networks. Mathematical Biosciences & Engineering, 2013, 10 (1) : 103-120. doi: 10.3934/mbe.2013.10.103

[6]

Ying Sue Huang. Resynchronization of delayed neural networks. Discrete & Continuous Dynamical Systems - A, 2001, 7 (2) : 397-401. doi: 10.3934/dcds.2001.7.397

[7]

James Walsh. Diffusive heat transport in Budyko's energy balance climate model with a dynamic ice line. Discrete & Continuous Dynamical Systems - B, 2017, 22 (7) : 2687-2715. doi: 10.3934/dcdsb.2017131

[8]

Jin Soo Park, Kyung Jae Kim, Yun Han Bae, Bong Dae Choi. Admission control by dynamic bandwidth reservation using road layout and bidirectional navigator in wireless multimedia networks. Numerical Algebra, Control & Optimization, 2011, 1 (4) : 627-638. doi: 10.3934/naco.2011.1.627

[9]

Hyeon Je Cho, Ganguk Hwang. Optimal design for dynamic spectrum access in cognitive radio networks under Rayleigh fading. Journal of Industrial & Management Optimization, 2012, 8 (4) : 821-840. doi: 10.3934/jimo.2012.8.821

[10]

Martin Gugat, Falk M. Hante, Markus Hirsch-Dick, Günter Leugering. Stationary states in gas networks. Networks & Heterogeneous Media, 2015, 10 (2) : 295-320. doi: 10.3934/nhm.2015.10.295

[11]

Tatyana S. Turova. Structural phase transitions in neural networks. Mathematical Biosciences & Engineering, 2014, 11 (1) : 139-148. doi: 10.3934/mbe.2014.11.139

[12]

Tone Arnold, Myrna Wooders. Dynamic club formation with coordination. Journal of Dynamics & Games, 2015, 2 (3&4) : 341-361. doi: 10.3934/jdg.2015010

[13]

Ricai Luo, Honglei Xu, Wu-Sheng Wang, Jie Sun, Wei Xu. A weak condition for global stability of delayed neural networks. Journal of Industrial & Management Optimization, 2016, 12 (2) : 505-514. doi: 10.3934/jimo.2016.12.505

[14]

Benedetta Lisena. Average criteria for periodic neural networks with delay. Discrete & Continuous Dynamical Systems - B, 2014, 19 (3) : 761-773. doi: 10.3934/dcdsb.2014.19.761

[15]

Larry Turyn. Cellular neural networks: asymmetric templates and spatial chaos. Conference Publications, 2003, 2003 (Special) : 864-871. doi: 10.3934/proc.2003.2003.864

[16]

Andrzej Nowakowski, Jan Sokolowski. On dual dynamic programming in shape control. Communications on Pure & Applied Analysis, 2012, 11 (6) : 2473-2485. doi: 10.3934/cpaa.2012.11.2473

[17]

Jonathan H. Tu, Clarence W. Rowley, Dirk M. Luchtenburg, Steven L. Brunton, J. Nathan Kutz. On dynamic mode decomposition: Theory and applications. Journal of Computational Dynamics, 2014, 1 (2) : 391-421. doi: 10.3934/jcd.2014.1.391

[18]

Steven L. Brunton, Joshua L. Proctor, Jonathan H. Tu, J. Nathan Kutz. Compressed sensing and dynamic mode decomposition. Journal of Computational Dynamics, 2015, 2 (2) : 165-191. doi: 10.3934/jcd.2015002

[19]

Tao Li, Suresh P. Sethi. A review of dynamic Stackelberg game models. Discrete & Continuous Dynamical Systems - B, 2017, 22 (1) : 125-159. doi: 10.3934/dcdsb.2017007

[20]

Tai-Ping Liu, Shih-Hsien Yu. Hyperbolic conservation laws and dynamic systems. Discrete & Continuous Dynamical Systems - A, 2000, 6 (1) : 143-145. doi: 10.3934/dcds.2000.6.143

2017 Impact Factor: 1.23

Metrics

  • PDF downloads (5)
  • HTML views (0)
  • Cited by (0)

Other articles
by authors

[Back to Top]