• Previous Article
    Feynman path formula for the time fractional Schrödinger equation
  • DCDS-S Home
  • This Issue
  • Next Article
    An application of interpolation inequalities between the deviation of curvature and the isoperimetric ratio to the length-preserving flow
doi: 10.3934/dcdss.2020374

A simple digital spiking neural network: Synchronization and spike-train approximation

Department of Electrical and Electronic Engineering, Faculty of Science and Engineering, Hosei University, Koganei, Tokyo, 184-8584, Japan

* Corresponding author: Toshimichi Saito

Received  October 2019 Revised  March 2020 Published  May 2020

Fund Project: This work is supported in part by JSPS KAKENHI 18K11480

This paper studies synchronization phenomena of spike-trains and approximation of target spike-trains in a simple network of digital spiking neurons. Repeating integrate-and-fire behavior between a periodic base signal and constant firing threshold, the neurons can generate various spike-trains. Connecting multiple neurons by cross-firing with delay, the network is constructed. The network can exhibit multi-phase synchronization of various spike-trains. Stability of the synchronization phenomena can be guaranteed theoretically. Applying a simple winner-take-all switching, the network can approximate target spike-trains automatically. In order to evaluate the approximation performance, we present two metrics: spike-position error and spike missing rate. Using the metrics, approximation capability of the network is investigated in typical target signals. Presenting an FPGA based hardware prototype, typical synchronization phenomenon and spike-train approximation are confirmed experimentally.

Citation: Hiroaki Uchida, Yuya Oishi, Toshimichi Saito. A simple digital spiking neural network: Synchronization and spike-train approximation. Discrete & Continuous Dynamical Systems - S, doi: 10.3934/dcdss.2020374
References:
[1]

L. Appeltant, M. C. Soriano, G. Van der Sande, J. Danckaert, S. Massar, J. Dambre, B. Schrauwen, C. R. Mirasso and I. Fischer, Information processing using a single dynamical node as complex system, Nat. Commun., 2 (2011), Article number: 468. doi: 10.1038/ncomms1476.  Google Scholar

[2]

S. R. CampbellD. L. Wang and C. Jayaprakash, Synchrony and desynchrony in integrate-and-fire oscillators, Neural Comput., 11 (1999), 1595-1619.  doi: 10.1109/IJCNN.1998.685998.  Google Scholar

[3]

L. O. Chua, A Nonlinear Dynamics Perspective of Wolfram's New Kind of Science I, World Scientific, 2006.  Google Scholar

[4]

T. Iguchi, A. Hirata and H. Torikai, Theoretical and heuristic synthesis of digital spiking neurons for spike-pattern-division multiplexing, IEICE Trans. Fundam., E93-A (2010), 1486–1496. doi: 10.1587/transfun.E93.A.1486.  Google Scholar

[5]

A. Lasota and M. C. Mackey, Chaos, Fractals, and Noise, second edition, Springer-Verlag, 1994. doi: 10.1007/978-1-4612-4286-4.  Google Scholar

[6]

G. Lee and N. H. Farhat, The bifurcating neuron network 1, Neural networks, 14 (2001), 115-131.  doi: 10.1016/S0893-6080(00)00083-6.  Google Scholar

[7]

A. Lozano, M. Rodriguez and R. Barrio, Control strategies of 3-cell central pattern generator via global stimuli, Sci. Rep., 6 (2016), 23622. doi: 10.1038/srep23622.  Google Scholar

[8]

A. Matoba, N. Horimoto and T. Saito, Basic dynamics of the digital logistic map, IEICE Trans. Fundam., E96-A (2013), 1808–1811. doi: 10.1587/transfun.E96.A.1808.  Google Scholar

[9]

R. May, Simple mathematical models with very complicated dynamics, Nature, 261 (1976), 459-467.   Google Scholar

[10] E. Ott, Chaos in Dynamical Systems,, Second edition, Cambridge University Press, Cambridge, 2002.  doi: 10.1017/CBO9780511803260.  Google Scholar
[11]

R. Perez and L. Glass, Bistability, period doubling bifurcations and chaos in a periodically forced oscillator, Phys. Lett. A, 90 (1982), 441-443.  doi: 10.1016/0375-9601(82)90391-7.  Google Scholar

[12]

R. Sato and T. Saito, Stabilization of desired periodic orbits in dynamic binary neural networks, Neurocomputing, 248 (2017), 19-27.  doi: 10.1016/j.neucom.2016.10.084.  Google Scholar

[13]

M. Schüle and R. Stoop, A full computation-relevant topological dynamics classification of elementary cellular automata, Chaos, 22 (2012), 043143, 10pp. doi: 10.1063/1.4771662.  Google Scholar

[14]

G. TanakaT. YamaneJ. B. HérouxR. NakaneN. KanazawaS. TakedaH. NumataD. Nakano and A. Hirose, Recent advances in physical reservoir computing: A review, Neural Networks, 115 (2019), 100-123.  doi: 10.1016/j.neunet.2019.03.005.  Google Scholar

[15]

H. Torikai and T. Saito, Resonance phenomenon of interspike intervals from a spiking oscillator with two periodic inputs, IEEE Trans. Circuits Syst. Ⅰ, 48 (2001), 1198-1204. doi: 10.1109/81.788809.  Google Scholar

[16]

H. Torikai and T. Saito, Analysis of a quantized chaotic system, Int'l J. of Bifurcation and Chaos, 12 (2002), 1207-1218.  doi: 10.1142/S0218127402005054.  Google Scholar

[17]

H. Torikai and T. Saito, Synchronization phenomena in pulse-coupled networks driven by spike-train inputs, IEEE Trans. Neural Networks, 15 (2004), 337-347.  doi: 10.1109/TNN.2004.824403.  Google Scholar

[18]

H. TorikaiH. Hamanaka and T. Saito, Reconfigurable spiking neuron and its pulse-coupled networks: basic characteristics and potential applications, IEEE Trans. Circuits Syst. Ⅱ, 53 (2006), 734-738.  doi: 10.1109/TCSII.2006.876381.  Google Scholar

[19]

H. TorikaiA. Funew and T. Saito, Digital spiking neuron and its learning for approximation of various spike-trains, Neural Networks, 21 (2008), 140-149.  doi: 10.1016/j.neunet.2007.12.045.  Google Scholar

[20]

H. Uchida and T. Saito, Multi-phase synchronization phenomena in a ring-coupled system of digital spiking neurons, IEICE Trans. Fundam., E102-A (2019), 235-241.  doi: 10.1587/transfun.E102.A.235.  Google Scholar

[21]

W. WadaJ. Kuroiwa and S. Nara, Completely reproducible description of digital sound data with cellular automata, Phys. Lett. A, 306 (2002), 110-115.  doi: 10.1016/S0375-9601(01)00610-7.  Google Scholar

[22]

D. L. Wang and D. Terman, Locally excitatory globally inhibitory oscillator networks, IEEE Trans. Neural Networks, 6 (1995), 283-286.  doi: 10.4249/scholarpedia.1620.  Google Scholar

[23]

S. Wolfram, A New Kind of Science, Wolfram Media, 2002.  Google Scholar

[24]

H. Yamaoka and T. Saito, Steady-versus-transient plot for analysis of digital maps, IEICE Trans. Fundam., E99-A (2016), 1806-1812.  doi: 10.1587/transfun.E99.A.1806.  Google Scholar

show all references

References:
[1]

L. Appeltant, M. C. Soriano, G. Van der Sande, J. Danckaert, S. Massar, J. Dambre, B. Schrauwen, C. R. Mirasso and I. Fischer, Information processing using a single dynamical node as complex system, Nat. Commun., 2 (2011), Article number: 468. doi: 10.1038/ncomms1476.  Google Scholar

[2]

S. R. CampbellD. L. Wang and C. Jayaprakash, Synchrony and desynchrony in integrate-and-fire oscillators, Neural Comput., 11 (1999), 1595-1619.  doi: 10.1109/IJCNN.1998.685998.  Google Scholar

[3]

L. O. Chua, A Nonlinear Dynamics Perspective of Wolfram's New Kind of Science I, World Scientific, 2006.  Google Scholar

[4]

T. Iguchi, A. Hirata and H. Torikai, Theoretical and heuristic synthesis of digital spiking neurons for spike-pattern-division multiplexing, IEICE Trans. Fundam., E93-A (2010), 1486–1496. doi: 10.1587/transfun.E93.A.1486.  Google Scholar

[5]

A. Lasota and M. C. Mackey, Chaos, Fractals, and Noise, second edition, Springer-Verlag, 1994. doi: 10.1007/978-1-4612-4286-4.  Google Scholar

[6]

G. Lee and N. H. Farhat, The bifurcating neuron network 1, Neural networks, 14 (2001), 115-131.  doi: 10.1016/S0893-6080(00)00083-6.  Google Scholar

[7]

A. Lozano, M. Rodriguez and R. Barrio, Control strategies of 3-cell central pattern generator via global stimuli, Sci. Rep., 6 (2016), 23622. doi: 10.1038/srep23622.  Google Scholar

[8]

A. Matoba, N. Horimoto and T. Saito, Basic dynamics of the digital logistic map, IEICE Trans. Fundam., E96-A (2013), 1808–1811. doi: 10.1587/transfun.E96.A.1808.  Google Scholar

[9]

R. May, Simple mathematical models with very complicated dynamics, Nature, 261 (1976), 459-467.   Google Scholar

[10] E. Ott, Chaos in Dynamical Systems,, Second edition, Cambridge University Press, Cambridge, 2002.  doi: 10.1017/CBO9780511803260.  Google Scholar
[11]

R. Perez and L. Glass, Bistability, period doubling bifurcations and chaos in a periodically forced oscillator, Phys. Lett. A, 90 (1982), 441-443.  doi: 10.1016/0375-9601(82)90391-7.  Google Scholar

[12]

R. Sato and T. Saito, Stabilization of desired periodic orbits in dynamic binary neural networks, Neurocomputing, 248 (2017), 19-27.  doi: 10.1016/j.neucom.2016.10.084.  Google Scholar

[13]

M. Schüle and R. Stoop, A full computation-relevant topological dynamics classification of elementary cellular automata, Chaos, 22 (2012), 043143, 10pp. doi: 10.1063/1.4771662.  Google Scholar

[14]

G. TanakaT. YamaneJ. B. HérouxR. NakaneN. KanazawaS. TakedaH. NumataD. Nakano and A. Hirose, Recent advances in physical reservoir computing: A review, Neural Networks, 115 (2019), 100-123.  doi: 10.1016/j.neunet.2019.03.005.  Google Scholar

[15]

H. Torikai and T. Saito, Resonance phenomenon of interspike intervals from a spiking oscillator with two periodic inputs, IEEE Trans. Circuits Syst. Ⅰ, 48 (2001), 1198-1204. doi: 10.1109/81.788809.  Google Scholar

[16]

H. Torikai and T. Saito, Analysis of a quantized chaotic system, Int'l J. of Bifurcation and Chaos, 12 (2002), 1207-1218.  doi: 10.1142/S0218127402005054.  Google Scholar

[17]

H. Torikai and T. Saito, Synchronization phenomena in pulse-coupled networks driven by spike-train inputs, IEEE Trans. Neural Networks, 15 (2004), 337-347.  doi: 10.1109/TNN.2004.824403.  Google Scholar

[18]

H. TorikaiH. Hamanaka and T. Saito, Reconfigurable spiking neuron and its pulse-coupled networks: basic characteristics and potential applications, IEEE Trans. Circuits Syst. Ⅱ, 53 (2006), 734-738.  doi: 10.1109/TCSII.2006.876381.  Google Scholar

[19]

H. TorikaiA. Funew and T. Saito, Digital spiking neuron and its learning for approximation of various spike-trains, Neural Networks, 21 (2008), 140-149.  doi: 10.1016/j.neunet.2007.12.045.  Google Scholar

[20]

H. Uchida and T. Saito, Multi-phase synchronization phenomena in a ring-coupled system of digital spiking neurons, IEICE Trans. Fundam., E102-A (2019), 235-241.  doi: 10.1587/transfun.E102.A.235.  Google Scholar

[21]

W. WadaJ. Kuroiwa and S. Nara, Completely reproducible description of digital sound data with cellular automata, Phys. Lett. A, 306 (2002), 110-115.  doi: 10.1016/S0375-9601(01)00610-7.  Google Scholar

[22]

D. L. Wang and D. Terman, Locally excitatory globally inhibitory oscillator networks, IEEE Trans. Neural Networks, 6 (1995), 283-286.  doi: 10.4249/scholarpedia.1620.  Google Scholar

[23]

S. Wolfram, A New Kind of Science, Wolfram Media, 2002.  Google Scholar

[24]

H. Yamaoka and T. Saito, Steady-versus-transient plot for analysis of digital maps, IEICE Trans. Fundam., E99-A (2016), 1806-1812.  doi: 10.1587/transfun.E99.A.1806.  Google Scholar

Figure 1.  DSNs and Dmaps for $ T_p = 18 $. (a) A PST with period $ 6T_p $ and corresponding PEO with period 6. $ \delta = (16,17,18,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15) $. (b) A direct stable PST with period $ 6T_p $ and corresponding PEO with period 6. $ \delta = (17,17,17,2,2,2,5,5,5,8,8,8,11,11,11,14,14,14) $
Figure 2.  Digital spiking neural network (DSNN) in a ladder form
Figure 3.  Cross-firing
Figure 4.  6-phase synchronization of PSTs with period 6 in DSNN
Figure 5.  Stability of master-slave synchronization
Figure 6.  WTA switching for $ M = 6 $. (a) WTA switching. (b) Target spike-train $ y_t $ and approximated spike-train $ y_a $
Figure 7.  Dmap for each DSN in the DSNN for $ T_p = 36 $ and $ M = 12 $. $ \delta = (35,35,35,2,2,2,5,5,5,8,8,8,11,11,11,14, $ $ 14,14,17,17,17,20,20,20,23,23,23,26,26,26,29,29,29,32,32,32) $. The PEO with period 12 corresponds to the PST with period $ 12T_p $
Figure 8.  Target PST $ y_t $, approximated PST $ y_a $, and 12-phase synchronization of of PSTs $ y_1 \sim y_{12} $ with period $ 12 $. Verilog simulation of DSNN for $ M = 12 $
Figure 9.  Discrete exponential distribution for target PSTs
Figure 10.  Selection of activated DSNs from 6DSNs. (a) Selection of spike missing: the 1st, the 2nd, and the 3rd DSNs. (b) Selection of no spike missing: the 1st, the 3rd, and the 5th DSNs
Figure 11.  Circuit design of DSNN with the WTA
Figure 12.  Target spike-train, approximated spike-train and multi-phase synchronization in an FPGA board
Table 1.  Case 1 results
#DSN AVG $ \varepsilon_p $ SD $ \varepsilon_p $ SMR
3 6.60 1.35 30.3
4 5.17 0.74 17, 5
5 4.19 0.43 10.1
6 3.56 0.31 5.10
7 3.02 0.30 0.82
8 2.63 0.24 0
9 2.43 0.19 0
10 2.25 0.18 0
11 2.11 0.14 0
12 2.02 0.10 0
#DSN AVG $ \varepsilon_p $ SD $ \varepsilon_p $ SMR
3 6.60 1.35 30.3
4 5.17 0.74 17, 5
5 4.19 0.43 10.1
6 3.56 0.31 5.10
7 3.02 0.30 0.82
8 2.63 0.24 0
9 2.43 0.19 0
10 2.25 0.18 0
11 2.11 0.14 0
12 2.02 0.10 0
Table 2.  Case 2 results
#DSN AVG $\varepsilon_p$ SD $\varepsilon_p$ SMR
3 8.91 1.84 44.5
4 6.22 0.90 21.7
5 5.12 0.74 10.8
6 4.18 0.46 2.78
7 3.62 0.44 1.09
8 3.22 0.30 0.27
9 2.91 0.29 0
10 2.59 0.22 0
11 2.28 0.14 0
12 2.03 0.11 0
#DSN AVG $\varepsilon_p$ SD $\varepsilon_p$ SMR
3 8.91 1.84 44.5
4 6.22 0.90 21.7
5 5.12 0.74 10.8
6 4.18 0.46 2.78
7 3.62 0.44 1.09
8 3.22 0.30 0.27
9 2.91 0.29 0
10 2.59 0.22 0
11 2.28 0.14 0
12 2.03 0.11 0
[1]

Raoul-Martin Memmesheimer, Marc Timme. Stable and unstable periodic orbits in complex networks of spiking neurons with delays. Discrete & Continuous Dynamical Systems - A, 2010, 28 (4) : 1555-1588. doi: 10.3934/dcds.2010.28.1555

[2]

Pierre Guiraud, Etienne Tanré. Stability of synchronization under stochastic perturbations in leaky integrate and fire neural networks of finite size. Discrete & Continuous Dynamical Systems - B, 2019, 24 (9) : 5183-5201. doi: 10.3934/dcdsb.2019056

[3]

Karim El Laithy, Martin Bogdan. Synaptic energy drives the information processing mechanisms in spiking neural networks. Mathematical Biosciences & Engineering, 2014, 11 (2) : 233-256. doi: 10.3934/mbe.2014.11.233

[4]

Tingting Su, Xinsong Yang. Finite-time synchronization of competitive neural networks with mixed delays. Discrete & Continuous Dynamical Systems - B, 2016, 21 (10) : 3655-3667. doi: 10.3934/dcdsb.2016115

[5]

Ricai Luo, Honglei Xu, Wu-Sheng Wang, Jie Sun, Wei Xu. A weak condition for global stability of delayed neural networks. Journal of Industrial & Management Optimization, 2016, 12 (2) : 505-514. doi: 10.3934/jimo.2016.12.505

[6]

Ruoxia Li, Huaiqin Wu, Xiaowei Zhang, Rong Yao. Adaptive projective synchronization of memristive neural networks with time-varying delays and stochastic perturbation. Mathematical Control & Related Fields, 2015, 5 (4) : 827-844. doi: 10.3934/mcrf.2015.5.827

[7]

Jianping Zhou, Yamin Liu, Ju H. Park, Qingkai Kong, Zhen Wang. Fault-tolerant anti-synchronization control for chaotic switched neural networks with time delay and reaction diffusion. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020357

[8]

Hiroyuki Torikai. Basic spike-train properties of a digital spiking neuron. Discrete & Continuous Dynamical Systems - B, 2008, 9 (1) : 183-198. doi: 10.3934/dcdsb.2008.9.183

[9]

Per Danzl, Ali Nabi, Jeff Moehlis. Charge-balanced spike timing control for phase models of spiking neurons. Discrete & Continuous Dynamical Systems - A, 2010, 28 (4) : 1413-1435. doi: 10.3934/dcds.2010.28.1413

[10]

Roberta Sirovich, Laura Sacerdote, Alessandro E. P. Villa. Cooperative behavior in a jump diffusion model for a simple network of spiking neurons. Mathematical Biosciences & Engineering, 2014, 11 (2) : 385-401. doi: 10.3934/mbe.2014.11.385

[11]

Sylvia Novo, Rafael Obaya, Ana M. Sanz. Exponential stability in non-autonomous delayed equations with applications to neural networks. Discrete & Continuous Dynamical Systems - A, 2007, 18 (2&3) : 517-536. doi: 10.3934/dcds.2007.18.517

[12]

Ivanka Stamova, Gani Stamov. On the stability of sets for reaction–diffusion Cohen–Grossberg delayed neural networks. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020370

[13]

Quan Hai, Shutang Liu. Mean-square delay-distribution-dependent exponential synchronization of chaotic neural networks with mixed random time-varying delays and restricted disturbances. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020221

[14]

Ying Sue Huang. Resynchronization of delayed neural networks. Discrete & Continuous Dynamical Systems - A, 2001, 7 (2) : 397-401. doi: 10.3934/dcds.2001.7.397

[15]

Ozlem Faydasicok. Further stability analysis of neutral-type Cohen-Grossberg neural networks with multiple delays. Discrete & Continuous Dynamical Systems - S, 2020  doi: 10.3934/dcdss.2020359

[16]

Zengyun Wang, Jinde Cao, Zuowei Cai, Lihong Huang. Finite-time stability of impulsive differential inclusion: Applications to discontinuous impulsive neural networks. Discrete & Continuous Dynamical Systems - B, 2020  doi: 10.3934/dcdsb.2020200

[17]

Sujit Nair, Naomi Ehrich Leonard. Stable synchronization of rigid body networks. Networks & Heterogeneous Media, 2007, 2 (4) : 597-626. doi: 10.3934/nhm.2007.2.597

[18]

Inmaculada Leyva, Irene Sendiña-Nadal, Stefano Boccaletti. Explosive synchronization in mono and multilayer networks. Discrete & Continuous Dynamical Systems - B, 2018, 23 (5) : 1931-1944. doi: 10.3934/dcdsb.2018189

[19]

Daniel M. N. Maia, Elbert E. N. Macau, Tiago Pereira, Serhiy Yanchuk. Synchronization in networks with strongly delayed couplings. Discrete & Continuous Dynamical Systems - B, 2018, 23 (8) : 3461-3482. doi: 10.3934/dcdsb.2018234

[20]

Tatyana S. Turova. Structural phase transitions in neural networks. Mathematical Biosciences & Engineering, 2014, 11 (1) : 139-148. doi: 10.3934/mbe.2014.11.139

2019 Impact Factor: 1.233

Metrics

  • PDF downloads (22)
  • HTML views (51)
  • Cited by (0)

Other articles
by authors

[Back to Top]