Brain-Inspired Computation based on Spiking Neural Networks
Download as PPTX, PDF0 likes147 views
The document discusses brain-inspired computation utilizing spiking neural networks (SNNs), emphasizing the human brain's sophisticated learning and memory mechanisms. It details various architectures and models for SNNs, including unsupervised and supervised learning methods, and their applications across diverse fields such as biomedical signals and environmental prediction. Additionally, it highlights the advantages and limitations of SNNs and outlines current advancements in neuromorphic hardware systems.
![Methods for supervised learning in SNN
Rank order (RO) learning rule (Thorpe et al, 1998)
nkasabov@aut.ac.nz
)
(
order j
ji m
w
else
fired
if
0
)
(
)
(
|
)
(
order
t
j
f
j
j
i
ji
i m
w
t
u
PSP max (T) = SUM [(m order (j(t)) wj,i(t)], for j=1,2.., k; t=1,2,...,T;
PSPTh=C. PSPmax (T)
- Earlier coming spikes are more important (carry more information)
- Early spiking can be achieved, depending on the parameter C.](https://image.slidesharecdn.com/brain-inspiredcomputaionbasedonsnn-220602110526-7ce00b18/85/Brain-Inspired-Computation-based-on-Spiking-Neural-Networks-7-320.jpg)
Brain-Inspired Computation based on Spiking Neural Networks
- 1. Brain-Inspired Computation based on Spiking Neural Networks Nikola Kasabov FIEEE, FRSNZ, FINNS, FIITP NZ Professor of Knowledge Engineering, KEDRI, Auckland University of Technology George Moore Chair of Data Analytics, Ulster University, UK Honorary Professor Teesside University UK and ABI the University of Auckland Doctor Honoris Causa Obuda University Budapest [email protected] N. Kasabov, Time-Space, Spiking Neural Networks and Brain-Inspired Artificial Intelligence, Springer (2019)
- 2. [email protected] The brain (80bln neurons, 100 trillions of connections, 200 mln years of evolution) is the ultimate learning machine Three, mutually interacting, memory types: - short term (membrane potential); - long term (synaptic weights); - genetic (genes in the nuclei). Temporal data at different time scales: - Nanoseconds: quantum processes; - Milliseconds: spiking activity; - Minutes: gene expressions; - Hours: learning in synapses; - Many years: evolution of genes. Spatially evolved structure with spatially allocated functions Learning and knowledge is represented as deep spatio- temporal patterns. The human brain as an ultimate inspiration The human brain, the most sophisticated product of the evolution, is a deep learning machine for information processing based on learned knowledge
- 3. Learned knowledge is represented as trajectories of connected neuronal clusters Deep serial processing of visual stimuli in humans for image classification and action. Location of cortical areas: V1 = primary visual cortex, V2 = secondary visual cortex, V4 = quartiary visual cortex, IT = inferotemporal cortex, PFC = prefrontal cortex, PMC = premotor cortex, MC = motor cortex. (from L.Benuskova, N.Kasabov, Computational neurogenetic modelling, Springer, 2007) [email protected] Thalamus Pr ef ron ta l co rte x Frontal cortex Eye H i g h e r - o r d e r p a r i e t a l v i s u a l a r e a s Primary visual cortex Cerebellum IT 150 ms 250 ms V4 V2 V1 MC PMC PFC
- 4. Spiking Neural Networks Models of a spiking neurons – Hodgkin- Huxley – Spike response model – Integrate-and-fire ----------------> – Leaky integrator – Izhikevich model – Probabilistic and neurogenetic models [email protected]
- 5. Methods for unsupervised learning in SNN Spike-Time Dependent Plasticity (STDP) (Abbott and Nelson, 2000). • Hebbian form of plasticity in the form of long-term potentiation (LTP) and depression (LTD) • Effect of synapses are strengthened or weakened based on the timing of pre-synaptic spikes and post-synaptic action potential. • Through STDP connected neurons learn consecutive temporal associations from data. • Variations of the STDP Pre-synaptic activity that precedes post- synaptic firing can induce LTP, reversing this temporal order causes LTD: ∆t=tpre -tpost [email protected]
- 6. Spike encoding methods A spike is generated only if a change in the input data occurs beyond a threshold Silicon Retina (Tobi Delbruck, INI, ETH/UZH, Zurich ), DVS128: Retinotopic Silicon Cochlea ( Shih-Chii Liu, INI, ETH/UZH, Zurich): Tonotopic [email protected] www.kedri.aut.ac.nz/neucube Threshold-based encoding – retinotopic Tonotopic organization of the cochlea for spectro-temporal data mapping for spatio-temporal data https://sites.google.com/site/jayanthinyswebite
- 7. Methods for supervised learning in SNN Rank order (RO) learning rule (Thorpe et al, 1998) [email protected] ) ( order j ji m w else fired if 0 ) ( ) ( | ) ( order t j f j j i ji i m w t u PSP max (T) = SUM [(m order (j(t)) wj,i(t)], for j=1,2.., k; t=1,2,...,T; PSPTh=C. PSPmax (T) - Earlier coming spikes are more important (carry more information) - Early spiking can be achieved, depending on the parameter C.
- 8. Dynamic Evolving SNN (deSNN) (Kasabov, N., Dhoble, K., Nuntalid, N., G. Indiveri, Dynamic Evolving Spiking Neural Networks for On- line Spatio- and Spectro-Temporal Pattern Recognition, Neural Networks, v.41, 188-201, 2013) Combine: (a) RO learning for weight initialisation based on the first spikes: (b) Learning further input spikes at a synapse through a drift – positive and negative. wj,i(t) = ej(t)・Drift - A new output neuron may be added to a respective output repository for every new -input pattern. - Two types of output neuron activation: - deSNNm (spiking is based on the membrane potential) - deSNNs (spiking is based on synaptic similarity between the newly created output neuron and the existing ones) - Neurons may merge. ) ( order j ji m w [email protected]
- 9. Brain-inspired SNN (BI-SNN) Architectures. NeuCube [email protected] eSNN SNNcube 2 , 2 / , b a D b a e C p What is a BI-SNN architecture? - Input data is encoded into spatio-temporal events as spike trains; - A 3D SNN has spatially located neurons following a brain template, e.g Talairach, MNI etc. . - Inputs are mapped spatially (brain-like) into the SNN, a 3D structure organised as a brain template. - Unsupervised learning is spatio-temporal, adaptive and incremental resulting in evolved connectivity - The structure is self-organising - Supervised learning is evolving creating new output neurons - Allows for knowledge representation as spatio-temporal patterns, interpreted as rules, graphs, associations, ….
- 10. The NeuCube Architecture [email protected] Kasabov, N., NeuCube: A Spiking Neural Network Architecture for Mapping, Learning and Understanding of Spatio-Temporal Brain Data, Neural Networks, vol.52, 2014. E. Tu, N. Kasabov, J. Yang, Mapping Temporal Variables into the NeuCube Spiking Neural Network Architecture for Improved Pattern Recognition and Predictive Modelling, IEEE Trans. on Neural Networks and Learning Systems, 28 (6), 1305-1317,, 2017 DOI: 10.1109/TNNLS.2016.2536742, 2017.
- 11. Deep learning in NeuCube Spike Trains Entered to the SNNc Neuron Spiking Activity During the STDP Learning Creation of Neuron Connections During The Learning The More Spike Transmission, The More Connections Created [email protected] www.kedri.aut.ac.nz/neucube/
- 12. [email protected] Design and implementation of BI-SNN application systems A BI-SNN application system design procedure • Analysis of the type of data and possible solutions to the problem • SNN reservoir design according to a template (brain template or other) • Input data transformation into spike sequences; • Mapping input variables into spiking neurons • Deep unsupervised learning spatio-temporal spike sequences in a scalable 3D SNN reservoir; • Supervised learning and classification of data over time; • Dynamic parameter optimisation; • Model visualisation • Extracting deep knowledge from a trained SNN • Adaptation on new data in an on-line/ real time mode; • Extracting of modified knowledge • Implementation of a SNN model: von Neumann vs neuromorphic hardware systems
- 13. Some applications of BI-SNN • Audio/Visual signals – Speech, sound and music recognition – Moving object recognition – Odour recognition • Biomedical signals – EEG – fMRI; fMRI+ DTI: – EEG + MRI – EMG – ECG • Multisensory streaming data – Health risk event prediction from temporal climate data (stroke) – Hazardous environmental event prediction (e.g. risk of earthquakes in NZ; flooding in Malaysia; pollution in London area; extreme weather from satellite images) • Other [email protected] N. Kasabov, N. Scott, E.Tu, S. Marks, N.Sengupta, E.Capecci, M.Othman,M. Doborjeh, N.Murli, R.Hartono, J.Espinosa-Ramos, L.Zhou, F.Alvi, G.Wang, D.Taylor, V. Feigin,S. Gulyaev, M.Mahmoudh, Z-G.Hou, J.Yang, Design methodology and selected applications of evolving spatio- temporal data machines in the NeuCube neuromorphic framework, Neural Networks, v.78, 1-14, 2016 (best NN paper for 2016)
- 14. Using tonotopic, stereo mapping of sound and deep learning in NeuCube Mozart Bach Vivaldi Predicted 1 171 3 1 Predicted 2 9 176 1 Predicted 3 0 1 178 https://kedri.aut.ac.nz/R-and-D-Systems/audio-visual-data-processing-and-concept-formation#auditory
- 15. Two approaches for visual information processing [email protected]
- 16. BI-SNN for fast moving object recognition from video streaming data (Surveillance systems; Cybersecurity; Military applications; Autonomous vehicles) https://kedri.aut.ac.nz/R-and-D-Systems/fast-moving-object-recognition The spike event based frames are divided into 100 blocks, each block consists of 128×72 pixels, the mean filter has been applied to evaluate the pixel output.
- 17. EEG Recording fMRI Recording Step1: STBD measurement Step2: Encoding STBD Encoding into Spike Trains Step3: Variable Mapping into 3D SNNc Talairach Template fMRI Voxels Step4:STDP learning & Dynamic clustering Neuron Connections Evolving Neuronal Clusters Step5: Analysis of the connectivity of the trained 3D SNNc as dynamic spatio-temporal clusters in the STBD, related to brain processes Learning and representation of brain signals (EEG, fMRI, DTI, genetic,..) Methodology [email protected]
- 18. Modelling brain EEG signals while working with a VR/AR A prototype virtual environment of a hand attempting to grasp a glass controlled with EEG signals. A virtual environment to control a quadrotor using EEG signals. A virtual environment (3D) using Oculus rift DK2 to move in an environment using EEG.
- 19. Seismic data modelling for earthquake prediction N. Kasabov, N. Scott, E.Tu, S. Marks, N.Sengupta, E.Capecci, M.Othman,M. Doborjeh, N.Murli,R.Hartono, J.Espinosa-Ramos, L.Zhou, F.Alvi, G.Wang, D.Taylor, V. Feigin,S. Gulyaev, M.Mahmoudh, Z-G.Hou, J.Yang, Design methodology and selected applications of evolving spatio- temporal data machines in the NeuCube neuromorphic framework, Neural Networks, v.78, 1-14, 2016. http://dx.doi.org/10.1016/j.neunet.2015.09.011. [email protected] Measure NeuCube SVM MLP 1h ahead 91.36% 65% 60% 6h ahead 83% 53% 47% 12h ahead 75% 43% 46% Predicting risk for earthquakes, tsunami, land slides, floods – how early and how accurate?
- 20. Creating novel BI-AI [email protected] NI BI CI, Mathematics, Physics, Chemistry, Engineering New Data Technologies New BI-, and hybrid computational methods and systems • Modelling emergence of symbolic representation • Multimodal and multi- model SNN systems • Quantum-inspired computation: Spikes as q- bits - in a superposition of 1/0 • Real time event prediction systems • Embedded systems • Mental health evaluation systems • Neurological prosthetics • Brain-inspired SNN for quantum computation Social and ethical problems of AI: www.mindthegap.ai
- 21. SNN Implementations on hardware platforms From von Neuman Machines to Neuromorphic Platforms - The computer architecture of John von Neumann separates data and programmes (kept in the memory unit) from the computation (ALU); uses bits. First machine ABC by John Atanassov and Berry. - A Neuromorphic architecture integrates the data, the programme and the computation in a SNN structure, similar to how the brain works; uses spikes (bits at times). - A quantum computer uses q-bits (bits in a superposition) . A SNN application system can be implemented using either of: - von Neumann architecture; - Neuromorphic architecture; - Quantum computers [email protected] N. Sengupta et al, (2018), From von Neumann architecture and Atanasoffs ABC to Neuromorphic Computation and Kasabov’s NeuCube: Principles and Implementations, Chapter 1 in: Advances in Computational intelligence, Jotzov et al (eds) Springer 2018.
- 22. Neuromorphic hardware systems Carver Mead (1989): A hardware model of an IF neuron: The Axon-Hillock circuit. SpiNNaker (Furber, S., To Build a Brain, IEEE Spectrum, vol.49, Number 8, 39-41, 2012). INI Zurich SNN chips (Giacomo Indiveri) Silicon retina (the DVS) and silicon cochlea (ETH, Zurich, Toby Delbruck)) The IBM True North (D.Modha et al, 2016): 1mln neurons and 1 billion of synapses FPGA SNN realisations (M. McGinnity, Ulster and NTU) Neuromorphic chips developed in China High speed and low power consumption. [email protected] [email protected]
- 23. Discussions and future directions Advantages of BI-SNN: 1. Self-organised, evolvable structure (no fixed number of layers/neurons, etc.) 2. Event based (asynchronous), fast, incremental, potentially “life-long” learning. 3. Temporal (spatio-temporal) associations learned. 4. Interpretability, e.g. TSK representation 5. Low computational power 6. Fault tolerance Problems and limitations of BI-SNN • Sensitive to parameter values • Large number of parameters to be optimised • No rigid theory yet. [email protected] N.Kasabov, Time-Space, Spiking Neural Networks and Brain- Inspired Artificial Intelligence, Springer (2019)
- 25. [email protected] The NeuCube community is growing
Editor's Notes
- #20: 19