Spiking Neural Network Architecture Design and Performance Exploration towards the Design of a Scalable Neuro-Inspired System for Complex Cognition Applications.
Abstract
Research into artificial neural networks (ANNs) is inspired by how information is dynamically and massively processed by biological neurons.
Conventional ANNs research has received a wide range of applications including automation, but there are still problems of timing, power consumption, and massive parallelism.
Spiking neural networks (SNNs), being the third-generation of neural networks, has drawn attention from a greater number of researchers due to the timing concept, which defines its closeness to biological Spiking Neural Network (bio-SNN tested) functions.
Spike timing plays an important role in every spiking neuron and proves computationally more plausible than other conventional ANNs.
The real biological and distinct neuron timing and spike firing can be modelled artificially using neurodynamics and spike neuron models.
The spike timing dependent plasticity (STDP) learning rule also incorporates timing concepts and is suitable for training SNNs which describes general plasticity rules that depend on the actual timing of pre- and postsynaptic spikes.
This work presents a software implementation of an SNN based on the Leaky Integrate-and-Fire (LIF) neuron model and STDP learning algorithm.
Also, we present a novel hardware design and architecture of a lightweight neuro-processing core (NPC) to be implemented in a packet-switched based neuro-inspired system, named NASH.
Introduction
Background of Study
Artificial Neural Network is an attractive, competitive and colossal research area in artificial intelligence which is inspired by the incredible and powerful performance of the interconnected biological brain.
According to one of the first inventors of neurocomputing, Robert Hecht-Nielsen, Neural Networks is defined as ‘a computing system made up of some simple, highly interconnected processing elements, which process information by their dynamic state response to external inputs’.
The whole idea of biological neural networks of the brain gave birth to Artificial Neural Networks (ANNs), with scientists digging deep on how and the best way to mimic the brain functionalities using silicon chips.
It is based on the fact that the biological brain connection architecture can be mimicked with silicon and wires in place of living neurons and dendrites.
The human brain is a structure made of 100 billion cells named neurons which connect thousands of cells by axons (von Bartheld, Bahney & Herculano-Houzel, 2016).
Inputs from sensory organs and the external environment are accepted by dendrites which create electric impulses that rapidly travel within the neural networks. Messages are sent across from neurons to other neurons.
However, Deep Learning as an exciting research area in machine learning is concerned with developing different algorithms inspired by the structure and function of the brain.
Artificial Neural Networks have been developed to solve various computational problems, but earlier research never considered timing issues which are the hallmark of Spiking Neural Networks (SNNs).
References
Ahmed, F., Yusob, B., & Hamed, H. N. A. (2014). Computing with spiking neuron networks: A review. International Journal of Advances in Soft Computing and Its Applications, 6(1). https://doi.org/10.1007/978-3-540-92910-9_10
Ahmed, A. B., & Abdallah, A. B. (2012). LA-XYZ: Low latency, high throughput look-ahead routing algorithm for 3D network-on-chip (3D-NoC) architecture. In Proceedings – IEEE 6th International Symposium on Embedded Multicore SoCs, MCSoC 2012 (pp. 167– 174). https://doi.org/10.1109/MCSoC.2012.24
Ahmed, A. B., Abdallah, A. B., & Kuroda, K. (2010). Architecture and design of efficient 3D network-on-chip (3D NoC) for custom multicore SoC. In Proceedings – 2010 International Conference on Broadband, Wireless Computing Communication and Applications, BWCCA 2010 (pp. 67–73). https://doi.org/10.1109/BWCCA.2010.50
Akopyan, F., Sawada, J., Cassidy, A., Alvarez-Icaza, R., Arthur, J., Merolla, P., … Modha, D.(2015). TrueNorth: Design and tool flow of a 65 mW 1 million neuron programmable neurosynaptic chip. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 34(10), 1537–1557. https://doi.org/10.1109/TCAD.2015.2474396
Alnajjar, F., Bin Mohd Zin, I., & Murase, K. (2008). A Spiking Neural Network with dynamic memory for a real autonomous mobile robot in dynamic environment. Neural Networks, 2008. IJCNN 2008. (IEEE World Congress on Computational Intelligence). IEEE International Joint Conference on, 2207–2213.
Basegmez, E. (2014). The next generation Neural Networks : Deep learning and Spiking Neural Networks. Technische Universitat Munchen.