Modular Neural Tile Architecture for Compact Embedded Hardware Spiking Neural Network

Sandeep Pande, Fearghal Morgan, Seamus Cawley, Tom Bruintjes, Gerardus Johannes Maria Smit, Brian McGinley, Snaider Carrillo, Jim Harkin, Liam McDaid

    Research output: Contribution to journalArticleAcademicpeer-review

    22 Citations (Scopus)
    129 Downloads (Pure)

    Abstract

    Biologically-inspired packet switched network on chip (NoC) based hardware spiking neural network (SNN) architectures have been proposed as an embedded computing platform for classification, estimation and control applications. Storage of large synaptic connectivity (SNN topology) information in SNNs require large distributed on-chip memory, which poses serious challenges for compact hardware implementation of such architectures. Based on the structured neural organisation observed in human brain, a modular neural networks (MNN) design strategy partitions complex application tasks into smaller subtasks executing on distinct neural network modules, and integrates intermediate outputs in higher level functions. This paper proposes a hardware modular neural tile (MNT) architecture that reduces the SNN topology memory requirement of NoC-based hardware SNNs by using a combination of fixed and configurable synaptic connections. The proposed MNT contains a 16:16 fully-connected feed-forward SNN structure and integrates in a mesh topology NoC communication infrastructure. The SNN topology memory requirement is 50 % of the monolithic NoC-based hardware SNN implementation. The paper also presents a lookup table based SNN topology memory allocation technique, which further increases the memory utilisation efficiency. Overall the area requirement of the architecture is reduced by an average of 66 % for practical SNN application topologies. The paper presents micro-architecture details of the proposed MNT and digital neuron circuit. The proposed architecture has been validated on a Xilinx Virtex-6 FPGA and synthesised using 65 nm low-power CMOS technology. The evolvable capability of the proposed MNT and its suitability for executing subtasks within a MNN execution architecture is demonstrated by successfully evolving benchmark SNN application tasks representing classification and non-linear control functions. The paper addresses hardware modular SNN design and implementation challenges and contributes to the development of a compact hardware modular SNN architecture suitable for embedded applications
    Original languageUndefined
    Pages (from-to)131-153
    Number of pages23
    JournalNeural processing letters
    Volume38
    Issue number2
    DOIs
    Publication statusPublished - Oct 2013

    Keywords

    • EWI-22722
    • Modular neural networks (MNN)
    • Network on Chip (NoC)
    • IR-84278
    • Spiking Neural Networks (SNN)
    • Synaptic Connectivity
    • METIS-296178
    • EMBRACE

    Cite this

    Pande, Sandeep ; Morgan, Fearghal ; Cawley, Seamus ; Bruintjes, Tom ; Smit, Gerardus Johannes Maria ; McGinley, Brian ; Carrillo, Snaider ; Harkin, Jim ; McDaid, Liam. / Modular Neural Tile Architecture for Compact Embedded Hardware Spiking Neural Network. In: Neural processing letters. 2013 ; Vol. 38, No. 2. pp. 131-153.
    @article{94ec3a8245164caeb45b7cb7b5a1a5ef,
    title = "Modular Neural Tile Architecture for Compact Embedded Hardware Spiking Neural Network",
    abstract = "Biologically-inspired packet switched network on chip (NoC) based hardware spiking neural network (SNN) architectures have been proposed as an embedded computing platform for classification, estimation and control applications. Storage of large synaptic connectivity (SNN topology) information in SNNs require large distributed on-chip memory, which poses serious challenges for compact hardware implementation of such architectures. Based on the structured neural organisation observed in human brain, a modular neural networks (MNN) design strategy partitions complex application tasks into smaller subtasks executing on distinct neural network modules, and integrates intermediate outputs in higher level functions. This paper proposes a hardware modular neural tile (MNT) architecture that reduces the SNN topology memory requirement of NoC-based hardware SNNs by using a combination of fixed and configurable synaptic connections. The proposed MNT contains a 16:16 fully-connected feed-forward SNN structure and integrates in a mesh topology NoC communication infrastructure. The SNN topology memory requirement is 50 {\%} of the monolithic NoC-based hardware SNN implementation. The paper also presents a lookup table based SNN topology memory allocation technique, which further increases the memory utilisation efficiency. Overall the area requirement of the architecture is reduced by an average of 66 {\%} for practical SNN application topologies. The paper presents micro-architecture details of the proposed MNT and digital neuron circuit. The proposed architecture has been validated on a Xilinx Virtex-6 FPGA and synthesised using 65 nm low-power CMOS technology. The evolvable capability of the proposed MNT and its suitability for executing subtasks within a MNN execution architecture is demonstrated by successfully evolving benchmark SNN application tasks representing classification and non-linear control functions. The paper addresses hardware modular SNN design and implementation challenges and contributes to the development of a compact hardware modular SNN architecture suitable for embedded applications",
    keywords = "EWI-22722, Modular neural networks (MNN), Network on Chip (NoC), IR-84278, Spiking Neural Networks (SNN), Synaptic Connectivity, METIS-296178, EMBRACE",
    author = "Sandeep Pande and Fearghal Morgan and Seamus Cawley and Tom Bruintjes and Smit, {Gerardus Johannes Maria} and Brian McGinley and Snaider Carrillo and Jim Harkin and Liam McDaid",
    note = "eemcs-eprint-22722",
    year = "2013",
    month = "10",
    doi = "10.1007/s11063-012-9274-5",
    language = "Undefined",
    volume = "38",
    pages = "131--153",
    journal = "Neural processing letters",
    issn = "1370-4621",
    publisher = "Springer",
    number = "2",

    }

    Pande, S, Morgan, F, Cawley, S, Bruintjes, T, Smit, GJM, McGinley, B, Carrillo, S, Harkin, J & McDaid, L 2013, 'Modular Neural Tile Architecture for Compact Embedded Hardware Spiking Neural Network', Neural processing letters, vol. 38, no. 2, pp. 131-153. https://doi.org/10.1007/s11063-012-9274-5

    Modular Neural Tile Architecture for Compact Embedded Hardware Spiking Neural Network. / Pande, Sandeep; Morgan, Fearghal; Cawley, Seamus; Bruintjes, Tom; Smit, Gerardus Johannes Maria; McGinley, Brian; Carrillo, Snaider; Harkin, Jim; McDaid, Liam.

    In: Neural processing letters, Vol. 38, No. 2, 10.2013, p. 131-153.

    Research output: Contribution to journalArticleAcademicpeer-review

    TY - JOUR

    T1 - Modular Neural Tile Architecture for Compact Embedded Hardware Spiking Neural Network

    AU - Pande, Sandeep

    AU - Morgan, Fearghal

    AU - Cawley, Seamus

    AU - Bruintjes, Tom

    AU - Smit, Gerardus Johannes Maria

    AU - McGinley, Brian

    AU - Carrillo, Snaider

    AU - Harkin, Jim

    AU - McDaid, Liam

    N1 - eemcs-eprint-22722

    PY - 2013/10

    Y1 - 2013/10

    N2 - Biologically-inspired packet switched network on chip (NoC) based hardware spiking neural network (SNN) architectures have been proposed as an embedded computing platform for classification, estimation and control applications. Storage of large synaptic connectivity (SNN topology) information in SNNs require large distributed on-chip memory, which poses serious challenges for compact hardware implementation of such architectures. Based on the structured neural organisation observed in human brain, a modular neural networks (MNN) design strategy partitions complex application tasks into smaller subtasks executing on distinct neural network modules, and integrates intermediate outputs in higher level functions. This paper proposes a hardware modular neural tile (MNT) architecture that reduces the SNN topology memory requirement of NoC-based hardware SNNs by using a combination of fixed and configurable synaptic connections. The proposed MNT contains a 16:16 fully-connected feed-forward SNN structure and integrates in a mesh topology NoC communication infrastructure. The SNN topology memory requirement is 50 % of the monolithic NoC-based hardware SNN implementation. The paper also presents a lookup table based SNN topology memory allocation technique, which further increases the memory utilisation efficiency. Overall the area requirement of the architecture is reduced by an average of 66 % for practical SNN application topologies. The paper presents micro-architecture details of the proposed MNT and digital neuron circuit. The proposed architecture has been validated on a Xilinx Virtex-6 FPGA and synthesised using 65 nm low-power CMOS technology. The evolvable capability of the proposed MNT and its suitability for executing subtasks within a MNN execution architecture is demonstrated by successfully evolving benchmark SNN application tasks representing classification and non-linear control functions. The paper addresses hardware modular SNN design and implementation challenges and contributes to the development of a compact hardware modular SNN architecture suitable for embedded applications

    AB - Biologically-inspired packet switched network on chip (NoC) based hardware spiking neural network (SNN) architectures have been proposed as an embedded computing platform for classification, estimation and control applications. Storage of large synaptic connectivity (SNN topology) information in SNNs require large distributed on-chip memory, which poses serious challenges for compact hardware implementation of such architectures. Based on the structured neural organisation observed in human brain, a modular neural networks (MNN) design strategy partitions complex application tasks into smaller subtasks executing on distinct neural network modules, and integrates intermediate outputs in higher level functions. This paper proposes a hardware modular neural tile (MNT) architecture that reduces the SNN topology memory requirement of NoC-based hardware SNNs by using a combination of fixed and configurable synaptic connections. The proposed MNT contains a 16:16 fully-connected feed-forward SNN structure and integrates in a mesh topology NoC communication infrastructure. The SNN topology memory requirement is 50 % of the monolithic NoC-based hardware SNN implementation. The paper also presents a lookup table based SNN topology memory allocation technique, which further increases the memory utilisation efficiency. Overall the area requirement of the architecture is reduced by an average of 66 % for practical SNN application topologies. The paper presents micro-architecture details of the proposed MNT and digital neuron circuit. The proposed architecture has been validated on a Xilinx Virtex-6 FPGA and synthesised using 65 nm low-power CMOS technology. The evolvable capability of the proposed MNT and its suitability for executing subtasks within a MNN execution architecture is demonstrated by successfully evolving benchmark SNN application tasks representing classification and non-linear control functions. The paper addresses hardware modular SNN design and implementation challenges and contributes to the development of a compact hardware modular SNN architecture suitable for embedded applications

    KW - EWI-22722

    KW - Modular neural networks (MNN)

    KW - Network on Chip (NoC)

    KW - IR-84278

    KW - Spiking Neural Networks (SNN)

    KW - Synaptic Connectivity

    KW - METIS-296178

    KW - EMBRACE

    U2 - 10.1007/s11063-012-9274-5

    DO - 10.1007/s11063-012-9274-5

    M3 - Article

    VL - 38

    SP - 131

    EP - 153

    JO - Neural processing letters

    JF - Neural processing letters

    SN - 1370-4621

    IS - 2

    ER -