NASB: Neural Architecture Search for Binary Convolutional Neural Networks

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

4 Downloads (Pure)

Abstract

Binary Convolutional Neural Networks (CNNs) have significantly reduced the number of arithmetic operations and the size of memory storage needed for CNNs, which makes their deployment on mobile and embedded systems more feasible. However, after binarization, the CNN architecture has to be redesigned and refined significantly due to two reasons: 1. the large accumulation error of binarization in the forward propagation, and 2. the severe gradient mismatch problem of binarization in the backward propagation. Even though substantial effort has been invested in designing architectures for single and multiple binary CNNs, it is still difficult to find an optimized architecture for binary CNNs. In this paper, we propose a strategy, named NASB, which adapts Neural Architecture Search (NAS) to find an optimized architecture for the binarization of CNNs. In the NASB strategy, the operations and their connections define a unique searching space and the training and binarization of the network progress in the three-stage training algorithm. 1 Due to the flexibility of this automated strategy, the obtained architecture is not only suitable for binarization but also has low overhead, achieving a better trade-off between the accuracy and computational complexity compared to hand-optimized binary CNNs. The implementation of the NASB strategy is evaluated on the ImageNet dataset and demonstrated as a better solution compared to existing quantized CNNs. With insignificant overhead increase, NASB outperforms existing single and multiple binary CNNs by up to 4.0% and 1.0% Top-1 accuracy respectively, bringing them closer to the precision of their full precision counterpart.
Original languageEnglish
Title of host publication2020 International Joint Conference on Neural Networks, IJCNN 2020 - Proceedings
Place of PublicationPiscataway
PublisherIEEE
Pages1-8
Number of pages8
ISBN (Electronic)978-1-7281-6926-2
ISBN (Print)978-1-7281-6927-9
DOIs
Publication statusPublished - 2020
EventIJCNN 2020: International Joint Conference on Neural Networks - Glasgow, United Kingdom
Duration: 19 Jul 202024 Jul 2020

Publication series

NameProceedings of the International Joint Conference on Neural Networks

Conference

ConferenceIJCNN 2020
CountryUnited Kingdom
CityGlasgow
Period19/07/2024/07/20

Keywords

  • binary neural networks
  • neural architecture search
  • quantized neural networks
  • efficiency

Fingerprint Dive into the research topics of 'NASB: Neural Architecture Search for Binary Convolutional Neural Networks'. Together they form a unique fingerprint.

Cite this