Towards lossless binary convolutional neural networks using piecewise approximation

Research output: Chapter in Book/Conference proceedings/Edited volumeChapterScientificpeer-review

10 Downloads (Pure)

Abstract

Binary Convolutional Neural Networks (CNNs) can significantly reduce the number of arithmetic operations and the size of memory storage, which makes the deployment of CNNs on mobile or embedded systems more promising. However, the accuracy degradation of single and multiple binary CNNs is unacceptable for modern architectures and large scale datasets like ImageNet. In this paper, we proposed a Piecewise Approximation (PA) scheme for multiple binary CNNs which lessens accuracy loss by approximating full precision weights and activations efficiently, and maintains parallelism of bitwise operations to guarantee efficiency. Unlike previous approaches, the proposed PA scheme segments piece-wisely the full precision weights and activations, and approximates each piece with a scaling coefficient. Our implementation on ResNet with different depths on ImageNet can reduce both Top-1 and Top-5 classification accuracy gap compared with full precision to approximately 1.0%. Benefited from the binarization of the downsampling layer, our proposed PA-ResNet50 requires less memory usage and two times Flops than single binary CNNs with 4 weights and 5 activations bases. The PA scheme can also generalize to other architectures like DenseNet and MobileNet with similar approximation power as ResNet which is promising for other tasks using binary convolutions. The code and pretrained models will be publicly available.

Original languageEnglish
Title of host publicationECAI 2020: 24TH EUROPEAN CONFERENCE ON ARTIFICIAL INTELLIGENCE
Subtitle of host publication24th European Conference on Artificial Intelligence, including 10th Conference on Prestigious Applications of Artificial Intelligence, PAIS 2020 - Proceedings
EditorsGiuseppe De Giacomo, Alejandro Catala, Bistra Dilkina, Michela Milano, Senen Barro, Alberto Bugarin, Jerome Lang
Place of PublicationAmsterdam
PublisherIOS Press
Pages1730-1737
Number of pages8
Volume325
ISBN (Electronic)978-1-64368-101-6
ISBN (Print)978-1-64368-100-9
DOIs
Publication statusPublished - 2020
Event24th European Conference on Artificial Intelligence, ECAI 2020, including 10th Conference on Prestigious Applications of Artificial Intelligence, PAIS 2020 - Santiago de Compostela, Online, Spain
Duration: 29 Aug 20208 Sep 2020

Publication series

NameFrontiers in Artificial Intelligence and Applications
Volume325
ISSN (Print)0922-6389

Conference

Conference24th European Conference on Artificial Intelligence, ECAI 2020, including 10th Conference on Prestigious Applications of Artificial Intelligence, PAIS 2020
CountrySpain
CitySantiago de Compostela, Online
Period29/08/208/09/20

Fingerprint

Dive into the research topics of 'Towards lossless binary convolutional neural networks using piecewise approximation'. Together they form a unique fingerprint.

Cite this