Deep Binarized Convolutional Neural Network Inferences over Encrypted Data

Homomorphic encryption provides a way to perform deep learning over encrypted data and permits the user to encrypt the data before uploading, leaving the control of data on the user side. However, operations on encrypted data based on homomorphic encryption are time-consuming, especially in a deep convolutional neural network (CNN), which incorporates a large number of layers and operations. To speed up deep learning on encrypted data, we binarized the input data and weights of CNN model, while operations including the addition and multiplication in CNN become bit-wise operations. Therefore, the homomorphic evaluation of CNN can be performed in the binary field in a highly efficient way. We also construct an efficient pooling layer by designing circuits to perform comparison operations on the ciphertext. Simulation results clearly show that the convolution operation of the proposed model is at least 6.3 times faster than that of existing schemes. Last, our model exhibits no privacy leakage associated with the data being processed.