Dual Path Binary Neural Network with Adaptive Output Range

  • 游 輝亮

Student thesis: Master's Thesis

Abstract

In recent years deep neural networks (DNNs) have achieved state-of-the-art results in the fields of image recognition semantic segmentation and machine translation However powerful DNNs usually have a large number of parameters and complex calculations For instance ImageNet classification challenge winner in 2012 Alex Net has a model size of about 249MB and 60 million parameters which needs to perform about 1 5 billion FLOPs to classify a 224 x 224 image While perform such complex computations GPUs based machines usually used to speed up training process and inference time However for embedded devices such as smart phones or Internet of Things there is only a small amount of memory battery power and computing resources so it is difficult to deploy DNN to these devices In the field of model compression the binary neural network (BNN) is a very promising method which features are low power consumption and low storage usage but there is a large gap in prediction accuracy compared with full-precision networks This thesis proposed a BNN that about the same storage usage as other BNNs and prediction accuracy is close to full-precision network The method proposed in this thesis has three characteristics: First the convolution layers have two input sources by dual path method Second round the batch normalization output Third adjust each layer output by a trainable parameter The experiments show our model size is about equal to other BNNs but the prediction accuracy is much higher In CIFAR-10 dataset the prediction accuracy is at least 2 85% higher than other BNNs even better than ternary network only 0 69% loss compared to full-precision network In SVHN dataset the prediction accuracy is at least 0 21% higher than other BNNs and even more than 0 58% compared to full-precision network
Date of Award2018 Aug 15
Original languageEnglish
SupervisorPei-Yin Chen (Supervisor)

Cite this

'