| Binary quantization quantifies the weights and activations to 1 bit,greatly reducing the storage space occupied by parameters.Furthermore,binarization uses simple and effective bit operations to replace the complex multiplication and accumulation operations in convolution operations,which is a very efficient neural network compression method.However,with high compression rates,binarized network models often face the problem of degraded performance.Moreover,existing binarization methods have not changed the basic structure of convolutional neural networks,and their redundant convolutional kernel parameters still hinder the end-to-end deployment process of neural networks.As a nanoscale device,the memristor has characteristics of memory fusion,large-scale parallel computing,and low power consumption,which is the ideal device for building integrated circuits and deploying convolutional neural network models.Therefore,designing binary convolutional neural networks based on memristor could assist in resolving the problems of the existing deep convolutional neural network models with large computational complexity and difficulty in depeloying on end-to-end device,and has important research significance and practical application value.In response to the above issues,the thesis explored the performance improvement of binary convolutional neural networks by combining nanoscale device memristors,designing training methods,and building lightweight network structures.The main contributions are summarized as follows:(1)Aiming at the performance degradation of binary convolutional neural networks,a training framework for training binary convolutional neural networks,KDG-BNN,was designed by combining knowledge distillation with generative adversarial networks.This framework can improve the performance and operational efficiency of binarized convolutional neural networks by simultaneously optimizing resistance losses and distillation losses,forcing the output characteristic distribution of binary convolutional neural networks to be more similar to the output characteristic distribution of full precision convolutional neural networks.Then,the Wasserstein generated adversarial network with gradient penalty is used to replace the original generated adversarial network to handle the problem of gradient disappearance,and KDG-BNN is optimized to a more efficient training framework: KDWG-BNN.In addition,based on the conventional double column memristive crossbar arrays,a memristor circuit implementation scheme is designed for the core operation units of the binary convolutional neural network model.Experimental results show that both KDG-BNN and KDWG-BNN can accelerate convergence and improve the performance of binary convolutional neural networks.(2)Aiming at the problem that deep binary convolution neural networks still have a large number of convolution kernel parameters,a lightweight binary convolution neural network called BSSC-Net is proposed by introducing spatial separable convolution to reduce the parameters of the binary convolution kernel.Then,based on the network structure of BSSC-Net,a training framework called KDSNG-BNN is designed for training BSSC-Net.In this framework,BSSC-Net can improve its ability to learn feature distribution by simultaneously performing knowledge distillation and adversarial training.Secondly,by combining the single column memristive crossbar arrays and constant term circuit,a high efficiency and low power memristive neural network implementation scheme is designed for BSSC-Net,and the progressiveness and competitiveness of the scheme are analyzed through the power consumption of the memristor circuit.Experimental results show that compared to traditional binary convolutional neural networks,BSSC-Net can effectively reduce the amount of parameters and maintain similar accuracy. |
