Research On Model Compression And Acceleration Based On Network Growth Method

Recently,deep learning has become a hot topic in the fields of computer vision and natural language processing.Though deep neural networks are very powerful,the large number of parameters and complex structures consume considerable storage and calculation time,making it hard to deploy on limited hardware platforms.Traditional model compression and acceleration algorithms include network pruning,network distillation,quantization and binarization,etc.However,the state-of-the-art deep models are often engineered with superior manual and hyper-parameters adjustments,thus hard to compress.On the other hand,current methods often loss great performance in order to chase high compress-ratio and speed-up ratio.In order to achieve the relative balance between compress-ratio,speed-up ratio and model performance,our method extends the idea of distilling by introducing a student network population growing with evolution.The population is initialized with several basic structures,by reusing the weights,we provide five enhancement options to variate and strengthen the networks.We define a Fitness function to evaluate the overall performance of the model,and use Fitness score to select and eliminate model.We also discussed binarization method,which can speed up the network by using bit operations instead of traditional convolution multiplication.By using multiple sets of binarized weights to fit real value weights and integrate it into our network growth framework,we can further compress the network and achieve high speed-up ratio while guaranteeing the performance.We propose a network growth framework based on genetic algorithm and binarization method,compared with network distillation and binarization method,it can achieve relative balance between compress-ratio,speed-up ratio and model performance.The result shows our method can effectively compress and accelerate the original model while maintain the performance,which is a significant effort.