Empirical Study On The Effect Of Compression Methods On CNN Models

Today,deep network models are an important part of software systems.For exam-ple,in the field of computer vision,the convolutional neural network model is used for object recognition,image classification and other tasks,and most of the software built with these tasks as the core will be deployed on mobile computing devices.However,today's deep neural network models are very large,but the storage capacity,core fre-quency,and memory size of mobile computing devices are limited.For example,VGG architecture models require more than 500 Mb of memory space,take up up to 1Gb of memory at run time and consume a lot of CPU computation time.There are two solutions to this series of problems.The first is to come up with models that are lightweight and as accurate as the big ones.But the architecture of these models requires a lot of time to design from scratch,and the scope of their application is limited.They are also hard to keep up with large models in terms of accuracy.The second approach is to compress the existing model.Since almost all models contain redundant parameters that can be eliminated,the applicability of this method is very good.There is no need to redesign the method for different models,and even further optimization for lightweight models can be carried out.Existing compression methods all focus on reducing the size of the model and ensuring the accuracy of the model,but the model has other attributes besides these two points,such as fairness and robustness.In this thesis,three main model compression methods,namely pruning,quantization and knowledge distillation,are selected to study their effects on model attributes when they are used alone or in combination.In general,the main work of this thesis is as follows:(1)The influence of compression method on the model is analyzed from the per-spective of the efficiency and effectiveness of the model.Model efficiency refers to the size of the model in storage and runtime speed,memory,etc.The validity of the model represents the accuracy of the prediction of the model on the data set.These two are the most obvious effects of compression on the model.(2)The influence of compression method on the model is studied from the cred-ibility of the model.The reliability of the model includes two aspects: robustness and fairness.There is no clear and quantifiable definition of these two aspects in previous work.This article defines robustness in terms of the model's performance against at-tacks.In terms of fairness,this thesis extracts the color-gray feature,and transfers the concept of fairness used in machine learning field to image classification task.(3)In this thesis,three mainstream model compression methods,pruning,quan-tization and knowledge distillation,are selected,and they are mixed compression in different order to obtain a series of mixed compression methods.Combined with the ef-ficiency,effectiveness and reliability of the models,these hybrid compression methods are compared with pruning,quantization and knowledge distillation.This thesis studies the different effects of compression methods on the model when they are used indepen-dently and mixed,and gives the answer to the question of how to mix the compression methods and how to get the mixing effect.