Research On Model Compression Method Based On Knowledge Evolution

In the field of computer vision,convolutional neural networks(CNN),as the mainstream deep learning model,have demonstrated excellent performance in various image classification tasks.However,when CNN is actually applied to transportation mobile devices and embedded systems,it is often plagued by limited storage space and computing resources.Model compression technology can effectively solve these problems and help achieve efficient deployment of models on these devices.However,high model compression rates often lead to significant performance degradation.Therefore,in response to the issue of balancing compression rates and performance in the field of model compression,this thesis conducts a series of comparative and ablation experiments on four commonly used classification datasets to verify the effectiveness and universality of the proposed algorithm.The main research work and improvement effects of each part of this thesis are as follows:(1)By conducting a comparative study of various techniques in the current field of model compression,we selected a model compression technique based on knowledge distillation training and employed convolutional kernel splitting to achieve the extraction of a compressed small network.This technique reduces both the parameter count and computational complexity of the Res Net18 residual network model to one-fourth of its original size.(2)Addressing the issue of accuracy loss in the compressed network,we optimized the feature extraction stage of the original knowledge distillation network based on the characteristics of each layer.This was achieved through the integration of a layered embedding approach involving lightweight local feature enhancement and global feature suppression modules(referred to as the LBGS module).The LBGS module enhances the network’s perception and discriminative ability for crucial features while suppressing background interference,thereby improving the network’s capability to perceive feature details and discriminate between classes.As a result,the performance of both the uncompressed and compressed networks is simultaneously enhanced.With no significant increase in parameter count or computational complexity,the improved main network achieved an accuracy improvement of 1% to5% compared to the original network,while reducing the performance gap between the compressed and main networks from the original 4% to no more than 2%.(3)To further mitigate the performance degradation caused by compression and enhance the performance of the compressed small network,a two-stage training approach was devised,incorporating knowledge distillation with ensemble batch feature information and logits.This approach was seamlessly integrated with the knowledge distillation method.Following the distillation training,the accuracy of the sub-network surpassed that of the original network by varying margins ranging from2.5% to 5.2%.