Research On Knowledge Distillation Algorithm Based On YOLOv3

With the rapid development of deep learning,a lot of object detection technologies have begun to be applied in traditional fields,such as automatic driving,face recognition,and industrial defect detection.The large model in object detection has good results,but with the huge network which is depth and width,which consumes a lot of calculation power and memory resources and is difficult to be deployed on the mobile devices.Small models can be deployed on the mobile devices,but they can not work well.As a model compression method,knowledge distillation can transfer the knowledge of the large network to the small network,and improve the detection result of the small network without changing the structure of it.In the knowledge distillation,the big network is the teacher network,and the small network is the student network.How to effectively transfer the knowledge of the teacher network to the student network has always been the most important issue of knowledge distillation.YOLOv3 model is commonly used in the industrial field,and this thesis will research the effective transfer of knowledge in knowledge distillation.The main contributions are as follows:1)Aiming at the problem that the importance of knowledge delivered by the teacher network has not been measured.We improve the existing mask-map and distill the existing model based on the information map.The teacher network is YOLOv3,and the student network is yolov3-tiny.The student network is retrained by the output of the teacher network.During the distilling process,the weight of the teacher network is freezed,and the knowledge learned by the teacher network from the training dataset is passed to the student network after being filtered by the information map,so that the output of the student network gradually approaches that of the teacher network.Experiments show that the distillation method based on information map improves the m AP index value by 8.3% on the VOC data set without changing the yolov3-tiny network structure.2)Aiming at the problem that the knowledge of the teacher network is not fully transmitted to the student network,we introduce that the feature extraction layer and the feature fusion layer can be simultaneously distilled,so that the student network can learn the teacher network more fully.In the distillation process,the difference between the output of the feature fusion layer of the teacher network and the student network at the feature fusion layer is measured,and the weight of the student network is updated through the backpropagation algorithm to continuously reduce the difference,so that the output of the student network at the feature fusion layer gradually approaches teacher network.Experiments show that the distillation of the feature fusion layer can continue to increase the m AP index value by 1% on the basis of(1)...