Research On Fine-tuning Process And Optimization Method Of Pre-training Model Based On Visual Analysis

The understanding of the model and the demand for data have always been the key issues that cannot be ignored in the field of deep learning.Although the emergence of pre-training models reduces the data requirements for model training,it is still affected and limited by the "black box" characteristics of deep learning.Existing research mainly focuses on the improvement of fine-tuning strategies or model structures,while ignoring the understanding of the fine-tuning process of pre-trained models.With the increase of model complexity,the decision-making process inside the model becomes difficult to understand,and the improvement direction of the model is gradually blind,which makes it more and more important to establish an intuitive understanding of the model construction process.Research on interpretability methods that can be used to analyze the fine-tuning process of pre-training models,can analyze the differences before and after fine-tuning of pre-training models,understand how shared knowledge is reused in models,then optimize and improve the model,and provide a new perspective for future research.This thesis focuses on the visualization and visual analysis of the fine-tuning process of the pre-training model.Based on the combination of visual analysis results and model guidance method,an adaptive fine-tuning model is optimized.Its main research work includes:(1)Aiming at the problems of deep network layers and complex models of Res Net network,an important channel extraction and important inter-layer influence aggregation algorithm is proposed.Use Res Net as the basic network to build the pre-trained and fine-tuned models,take the two models as the analysis objects,and use the image channel as the research granularity.The method of activation value aggregation and layer-by-layer convolution is used to analyze the important channels in the model and aggregate the important inter-layer influence.impact to analyze key information in the model.In order to consider the transfer of shared knowledge during fine-tuning,the similarity calculation is performed on the important channels of the pre-trained model and the fine-tuned model,and the important channels with similar activations in each layer of the model are obtained.This provides data support for subsequent visual analysis.(2)Aiming at the problem of unclear reasons for parameter adjustment in the fine-tuning process,based on the visual analysis theory,a visual analysis method of the fine-tuning process based on model comparison is proposed.Taking the extracted important channels and important inter-layer influence as the data source,the visual view is designed and constructed for the purpose of model comparison and the mixed layout method as the layout.Based on the constructed view,comprehensively considering the importance of channels and the importance of inter-layer influence,the reusable feature channel screening criteria are formulated through the visual analysis of similarity comparison.Finally,a case study is performed on a generic image dataset to test the effect of the visual analysis method.(3)Aiming at the problem of bloated structure of an adaptive fine-tuning model called spottune,a model optimization method based on visual analysis is proposed.Combining the structural characteristics of the model with visual analysis,the model is optimized from two aspects: On the one hand,the structured pruning method is combined with the visual analysis of reusable feature channels,and the selection method of clipped channels based on visual analysis is proposed;On the other hand,a distortion data sensitive channel selection algorithm combined with visual analysis is proposed,which can improve the robustness of the model by fine tuning only the channels with high sensitivity on the source domain model.In order to analyze the effect of the optimized model,experiments are carried out on multiple general data sets of image classification.The experimental results show that the optimization method proposed in this thesis can effectively reduce the training parameters of the model and improve the robustness of the model,and the effect is the most obvious in reducing the training parameters of the model.