Research And Application Of Transfer Learning Methods For Deep Convulutional Neural Networks

With the continuous development of deep learning,deep convolutional neural networks have been successfully applied to many fields.However,the deep convolutional neural network is supported by a large amount of labeled data,which brings about more difficult problems on data collection and labeling.In addition,traditional machine learning algorithms generally require that the training set and test set obey the same and independent distribution,but the actual situation is difficult to meet.Transfer learning uses existing knowledge to solve problems in different but related fields.At the same time transfer learning is not restricted by the condition that training set and test set need to meet independent and identical distribution,which inspired us to use transfer learning to solve the problem of dependence of deep convolutional neural networks on labeled data.Therefore,it is of great significance to study how to apply deep convolutional neural networks to transfer learning.This research proposes new transfer learning methods for labeled small-data transfer learning task,domain adaptation task and zero-shot learning task.The main work is as follows:(1)This thesis introduces the subject background and research significance of this paper,investigates the research status of transfer learning in deep convolutional neural networks at home and abroad.This thesis also introduces the different methods of transfer learning under different conditions,and related technologies such as deep convolutional neural network models.(2)When the source domain and the target domain have labels,but the tasks of the source domain and the target domain are different.A transfer learning method based on improved deep residual networks is proposed to solve the problem of neglecting the feature recognition ability of the model in the target domain during the fine-tuning of the existing methods.Res Net-34 is used as the source model,and an adjustment module is added to improve the feature recognition of the model and reduce the content difference between the source and target domains.The experimental results show that the improved deep residual network transfer learning method has significantly improved the recognition accuracy on the training set and the test set.(3)When the source domain is labeled,the target domain is unlabeled,and the tasks of the source domain and the target domain are the same.The existing methods only adapt to the fully connected layer when reducing the domain shift,and ignore the spatial information and semantic context information of the convolutional layer,which causes the important information lost during the process of knowledge transfer,so a method of deep domain adaptation based on PE divergence instance filtering is proposed.The PE divergence is used to delete source domain samples that are liable to cause negative transfer.The maximum mean discrepancy criterion is used to jointly match the edge probability distributions of the convolutional layer and the fully connected layer to solve the problem of overfitting.At the sanme time,this method also introduces weight regular term.Finally,experiments verify that the proposed algorithm can reduce the distribution difference between the source domain and the target domain,and improve the domain adaptability.(4)The premise of domain adaptation is that the categories of source domain and target domain are the same.Zero-shot learning,as a transfer learning method,deals with the problem that the training set and the test set have different instance categories.The purpose is to identify and classify the testing set that has not appeared in the training set.Aiming at the inability of the existing methods to express and distinguish samples with unseen classes in the test phase,a deep ensemble zero-shot learning method based on attribute balance is proposed.The relationship between attributes and labels is established through the class semantic transfer layer.The attribute balance constraint term is introduced to balance the attributes associated with the seen and unseen classes,and then learn multiple classification functions through the ensemble network to enhance the diversity of the classifier.On the zero-shot learning task of the three mainstream attribute datasets,we achieve better performance.