Research On Transfer Learning For Classification Tasks

Data classification is an important way for humans to obtain knowledge from massive amounts of data,and it is also a fundamental task of machine learning.It has been widely studied and applied in various industries.Machine learning methods rely on a large amount of labeled data to learn a classification model.However,in real dynamic environments,the distribution of data is often constantly changing,and the generalization performance of existing models in new fields is often significantly reduced.Transfer learning is a new learning paradigm,which can use the knowledge of one or more auxiliary domains related to new domain to improve the performance of models in new domain,and achieve cross domain knowledge reuse and model learning.However,in cross domain classification tasks,transfer learning still has the following problems to be solved urgently:first,the distribution of multi-category data usually presents a multimodal structure,which makes it difficult to accurately measure and minimize the difference in feature distribution between domains,causing the problem of under adaptation in the model;Secondly,in the shared subspace learned by the model,the discriminability of target domain features is weak,and samples near the decision boundary are easily to be misclassified by the classifier;Thirdly,although multiple auxiliary domains can provide richer information for the target domain,the correlation between multiple auxiliary domains and the classification task of the target domain is difficult to accurately evaluate,leading to negative transfer;Finally,when multiple domain data is distributed and cannot be directly shared,it is difficult to effectively transfer knowledge,resulting in insufficient cross domain generalization ability of the model in distributed scenarios.This dissertation studies the problems of cross-domain classification tasks based on deep neural networks,and proposes solutions to the above problems.The main contributions of this dissertation are listed below:(1)In order to solve the problem that the adversarial learning based transfer learning methods can not fully adapt the feature distribution between domains,an Adversarial Sliced Wasserstein Domain Adaptation Network(AWDAN)is proposed.In AWDAN,the marginal distribution is adapted by adversarial training in feature space and the conditional distribution is matched by using Sliced Wasserstein distance in label space.The proposed AWDAN not only fully adapts the distribution between domains,but also can effectively reduce the distance between the target features and the decision boundary of the classifier,and enhance the discriminability of the target features.In addition,in order to further improve the discriminability of features,AWDAN freezes the parameters of the pre-trained network and adds some bottleneck layers for training.Theory analysis is provided to prove that AWDAN is able to optimize the upper bound for expected error on target domain.Experimental results show that the proposed AWDAN significantly outperforms existing transfer learning methods on three visual domain adaptation classification tasks.(2)In order to solve the problem that that the semantic alignment based transfer learning methods often difficult to accurately align the semantic information between domains,a Semantic Pairwise Centroid Alignment method(SPCA)is proposed.The training process of SPCA can be divided into three steps:pre-training,calculation,and adaptation.First,the credibility of target pseudo labels in the first few iterations of the calculation step is potentially improved.Next,we select high-confidence predicted target samples whose label probability is greater than a given threshold and assign pseudo labels to them,then search the high-confidence centroid points.Finally,the semantic information between domains is aligned accurately by using the proposed semantic centroid loss.Different from previous transfer learning methods,SPCA can form discriminative clusters in feature space during training.Furthermore,to accurately estimate the centroid of target domain in each iteration,SPCA constructs a global target centroid and updates it during training periodically.Extensive experiments on visual classification tasks verify the effectiveness of the proposed SPCA.(3)In order to solve the the negative transfer problem in distributed multi-source transfer learning,a Federated Hypothesis Transfer(FedHT)method is proposed.FedHT optimizes local models for each source domain and constructs the global model by aggregating local models to transfer the knowledge from source domains to target domain.To achieve domain adaptation without accessing source data,FedHT freezes the classifier(hypothesis)of the global model and fine-tunes its feature extractor.Meanwhile,a dynamic weighting strategy is designed to alleviate the negative transfer.Extensive experiments on four benchmark datasets verify the effectiveness of FedHT and also show that FedHT is robust to negative transfer.(4)In order to solve the problem that the feature distribution of multiple heterogeneous domains cannot effectively adapt,a Dual-Adversarial Transfer Network(DATN)is proposed.DATN contains two adversarial learning processes:domain-level adversarial training and classlevel adversarial training.In domain-level adversarial training,the Wasserstein distance between each pair of source and target domains is minimized to learn domain-invariant feature representation in feature space.In class-level adversarial training,DATN proposes a novel minimax entropy method to match the conditional distribution between domains,reduce the intra-domain discrepancy in target domain and enhance the discriminability of target features.Furthermore,in order to improve the performance of the model by modeling the correlations between multiple source domains and the target domain during the training,DATN adopts the weakly shared layers.Experimental results on a large number of multi-source heterogeneous transfer tasks verify the effectiveness of the proposed DATN.