Research On Deep Transfer Learning Based On Feature Mapping

Transfer learning is a new machine learning method that uses existing knowledge to solve different but related domain problems,which only limited or unavailable labeled target data are needed.Recent researches on transfer learning using deep neural network by have revealed that deep networks can learn transferable features that generalize well to novel tasks data for domain adaptation.However,the feature transferability drops significantly in task-specific layers with increasing domain discrepancy.Hence,the major bottleneck is how to formally measure and reduce the domain discrepancy and enhance the transferability in task-specific layers by matching different probability distributions of different domains effectively.This paper proposes several mappingbased deep migration learning methods: B-JMMD(Balanced Joint Maximum Mean Discrepancy),WB-JMMD(Weighted Balanced Joint Maximum Mean Discrepancy)and ADAMM(automatic domain alignment and moment matching),which aims to mapping instances from the source domain and target domain into a new data space,thereby reducing the dataset bias by some metrics of domain discrepancy.(1)B-JMMD and WB-JMMD:Justifying which components of the feature representations can reason about original joint distributions within the regime of deep architecture remains unclear.Hence,in this paper,two new metrics of Second-Order Maximum Mean Discrepancy: B-JMMD and WB-JMMD,are proposed to measure the difference of joint distribution across domains.BJMMD can leverage the importance of marginal and conditional distributions behind multiple domain-specific layers across domains adaptively to get a good match for the joint distributions in a second-order reproducing kernel Hilbert space(RKHS).On the basis of B-JMMD,WB-JMMD further reduces the shift of domains by dynamically considering the class weight in the last layer of the network within a batch size throughout the training process to restrain the harmful effect caused by class imbalance issues,and we only use WB-JMMD to solve the problem of domain adaptation in semi-supervised setting.B-JMMD and JMMD can be performed by a new form of stochastic gradient descent,in which the gradient is computed by backpropagation with a balanced strategy(2)ADAMM:in order to further improve the domain adaptation of feature mapping method based on moment matching,a deep transfer learning method based on automatic domain alignment and moment matching is proposed.Firstly,an automatic domain alignment layer(DA-Layer)is embedded in front of each domain specific layer in the deep neural network,and the source domain and the target domain are aligned preliminarily.Then a moment matching metric(such as MMD distance)is embedded between multiple domain-specific layers across domains,which take the source and target features of the DA-Layer output as input and map these features to a common RKHS to further reduces the distribution differences between the two domains and improves the performance of domain adaptation.