Research Of Domain Adaptation Methods Based On Cross-Domain Regularization Model

Traditional machine learning methods assume that training data and test data are independent and identically distributed.Therefore,to ensure the generalization performance of the learning model,they require a large number of labeled samples distributed with the test data.However,in practical applications,it is difficult or even impossible to collect enough training samples for a particular application due to the changes of environment or the constraint of sampling condition.In order to solve the problem of the scarcity of labeled samples,domain adaptation has been proposed and received wide attention.Domain adaptation is a branch of transfer learning,which relaxes the requirement of traditional machine learning for data to be independent and distributed.Given a large number of labeled samples from the source domain and a large number of unlabeled samples from the target domain,domain adaptation assumes that data distributions of the source domain and target domain are different,but their tasks are the same.In other words,they have the same category labels.The goal of domain adaptation is to make use of the labeled samples from source domain to help learn a better classifier in the target domain,such that the demand for the labeled samples from target domain is reduced.Despite the performance is improved,the existing domain adaptation methods mainly focus on how to reduce the distribution discrepancy between the source domain and target domain,ignoring the role of unlabeled samples from target domain when learning the classifier.It is difficult to obtain an ideal result when the distribution discrepancy between source domain and target domain is huge.Basing on the empirical risk minimization criterion and regularization theory,in order to improve the generalization and classification accuracy of the classification model,this dissertation aims to research how to design and add regularization items to the empirical risk.The specific research contents are as follows:1.A model for solving the domain adaptation problems is proposed: cross-domain regularization model.Compared with the traditional regularization model,in addition to minimizing the empirical and structural risks,the cross-domain regularization model can also effectively reduce the distribution discrepancy between the source domain and the target domain,making it possible to train a target classifier by using samples from the source domain.In addition,on the basis of the traditional regularization model,the cross-domain regularization model can fully excavate and utilize the prior informationprovided by the unlabeled samples in the target domain by adding appropriate constraints,such that the influence of the distribution discrepancy between the domains is further reduced,and the performance of the classifier in the target domain is improved.This model not only gives a new interpretation of the domain adaptive learning problem from the perspective of semi-supervised learning,but also provides a platform for improving the existing algorithms and designing new ones.2.The domain adaptation method based on cross domain discriminative analysis and soft label regularization is proposed.In this dissertation,the optimization of cross-domain regularization model is divided into two steps: feature learning and classifier learning.The goal of feature learning is to learn a feature transformation function,to project the samples of the source domain and the target domain into a common subspace,in which the discrepancy between the distributions are reduced.The goal of classifier learning is to learn a classifier in the result feature space.For this reason,this dissertation proposes a cross-domain semi-supervised feature learning method: cross-domain discriminant analysis,which can preserve the discriminative information of samples and guarantee the separability of samples in the result space while reducing the distribution discrepancy between domains.In addition,this dissertation presents a cross-domain semi-supervised classifier learning method: soft label regularization,which uses the Laplace regularization item to maintain the manifold structure of source domain and target domain.By introducing cross-domain constraints,the class prior probabilities of the source domain and the target domain can be balanced in the process of classification,and the prediction accuracy can be improved.The two methods constitutes a complete domain adaptation method,which can make use of the label information of the source domain samples and the structure information of the target domain samples,such that utilization of data is improved.Experimental results on multiple datasets show that this method is better than the existing ones in most cases.3.The structure preserved cross domain feature learning method is proposed.The existing domain adaptation methods ignore the structural information of the target domain samples in the feature learning process,which leads the loss of the structural information,such that the performance of the classifier may not improve although using the unlabeled samples from the target domain in the classifier learning process.This dissertation extends the cross-domain discriminant analysis approach and propose a structure preserved cross domain feature learning method,which can simultaneously achieve the followingthree objectives :(1)reduce the distribution discrepancy between source domain and target domain;(2)preserve the discrimination information of the labeled samples in the source domain;(3)preserve the structural information of unlabeled samples in the target domain.In addition,all of the above three objectives can be integrated into a unified optimization function,and the optimal solution can be solved through generalized eigenvalue decomposition.Experimental results on different tasks such as target recognition,face recognition,digit recognition,etc.show the effectiveness of this method.4.The multi-view joint regularization for domain adaptation is proposed.With the development of multimedia technology,one can more easily obtain auxiliary information along with image,such as text,voice and so on.Although using multimodal data can improve the performance of the classifier,this emerging data is often unlabeled.Although domain adaptation methods can use knowledge from related domains to reduce the demand for labeled samples,the existing methods only focus on single-mode data and cannot make use of auxiliary information other than images.To solve the domain adaptation problem from single viewpoint to multi viewpoint,namely the source domain only contains the images and the target domain contains images and auxiliary information.This dissertation propose multi-view joint regularization,which learns two classifiers: the visual classifier using images and the auxiliary classifier using auxiliary information.Finally,the category of the test data is determined through the weighted combination of the two classifiers.In this dissertation,the depth information is adopted as the auxiliary information to do experiments.The experimental results show that the auxiliary information is effective to improve the performance of image classification.