On Methods Of Domain Adaptation Based On Subspace Learning

Domain Adaptation is an important research direction in machine learning.Nowadays large sample learning has made significant progress,which however can't be implemented efficiently in some new real-world scenarios where training data is expensive or difficult to collect.Naturaly we wish to exploit data which is available and more sufficient in similar domain,but the distributions of two domains might not be match,and the data from other domain could bring negative effect if it is used directly with the data from the target domain.Thus,domain adaptation technique is critical in order to make fully use of auxiliary data and information.This paper focuses on how to reduce domain shift with subspace learning,and obtain re-representation of the original data in the subspace,to improve performance of task in the target domain.The main research findings are as follows:First,we analyzed the advantage and disadvantage of traditional domain adaptation methods,as well as the properties of sparse representation in domain adaptation,and proposed a new method termed as sparse reconstruction domain adaptation in embedding subspace(SRES).An embedding subspace is constructed where we can obtain the ideal low-dimensional re-representation of the original data.The target data in embedding subspace is supposed to be sparsely reconstructed from others,so the re-represented data can preserve the shared structure of both domains.Meanwhile,the introduction of embedding subspace could relax the burden of projection.By separating the representations of projected data and data in embedding subspace and aligning them gradually,we can then obtain a flexible projection and subspace.Finally,in order to reduce the domain shift more effectivly,we expanded SRES into an inlinear version(NSRES)by means of kernel trick.On benchmark datasets,SRES was proved to be a better adaptation method compared to traditional methods,and the classification performance has also been improved in target domain.Considering above methods matching merely the marginal possibility distribution while ignoring conditional possibility distribution,we proposed a class-specific reconstruction domain adaptation in discriminant subspace(CRDS)method.Making fully use of label information,CRDS not only enforce target data not be sparsely reconstructed from others in subspace,but also demands the data in subspace could be regressed linearly to label matrix,thus increasing the discriminability of the subspace.On one hand,we add a slack variable into the label matrix,thus making the alignment of the distributions be more reasonab and flexible.On the other hand,we observed the fact that for traditional methods,much information would be lost when the original data was projected directly to the label space,which is undesirable.So we enforce source and target data in the subspace,which share the same marginal possibility distribution,should also share the same conditional possibility distribution,so there would be more information could be preserved in the subspace,and the re-represented data would be more discriminant.In addition,the distribution discrepancy may not be reduced effectively by only one projection,so we add an offset on source data in the subspace,in order to reduce domain shift more effectivly.If domain adaptaion technique is deemed as a technique for small sample problem in targe domain,there should be another sort of small sample problem,the lackness of samples in the few classes,to draw our attention.Observing that the classification performance would be deteriorated when TWSVM was implemented on class-imbalance datasets,we proposed a between-class discriminant twin support vector machine(BDTSVM)method.Firstly,based on Fisher Discriminant Analysis,we design a between-class discriminant regularization which is insensitive to imbalance data.Combine the regularization with the two hyperplanes of TWSVM,we then obtain the classification model.In this way,the majority class and minority class would have equal influence in training procedure,so the classification performance would be improved on imbalanced datasets.