A Study On Transfer Learning And Its Application

Transfer learning as a new learning framework of machine learning does not make the identical distribution assumption as traditional machine learning. Therefore,it uses the data in a different domain to help the target task to effectively solve the learning problem in which the training and test data are of different distributions.This paper,based on Support vector machine(SVM) method,take transfer learning as research content and improve the precision and speed of the target domain as its aim. Therefore,several transfer learning methods are proposed,aiming at the problem that the sample selection and feature reconstruction in the transfer learning mode from “what to transfer”,” how to transfer” and “how many to transfer” views. The main research results include:1. Transfer learning algorithms usually focus on reusing data of related domains to help solving the learning tasks in the target domain. However,these methods ignore the ability of mutual learning between domains. In this paper,a collaborative constraint based symbiosis transfer learning method(CCSTL) is proposed. Symbiotic transfer mechanism is used to implement mutual learning among domains along with the collaborative constraint. With the help of the iterative optimizations,the proposed method can realize knowledge transfer between the source and target domains. Experimental results on synthetic and real world datasets show the superior or comparable performance of the proposed algorithm compared with existing algorithms.2. The traditional transfer feature algorithms usually focus on learning by using the common features between the source domain and the target domain but ignore the discriminant information of the specific features of each domain,which makes the existing algorithms lack the adaptability to some extent. For this issue,in this paper a novel subspace transfer learning algorithm integrating with heterogeneous features(STL-IHF) is proposed based on the empirical risk minimum framework. The proposed method takes the feature space of each domain as a combination of the common features and the specified features,and further develops the algorithm based on the SVM framework by integrating with this feature combination. The derived algorithm can not only realize the transfer learning from the common features but also effectively leverage the specified features of each domain,which makes it have much better adaptability in the learning procedure.3. Many transfer learning algorithm are transformed into quadratic programming(QP) problem to solve,the adaptabilities of the multiple source transfer learning are restricted due to the large complexity of time and space consuming of the kernel matrix. A novel common-decision-vector based multiple source transfer learning(CDV-MSTL) and its fast classification learning method was proposed to solve this problem. Those decision vectors of domains and common decision vector are embedded in the process of training support vector machine(SVM) based on structural risk minimization principle,So the proposed method is of the good extension ability and the better accuracy. And Combined with the theory of CVM, realize training and classification of large sample quickly by the extension of the CDV to the CDV-CVM. The experimental results on simulation and real data set show the power of the proposed CDV-MSTL and CDV-CVM algorithms.4.At present,in order to handle with the problem of concept drift,a novel SVM-based algorithm which aims at classification of the concept drifting called NMD-SVM is proposed accordingly. The concept of the neighbor projection mean discrepancy in the reproducing kernel Hilbert space is defined to measure the discrepancy between adjacent sub-classifiers and the distribution of data are embed into the process of global optimization. The adaptability of classification algorithm can be enhanced. The experimental results in the simulation and real indicate the power of the proposed algorithm.5. Feature selection is an important problem in the pattern recognition and machine learning areas. But many of the existing methods overly depend on the feature weighting technique and have many optimized parameters. In this paper,According to the single and synergetic feature phenomena of the data set,we propose a novel supervised feature ranking method based on the integrated square error and random permutation. Making the developed random permutation theory to adapt synergetic feature phenomena, we propose a multidimensional collaborative feature selection algorithm and sort the features. At last,optimal feature subset selection is determined by the classification accuracy of the KNN algorithm. Experimental results on synthetic and real world datasets show the superior or comparable performance of the proposed algorithm compared with existing algorithms.