Transfer Learning From Multiple Source Domains

The recent decades have seen a rapid development of artificial intelligence in which machine learning techniques play important roles.Traditional machine learning assumes that training data and test data follow independent identical distribution.However,this assumption may not be held in many real-world applications,i.e.,the training data(i.e.,source domain data)and the test data(i.e.,target domain data)have different distributions,or their feature spaces are different.To address this issue,transfer learning emerged and attracted considerable attention.Transfer learning aims to leverage the knowledge extracted from one or multiple source domains to assist the prediction of a target domain.Currently,most previous transfer learning works are based on one source domain,i.e.,single-source transfer learning.These works have achieved promising results in many real-world areas.However,one single source domain can not provide a comprehensive perspective of knowledge to the target domain,which is also known as the insufficient information issue.To address this issue,we need to extract knowledge from multiple source domains,called multi-source transfer learning.In addition,in the era of big data,multisource transfer learning may not response immediately to the new online coming data,which is called the response delay issue.Therefore,we jointly consider multi-source transfer learning and online learning,and propose to study online multi-source transfer learning paradigm.To boost multi-source classification and prediction tasks,this thesis presents a comprehensive study on the issues and solutions of multi-source transfer learning.We summarize the main contributions of the thesis as follows:1)We propose a multi-source transfer learning algorithm based on optimal transport,called Domain-attention Conditional Wasserstein Distance(DCWD).The proposed algorithm first incorporates the label information to measure the distance between the source and target domains,and then uses an attention scheme to compute the transferred weights based on the conditional Wasserstein distances across different domains.The transferred weights are used to evaluate the relevance across domains,and to control the amount of knowledge transferred from each source domain to the target domain,thus boost the performance of multi-source transfer learning.2)We propose a multi-source transfer learning algorithm based on feature selection and optimal transport,called Iterative Refinement based on Feature Selection and the Wasserstein distance(IRFSW).In each iteration,the proposed algorithm first uses feature selection strategy to learn representations so that the discrepancy between each source and target domains is minimized,and then adopts optimal transport to evaluate the domain relevance based on the Wasserstein distance between the learned source and target representations.As a result,both discrepancy and relevance across different domains are considered.3)We treat co-occurrence data as intermediate domain data,which together with source domain data to form a multi-source transfer learning problem.We propose an online heterogeneous multi-source transfer learning algorithm based on co-occurrence data,called Online Heterogeneous Knowledge Transition(OHKT),to solve the response delay issue.The proposed algorithm uses co-occurrence data to bridge heterogeneous source and target domains,so that the label information of the source domain can be transited to the target domain.In addition,the proposed method enforces the knowledge in the source domain and the previous target data sequence remains close to that in the coming target data.4)We further consider the situation that the co-occurrence data are unavailable or hard to collect,and propose online homogeneous and heterogeneous multi-source transfer learning algorithms to solve the response delay issue.The proposed algorithms jointly adopt ensemble and Hedge strategies to adaptively update the weights of each source and target domain.We also provide theoretical analyses of the proposed algorithms.