Realistic Few-shot Domain Adaptation

Machine learning has achieved vigorous development in the past decade,and has surpassed human ability in many fields,serving as a hot research topic today.The success of machine learning mainly depends on three keypoints: algorithm,data,and computational power(this papear focuses on algorithm and data).For special fields,e.g.,military,medicine,and finance,data are very scarce,which resulting from three aspects: 1)data are hard to collect;2)data owners do not release their data for protecting privacy;3)such special fields have strong professionalism,thus annotating data is very hard.To solve the data shortage problem,few-shot learning emerged,and became the research hotspot.However,naive few-shot learning is only suitable for the scenes where known data and target data come from the same distribution,which limits its usable range.Few-shot domain adaptation(FDA)is a frontier hot in the field of machine learning and can break down this barrier,aiming to train a target classifier with accessible labeled source data and few labeled target data.Noting that source domain and target domain are different but similar.It is unrealistic to collect many data in a specific field.It is particularly important to use existing data different from but similar to its distribution to assist these few data.However,accessing many similiar data is still hard in real scenarios.Therefore,this thesis aims to study few-shot domain adaptation in the case of lacking high-quality source data.Specificly,this thesis selects two novel and challenging settings: 1)few-shot domain adaptation without source data;2)few-shot domain adaptation with extremely noisy source data.Source-free few-shot domain adaptation.This setting is mainly used for scenes where source data contain much privacy,thus the data owner cannot directly provides them to others but a well-trained source model(i.e.,hypothesis)can.Then we propose a new problem called few-shot hypothesis adaptation(FHA).Motivated by the learnability of semi-supervised learning,we propose the target orientated hypothesis adaptation network(TOHAN).TOHAN first generates highlycompatible intermedaite domain data and then adapt them to target domain with adversarial domain adaptation framework.Experiments show that TOHAN solves FHA problem effectively and generated intermediate data contain no useful source information,thus the source privacy is protected well.Few-shot domain adaptation with extremely noisy source data.This setting is mainly used for scenes where annotating many data is too costly,and experts do not have the time to accurately label every data point for special fields.Thus we propose another new problem called wildly few-shot domain adaptation(WFDA).To solve WFDA,we propose the robust quadruple adaptation network(QAN)where it maintains two source hypotheses and two target hypotheses simutaneously.For each domain,one hypothesis filter the noisy data in a mini-batch and then deliver the filtered mini-batch to the other hypothesis for training.So it goes on,errors from one hypothesis will not accumulate and will be gradually eliminated in the process of data exchange.Experiments show that QAN solves WFDA problem effectively,and it can keep up good classification accuracy under very high noise rate(more than 40%).Excellent robustness and classification accuracy make QAN of high practical value.To sum up,this paper selects two complex and challenging real scenarios to study few-shot domain adaptation.In this paper,two important problems,namely,small sample hypothesis adaptation and chaotic small sample domain adaptation,are proposed and defined strictly for the first time,and effective solutions are given theoretically and methodicically,which opens up a new door for few-shot learning and transfer learning.