Research On Source Domain Transferability Estimation For Deep Domain Adaptation

As one of the most successful AI technologies,machine learning has been widely used in various applications in our daily life,including security,industrial manufacturing,medical filed and so on,bringing in enormous economic and social benefits.Traditional machine learning assumes that training samples and test samples are independent and identically distributed.However,due to the fact that real-world scenes are always complex and diversified,this assumption does not hold in real applications.For instance,various factors could lead to domain shift in vision-based applications,including lighting condition,image background,camera angle,image resolution and so on,resulting in visible performance reduction of conventional machine learning methods.Manually collecting and annotating massive training samples for each tasks would be prohibitively costly.Moreover,the cost of collecting training samples via different ways for the same tasks could also vary enormously.Therefore,there is a strong incentive to research on how to leverage supervision from a different but related source dataset to a target dataset.Domain adaptation aims to leverage the labeled source data and unlabeled target data,adapting model from source domain to target domain.Recent years have witnessed the great progresses in deep domain adaptation.A fruitful line of these works perform domain alignment in feature space,and then the discriminator could be directly adapted from source domain to target domain.MMD has been widely used to measure domain divergence.By minimizing MMD,the learned features are expected to be domain invariant.Despite the achieved progresses,these methods often suffer from the problem of unstable performance and low robustness.We aim to develop robust and efficient domain adaptation methods from three aspects,including the development of robust and effective domain discrepancy metric,accurate estimation of source samples’ transferabil-ity,and efficient transfer strategies for different source domains.The main contents and innovations of this thesis are summarized as follows:(1)To alleviate the negative transfer invoked by class prior distribution bias,weighted maximum mean discrepancy is proposed.The class prior distributions between domains are often inconsistent due to different sampling criteria or application scenarios in real world scenarios.MMD based approaches cannot cope with class weight bias across domains and suffer from severe negative transfer.Based on this consideration,WMMD is proposed by first reweighting the source classes.Concretely,the reweighted source would share the same class prior distribution with target and then the domain discrepancy is measured between weighted source and target domain.By incorporating WMMD into CNNs,WDAN is developed for unsupervised domain adaptation tasks.An extended classification expectation maximization algorithm is employed to solve WDAN.Experimental results show that WDAN is more robust to class weights bias between source and target,thus obtaining more stable and competitive domain adaptation performance.(2)Considering that distance between conditional distributions could serve as a more proper metric for domain discrepancy,we propose to perform domain adaptation by minimizing the discrepancy between the class conditional distributions between domains.Distance between class conditional distributions measures the inter-class distance and are immune to intra-class divergence,thus measuring domain distance more proper.A class-specific strategy is integrated into conventional MMD,leading to CMMD.Besides,pseudo labels are in both WMMD and CMMD.A discussion is presented over the impact of inaccurate pseudo labels on WMMD and CMMD based on which a more robust and efficient metric CWMMD is proposed.CWDAN is developed for domain adaptation tasks by integrating CWMMD into deep frameworks.CWMMD is verified based on various backbones on a wide range of domain adaptation tasks.Extensive experimental results show that CWDAN is susceptible of class weight bias and label noise,obtaining more stable and competitive domain adaptation performance.(3)To alleviate the negative transfer from irrelevant source samples,a PU learning based partial domain adaptation network is proposed.With the coming of the era of big data,a large-scale source dataset is often available for a new target dataset,resulting in a partial domain adaptation(PDA)tasks.Matching the whole source dataset and target dataset would suffer from severe negative transfer invoked by irrelevant source samples.The key issue lies in how to identify the irrelevant source samples thus estimating the transferability.A positive-unlabeled learning paradigm is first introduced in PDA tasks to estimate more accurate transferability for source samples.Experimental results on PDA tasks verified the effectiveness of the proposed transferability estimation strategy thus outperforming State-of-the-arts on a wide range of PDA tasks.(4)Given that adaptation performance are closely related to domain similarity,a weighted multi-source domain adaptation approach WMDAN is proposed.For a specific target domain,there are often more than one source domains are available,resulting in multi-source domain adaptation(MDA).Currently,MDA methods equally transfer different source domains.Diverse source data would not be sufficiently explored by ignoring the relationship between domain similarity and adaptation performance.Hence,we propose to guide the domain adaptation using the domain similarity and developed WMDAN for multi-source domain adaptation.Massive experiments are conducted on both object detection and grasp recognition to verify the efficiency of WMDAN.