Research On Unsupervised Domain Adaption Methods And Cross-modal Vision Applications

Unsupervised domain adaptation is an important research field in weakly super-vised learning based on the idea of transfer learning.The training of deep neural net-works is very dependent on data-driven,however,for many prediction tasks,especially dense-estimate tasks,the labeling process is usually expensive or labor-intensive,and it is difficult to collect large-scale and diverse datasets;And due to the domain shift between different modalities,when training a model on one domain and testing this model on another domain,the performance of the model will decrease significantly.This restricts the model from training on large-scale labeled datasets,and then transfers the model to new datasets from different fields.To overcome these problems,unsu-pervised domain adaptation provides an effective solution to reduce the domain shift between labeled source data and unlabeled target data.The methods on unsupervised domain adaptive can be roughly divided into two categories:non-adversarial learn-ing methods and adversarial learning methods.Traditional non-adversarial learning methods usually use a metric as the objective function to measure and minimize the discrepancy of high-dimensional features between the source domain and the target domain.This method is effective in simple distribution,but the performance is often poor in the complex distribution.Although generative adversarial learning methods have achieved the improvements,but existed generative adversarial learning methods often directly align the source and target domains,ignoring the intermediate distribu-tions between them.and they usually perform on the global features,which causes redundancy.To solve these problems,we use generative adversarial learning methods to propose a new unsupervised domain adaptation method based on bidirectional align-ment and a new unsupervised domain adaptation based on attention mechanism.At the same time,the effectiveness and generalization performance of the proposed method-s are verified on different cross-modal image datasets and different application tasks.The main contributions of this paper are summarized as follows:1.We propose a new unsupervised domain adaptive method based on bidirectional alignment.By image-to-image translator,the intermediate potential distributions be-tween the source domain and the target domain are generated bidirectionally,they are used as a bridge between the two domains.And the inter-domain consistency and intra-domain consistency learning strategies are proposed to align the intermediate domain generated by the source domain and the source domain,and the intermediate domain generated by the source domain and the target domain,and vice versa.The method mainly uses the generative adversarial learning method to carry out bidirectional unsu-pervised domain adaptation,eliminates domain differences from the overall style,and makes the domain distribution consistent.2.We further propose a new unsupervised domain adaptive method based on an attention mechanism.The self-attention map is generated through the dual attention mechanism,and the consistency learning based on the attention mechanism is realized by using generative adversarial learning,so that the model has both global consistency and local consistency,especially the local effective features.More effective domain adaptation is achieved by transferring most of the discriminative local features,so that the model can obtain good performance even in small-object or fine-grained recognition tasks.3.We apply the proposed unsupervised domain adaption method based on bidirec-tional alignment to the cross-modalities(photo-caricature)face attribute classification task.For this task,we also constructed a caricature attribute dataset,namely Web Cari A,which contains 50 inherent attributes of human faces to facilitate the study of carica-ture attributes.And the proposed unsupervised domain adaptive method based on the attention mechanism is applied to the cross-modalities(T₁—T₂)brain tumor MRI seg-mentation task.For this task,we also designed a dual-attention depth encoder-decoder model.We conducted expensive experiments on the Web Cari A dataset and the bra TS multi-modal brain tumor dataset to prove the effectiveness of the proposed method,and its performance is better than the state-of-the-art methods.