| In recent years,computer vision research which is based on artificial neural networks has achieved remarkable results.On the one hand,it is due to the continuous optimization of network structure,on the other hand,thanks to the improvement of sampling technology,the scale and comprehensiveness of datasets are also leaped forward,which is equivalently crucial.However,in real-world application scenarios,there are often differences between different datasets due to various factors such as the environment,equipment,and methods of sampling.The use of plain neural network-based models can no longer establish the connection between different data domains,which affects the generalization ability and the utilization of large-scale datasets of the models.Domain-adaptive transfer learning techniques use labeled multi-source domain information to filter and generate domaininvariant features by constructing domain relationships on a high-dimensional space,thus minimizing the discrepancies between source and target domains and adaptively improving the performance of the model on all data domains in the end.Existing feature-migration learning models are mainly based on the metric of Euclidean distance or adversarial neural networks,which attempt to search for domain-invariant features in a high-dimensional feature space by direct computation or adversarial learning between generators and discriminators.However,neither of them can take the influence of the category weights of data samples on the high-dimensional space into account,which increases the bias between the minority classes and the majority classes of samples,and both of their measurability degrade when the samples between different domains are too discrepant,i.e.,their support sets do not overlap.These factors eventually lead to the extracted features not having domain invariance knowledge,i.e.,causing negative transfer.In this paper,we propose a transfer learning model for class-imbalance based on the optimal transport theory.First,in order to utilize label information more completely and avoid metric degradation,this model uniformly maps the source and target domain datasets into a space based on the joint probability of feature-label distribution,and uses the earthmover distance to contain the domain-invariant features between multiple source and target domains.To address the imbalance of categories between samples in different domains,this model corrects the space of joint probability distribution using pre-estimated class ratios to encourage the transport of probabilistic masses of both minor category samples and principal component features,and obtains the optimal coupling matrix and optimal transport distance as the rebalanced metric for classification by the classifier networks in subsequent processes.Finally,to minimize the theoretical upper bound of the classification error,this model additionally measures the distance between multiple source domains and applies multi-classifier model to make full use of the information within the source domains to construct domain invariant features,which further enhances the generalization ability of the model.To verify the effectiveness and applicability of this model on the transfer learning task of multi-source domain adaption,three composite image classification datasets of different sizes and topics,Digit-Five,Office-Caltech,and Office-Home,are selected and adapted for13 experiments,and also compared with several models without knowledge transfer or without considering the category proportions.The average accuracy of 94.03%,98.37%,and 74.54% were achieved on the three datasets,respectively,which were higher than the original model and the comparative models.In order to further analyze and quantitatively evaluate the performance improvement of this model,ablation study and parameter sensitivity experiments are also designed in this paper,and the class clustering images of TSNE feature distribution before and after transfer learning are drawn to conform the contribution of each module on the general performance. |
