Effect Of Class Imbalance On Transfer Learning

Recently,transfer learning has achieved great progress,and domain adaptation algorithms based on deep learning have also received increasing attention.Transfer learning aims to transfer the learned knowledge from one task to another new similar task,which effectively alleviates the problem of data labelling in deep learning.Many state-of-the-art domain adaptation algorithms aim to align the conditional probability distribution of target features to that of source features.Therefore,the class imbalance has a significant effect on the performance of these algorithms.This thesis focuses on the topic of the effect of class imbalance on transfer learning and the main contributions can be summarized as follows:(1)To investigate the effect of source category distribution on the developed transfer learning algorithms,the commonly used imbalanced processing mechanisms in various classification algorithms have been employed to several transfer learning algorithms.Various imbalanced versions of transfer learning algorithms are compared and experiments show that oversampling and weighted random sampling are more suitable for transfer learning.(2)The difference of category distribution between source domain and target domain may have great impact on the performance of transfer learning,and this topic has been investigated with the assumption that the category distribution of target domain is known before the training.By resampling before training,we can obtain various new source and target datasets with any required category distribution.Then,symmetric Kullback-Leibler divergence is employed to measure the category distribution difference between two domains after training.Experiments show that the performance of transfer learning is better when the category distribution difference between two domains is smaller.(3)For the open set domain adaptation,two novel algorithms,including OSDA-EM and STA-EM algorithms,are proposed.By forcing the classifier to output the probability of 0.5 on the unknown class for any input target sample,OSDA-EMcould separate the unknown and known classes well in the target domain.Experiments show that the proposed OSDA-EM performs better than several existing algorithms on open set domain adaptation.STA-EM introduces an additional CategoryDiversity Loss during training,and achieves better performance,as verified by extensive experiments.