Research On Approaches To Knowledge Transfer In Domain Adaptation And Zero-shot Learning

In machine learning,the trained model may be overfitting when there is few labeled data and hence generalized performance is bad,which makes the trained model cannot meet the reality.In many realistic scenarios,collecting a large amount of the labeled data typically requires huge human,material and financial costs.As the commercial competition is more and more fierce,it is not feasible to enlarge labeled data in brute force and hence one explores other solving schemes.Knowledge transfer is one of the most popular schemes to solve this problem.It aims to leverage the useful knowledge in relevant domains to assist the modeling in target domain so as to avoid the overfitting phenomenon in modeling on small samples.This thesis involves two directions for knowledge transfer,that is,domain adaptation and zero-shot learning.Domain adaptation can be categorized as homogeneous and heterogeneous aspect.The content is as follows.Firstly,for homogeneous domain adaptation,this thesis proposes approaches of homologous component analysis and Log Det metric based second-order statistics alignment for single source case so as to deal with insufficient distribution alignment problem in previous works.For the former,this thesis proposes the notion of homology and extracts homologous components between domains which are incorporated into the maximum mean discrepancy metric in order to enhance the effect of distribution alignment.Furthermore,this thesis analyzes the generalized error bound for semi-supervised case.For the latter,this thesis theoretically proves that some scaled Log Det metric is more suitable for second-order statistics alignment than Frobenius.This kind of metric makes the second-order statistics alignment more compact and hence enhances the effect of distribution alignment.Moreover,this thesis extends Log Det model in single source case to multi-source case by means of weighted Karcher mean,takes full advantage of information on second-order statistics and effectively solves the problem of joint knowledge using multiple different domains.Secondly,for heterogeneous domain adaptation,this thesis proposes a transferable multiple subspaces discovering based method for disentangling heterogeneity,which takes full consideration for transferability,diversity and heterogeneity of the knowledge so as to enhance the positive effect of knowledge transfer.Transferability relies on the distribution alignment in the homogeneous domain adaptation,which can be realized by minimizing maximum mean discrepancy.The diversity is achieved by borrowing clustering techniques,which makes heterogeneous knowledge diversely distributed over multiple subspaces.This thesis shows respect for the heterogeneity by proposing the Gauss-Rayleigh cross similarity,which can be applied to manifold regularization so as to dig out similar knowledge between domains.Combining these three aspects,this thesis solves the existing problems in previous works on one common latent subspace towards the knowledge transfer,which is lack of diversity and doesn't take full consideration for heterogeneity.Finally,for zero-shot learning,this thesis pays attention to transductive setting.Thesis proposes a framework of adversarial strategy and instantiates it as the bidirectional projection adversarial learning approach.It is a game-based method.There are two players,that is,projector and classifier.Projector expects to obtain a projection with good semantic preserving property while classifier expects to achieve high recognition accuracy.By successive competition,the Nash equilibrium between domain shift problem and precise classification is ultimately realized.Here,the adversarial meaning comes from the parallel design for loss functions on projector and semantic compatibility among them and loss function in classifier.Besides,this thesis also provides an explanation for the adversarial strategy.