Heterogeneous Transfer Learning Between Image And Text Data

Transfer learning as a new machine learning paradigm has gained increasingattention lately. In situations where the training data in a target domain arenot su?cient to learn predictive models e?ectively, transfer learning leveragesauxiliary source data from other related source domains for learning. While mostof the existing works in this area only focused on using the source data with thesame structure as the target data, in this thesis, we push this boundary furtherby proposing a heterogeneous transfer learning framework for knowledge transferbetween text and images. We observe that for a target-domain class?cationproblem, a large number of annotated images can be found on the Web, whichcan serve as a bridge to transfer knowledge from text documents to images.In this theis, ?rst we present the contextual adversting as an example to showhow knowldge can be learnt and transferred between from one feature spaceto another. From the example, we will demonstrate that the feature mapping,which bridges the two di?erent feature spaces, is vital for heterogenous transferllearining. Then, we will show an more sophisticated way of building the featuremapping using collective matrix factorization (CMF) in order to discovery thelatent semantic space underlying the image and text domains. Experimentalresult on real world data show that heterogenous transfer learning between imageand text is both e?ective and e?cient in problems that labeled image date isinsu?cuent, while a large amount of labeled text data can be leveraged.