Robust Classification And Ranking Methods Based On Metric Learning And Knowledge Transfer

Classification and ranking are both supervised learning tasks for pattern discrimination.Based on the essential demands of general supervised learning applications,the study of classification and ranking models should consider three aspects.First,the data distributions usually have com-plex nonlinear geometric structures.Since a space topology can be characterized by a distance metric function,we need to learn a generic nonlinear distance function to restore the general struc-tures of data distributions.Second,unreliable noisy samples commonly exist in real-world data.A model needs to distinguish the reliable data patterns and reflect the global distributions of data for learning robustness.Third,the rapidly arising new domains usually lack the available labeled training data.To handle the case where the training and the target data have different distribution-s,a model needs to extract from the training data the knowledge adaptable to the target data,for an effective cross-domain knowledge transfer.Based on the above analysis,we conduct our study in three aspects:exploring the intrinsic geometric structures of data,acquiring the model robustness,and obtaining the adaptable knowl-edge transfer.These aspects essentially focus on the three basic principles of machine learning respectively,the effectiveness,the robustness and the adaptability of modeling,which are mutual-ly collaborative and complementary.Specifically,the effectiveness emphasizes the exploration of the local nonlinear structures of data distributions,for an accurate fitness of the data.The robust-ness emphasizes the capture of the global structures of data distributions,for a holistic fitness of the data.The adaptability emphasizes the capability of extracting the shared knowledge between domains,for the exploration of unknown areas.Based on an extensive review of the previous works,we utilize the intrinsic interactions among the above three aspects,and propose sever-al novel ranking and classification models,for a joint optimization of effective learning,robust learning and adaptable knowledge transfer.The main contributions of this thesis are:1.We propose a structural learning to rank algorithm based on Bregman distance functions learning.The Bregman distance is a general family of distance functions with generic and flexible nonlinear forms.Due to the nonlinear modeling capability of Bregman metric,the proposed algorithm learns a data-driven Bregman distance function to restore the general structures of data distributions.On the other hand,we formulate the problem of learning Bregman distance to rank based on structural learning framework,so as to utilize the struc-tural information of the ranking lists for a ranking-task-driven model.The proposed model is a unified framework of distance metric learning and learning to rank.By joint modeling the nonlinear data patterns of data and the ranking structures,the proposed method performs a joint optimization of data adaptation and task adaptation for the ranking model.2.We propose a both effective and robust classification framework,Self-Paced Boost Learning(SPBL).We reveal and utilize the consistency and complementarity of boosting and self-paced learning schemes,where the former learns by effective model selection and the latter learns by robust sample selection.By learning in a joint manner from weak models to a strong model and from easy samples to hard samples,the SPBL model is capable of capturing the intrinsic interclass discriminative patterns while ensuring the reliability of the samples involved in learning.Through this,the proposed SPBL obtains a simultaneous enhancement of learning effectiveness and robustness for classification.3.We study zero-shot learning(ZSL)from the perspective of knowledge transfer,and propose the BZ-SCR model:boosted zero-shot learning with semantic correlation regularization.The model integrates a novel semantic-based regularization SCR,to constrain the classifier output to be consistent with the inter-class semantic correlations.With SCR,the model is capable of capturing the shared feature and semantic patterns between the seen and the target classes.Formulated by SCR-embedded boosting optimization with self-controlled sample selection,the BZ-SCR performs a joint enhancement of learning effectiveness,robustness,and cross-semantics adaptation,and thus achieves sufficient knowledge transfer for ZSL.