Research On The Theory And Method Of Prototype Learning In Machine Learning

With the in-depth penetration of information technology in various fields,there exist many data in the real world.This can help data-driven algorithms in machine learning to obtain valuable knowledge.Meanwhile,high-dimension,excessive redundancy,and strong noise are inherent characteristics of these various and complex data.In order to eliminate redundancy,discover data structure,and improve data quality,prototype learning is developed.By finding a prototype set from the target set,we can reduce the data in the sample space,and then improve the efficiency and effectiveness of machine learning algorithms.Its feasibility has been proven in many applications.Thus,the research on prototype learning has important theoretical significance and application value.To improve the diversity,interpretability,and compatibility of prototypes,we review the existing literature,and develop novel prototype learning methods with theoretical analysis on metric learning,mutual exclusion,quality evaluation,knowledge transfer,model optimization,and so on.Meanwhile,some related applications are exploited,such as prototype exclusive/structural selection for representation learning and prototype generation for task assignment.The main contributions and creative research results include:? We develop a Prototype Selection induced Local Feature Aggregation(Pro LFA)method.It is capable of generating compact yet interpretable representations by selecting representative and exclusive prototypes from numerous local descriptors,under an exclusivity constraint.Meanwhile,to improve the discriminability of aggregated results,the procedures of codebook construction,encoding,and pooling are integrated into a unified prototype selection framework.Finally,the proposed method can achieve up at least to 1.94% and 1.69% gains in classification and retrieval over currently available alternatives about feature aggregation.? We propose a prototype generation method for task assignment.To assign a target set to a prototype set,we perform prototype generation and task assignment synchronously by minimizing the assignment cost between the two sets in the lowdimensional space.Furthermore,by boosting the consistency between the expected and practical assignment solution,our model can deal flexibly with the unsupervised,semi-supervised,and fully supervised scenarios.An alternating iterative algorithm is presented to solve this model.Finally,the proposed method improves the performances in motion segmentation,activities recognition,and scene categorization over the strongest competitor by 1.64%,2.39%,and 0.61%,respectively.? We develop a 1-norm induced Prototype Selection(1-Pro Se)method.It describes the first unified framework to jointly pursue metric learning and prototype selection.Specifically,to characterize the pairwise similarity in the learned sparse representation space,a 1-norm metric is applied for robust selection due to the outliers and uncertain distribution of the data.Furthermore,our method is extended to support online prototype selection by using already obtained prototypes and newly arrived data.Experimental results on some applications such as video summarization and motion segmentation demonstrate that the proposed method is considerably superior to the state-of-the-art methods in the prototype selection.? We propose a Self-supervised Deep Low-rank Assignment(SDLA)model for robust prototype selection.It dynamically integrates the assignment model with deep representation learning based on a denoising autoencoder.Consequently,the dissimilarity metrics on target set are self-refined in an embedding space with supervision from prototypes.While working on this metric space,similar samples tend to select the same prototypes via a low-rank assignment model,thus guaranteeing diverse prototype selection.Experimental results on video summarization,text clustering,and image classification demonstrate nearly 3.72% gains are achieved for the proposed method than DS3.? We propose a knowledge transfer based prototype generation method.It introduces a domain disjoint auxiliary dataset in unsupervised prototype generation for the target set.Then by minimizing the coding cost and maximizing the structure consistency of their prototypes,a Hierarchical Prototype Generation(HPG)model is developed.Consequently,the visual and semantic super-prototypes of other large-scale datasets are shared and transferred to the target set,so as to significantly improve the discriminability and interpretability of its prototypes.Experimental results in unsupervised classification show that the proposed method achieves up to5.6% to 30.3% gains when compared with other existing methods of the same type.