Semantic Label Encoding For Characterizing Implicit Correlation In Deep Representation Learning

As an important branch of machine learning,deep representation learning plays an increasingly important role in various data processing and analysis tasks with the rapid development of big data and Internet technology.Its main purpose is to use deep neural network models to learn a higher-level representation from raw data to improve the performance of machine learning,and to enable machines to more intelligently understand and process various types of data.However,most previous deep representation learning methods were based on supervised training with one-hot label encoding,which had limitations in the semantic analysis of labels and the extraction of relevant relationships.When training data has problems such as insufficient labels and samples,label imbalance,sparse data,and complex label relationships,the existing label encoding methods are difficult to effectively alleviate these issues.Therefore,this thesis thoroughly studied the label encoding problem,and based on deep learning models,proposed three semantic label encoding methods that can effectively improve deep representation learning: hybrid label encoding,dynamic label relationship encoding,and learnable label encoding.The research of these three methods focused on semantic label encoding and label relationship encoding,and they were respectively encoded based on local or global label relationships,with the ultimate goal of using an encoding method that characterizes the implicit association between labels to improve the feature representation ability of deep models.The specific research work is as follows:(1)A novel hybrid feature enhancement method is proposed to address problems such as insufficient labels and data,noisy samples,and multi-label classification that may occur in deep model training.The method is based on hybrid label encoding,designed in a high-dimensional hidden representation through a generalized linear combination,which can greatly increase the size of data and labels.At the same time,this method has properties similar to feature regularization,which can significantly improve the model’s generalization ability and have superior robustness to data sparsity.In addition,by combining hybrid features with noise injection,a noise feature mixing method can be implemented to further improve the model’s robustness to sample noise.This method not only performs well in single-label classification tasks but also effectively enhances the model’s representation learning performance in multilabel image classification tasks.(2)A dynamic label relationship regularization method for deep representation learning is proposed to address problems such as label imbalance and data sparsity that may exist in largescale datasets.The method is based on dynamic label relationship encoding,which can incorporate semantic correlations between categories into feature learning,and use online center constraints to reduce feature variations between samples of the same category.At the same time,by combining this method with the hybrid label encoding method,a novel label relationship enhancement method is achieved,which further promotes deep models to obtain excellent representation learning performance.In addition,since the potential label association relationship is used to capture the correlation between different categories,this method can make the deep learning model more robust to sparse and long-tailed data distributions.(3)A label encoding auxiliary regularization term and a novel learnable parameter optimization-based label encoding method are proposed to address problems such as complex label relationships and category differences that may exist in different datasets.The method mines label encoding with category relationships through learnable parameters and uses them as regularization terms to assist the model in deep representation learning.In addition,this method introduces the idea of learnable label encoding into supervised classification tasks for the first time,which can not only effectively constrain the model to capture complex semantic relationships between different categories but also improve the representation of minority classes through the constraints between majority and minority classes.Furthermore,by incorporating the proposed label encoding auxiliary regularization term and the proposed learnable label encoding method,a novel end-to-end deep representation learning framework is achieved,which can achieve state-of-the-art performance on different datasets.Overall,this thesis proposes three effective semantic label encoding methods that can improve deep representation learning performance,and provides a valuable reference for future research in this field.