Research On Learning With Noisy Labels Based On Label Distribution

In recent years,deep neural networks have achieved remarkable performance in many tasks,such as speech recognition,image classification,natural language processing,etc.With the advent of DNNs,the need for more labeled data makes it inevitable that not all examples will have high-quality labels,especially for large datasets generated by network crowdsourcing or automatic labeling methods.In other words,the data labels of many large datasets are inevitably noisy.It is very difficult to train a deep network that is robust to noise because the deep network can totally fit these noisy labels.In addition,since deep networks are very easy to fit the noisy labels in the data set,the possible presence of biased data labels in the data set will also lead to the degradation of the model performance,and even make the model make predictions that are contrary to social safety.In this paper,under the assumption that the labels provided in the dataset are unreliable,aiming at the problem of how to avoid the problem of overfitting the noise labels in the deep network,the following two aspects are mainly done:Inspired by label distribution learning,this paper proposes a novel approach to deal with label noise,named Feature-Induced Label Distribution(FILD).FILD uses the recovered label distribution to replace the original noise labels to reduce the influence of noise labels.Specifically,FILD uses topological information in the feature space to recover the label distribution for the sample,and the feature representation of the sample is adjusted by the label distribution recovered by fitting the predictive model.The training process is divided into two stages: in the first stage,we generates a neighbor graph for the training samples through the feature representation of the samples,and the weights of the edges in the neighbor graph are calculated by the Euclidean distance between the sample features and the corresponding loss value of the network.Since samples with low loss values are usually clean samples,the labels of these samples should remain unchanged,while the labels of samples with high loss values should be more influenced by the labels of samples with low loss values in their neighbors.In the second stage,the network is trained using the recovered label distribution.The two stages are alternately optimized to reach the final convergent result.Considering that the process of recovering the label distribution from the feature representation of training samples is a process of label enhancement,we combines the most recent method in the field of label enhancement,Variational Label Enhancement(VLE),to generate label distributions for samples.In order to improve the quality of the generated label distribution,the Beta mixture model is used to model the loss distribution on the basis of variational label enhancement,and the posterior probability that the sample is a noise sample is obtained according to the Beta mixture model and the loss value of the sample.The original labels and network predictions are combined according to the posterior probability that the sample is a noise sample.The new method is named Variational Lebel Correction(VLC).In this paper,extensive experiments are done on the CIFAR-10,CIFAR-100 and Clothing1 M datasets,and the experimental results show that FILD has obvious superiority compared with VLC and other baseline methods.This paper is divided into five chapters.The first chapter introduces the research background.The second chapter introduces some related works about learning with noisy labels.The third chapter introduces the feature-induced label distribution FILD.The fourth chapter introduces the variational label correction VLC.And the fifth chapter summarizes the full work.