Research And Application Of Class-Incremental Learning Algorithm

The classification models based on Deep Neural Networks(DNN)are quite effective in many tasks.However,for a model trained on a certain task,if it is made to learn on some new-classes data,the model's predictive ability on the old classes will be significantly reduced,i.e.,the model suffers from Catastrophic Forgetting.ClassIncremental Learning(CIL)aims at overcoming Catastrophic Forgetting,it requires that the classification model can dynamically learn new-classes data that appear at different times,provided not using or only using a small number of old-classes data during the training process.After finishing training,the model needs to be able to effectively classify all categories that have been seen so far.To adapt the deep model to the mode of CIL,overcome Catastrophic Forgetting and improve the classification accuracy,this dissertation proposes an algorithm named“Class-Incremental Learning via Knowledge Distillation and Impartial prediction(KDIP)”.Three sub-algorithms are designed to address the key issues of the selection strategy for exemplars,retention of old knowledge,and overcoming prediction bias.The main contents of the algorithm are as follows.(1)The proposed “Exemplar selection algorithm based on the principle of maximum interference priority” draws on the idea of Hard Example Mining.The samples selected for the exemplar set are the most severely forgotten(i.e.the maximum interference)samples before and after the model update,and the repeated learning of these samples helps to overcome Catastrophic Forgetting to the greatest extent.(2)The proposed “Incremental training algorithm based on knowledge distillation”fully mines the knowledge in the old model and transfers it into the new model through the Knowledge Distillation method.Specifically,cosine similarity is used to measure the prediction results of the old and new models before knowledge distillation.For feature vectors extracted by the new and the old model,respectively,the similarity of them was maximized to obtain the invariance of model representation.By increasing the distance between the training samples and the embedding of other classes in the feature space,the probability of misclassification is reduced.(3)The proposed “Unbiased prediction algorithm based on momentum effect removal” aims at addressing the prediction bias problem in CIL.By calculating the pure-bias effect under the condition of null input and removing it in the test phase,unbiased prediction results are obtained,thus improving the classification accuracy of unbalanced data sets used in the incremental training phase.The KDIP algorithm proposed in this paper is experimented on public datasets commonly used in the field and a real radar signal dataset,respectively.The experimental results on three public datasets,CIFAR100,Image Net-Subset and Image Net-Full,show that the KDIP algorithm achieves higher classification accuracy while following the same benchmarks compared to the baseline and SOTA algorithms in recent years.Experimental results on the real radar signal dataset show that it is practical to apply the KDIP algorithm in the field of radar individual recognition to accomplish the training task of a radar signal classifier in the mode of CIL.When only a very few samples in each class are saved for subsequent incremental training,the model trained with the KDIP algorithm has very little degradation in classification performance compared to the expert model,and saves significant time and storage resources.