| In recent years,deep learning technology has made great success.One reason is the annotation and disclosure of large amounts of high-quality data.However,in many realistic scenarios,it is still very difficult to acquire large-scale tag data sets.How to learn knowledge from a small amount of sample data is a challenge in the era of deep learning.In view of this problem,small sample learning and self-supervised learning are mainly two research directions.Mainstream small sample learning methods use supervised learning to obtain prior knowledge from relevant auxiliary data sets to compensate for the deviation caused by insufficient training data.Re-cently,some self-supervised methods show great advantages in learning without a label,which creates conditions for eliminating small sample learning dependence on auxiliary labeled data set..In this paper,we study the small sample learning tech-nology based on self-supervision method,propose a noise filtering method based on self-attention mechanism for the noise problem in negative samples,and pro-pose a dynamic momentum method for the siamese network structure with partial parameter sharing.The main work of this paper is as follows:（1）Aiming at the problem of negative sample noise in self-supervised learning using positive and negative sample contrast,a noise filtering method based on self-attention mechanism was proposed.The self-supervised learning method based on the comparison of positive and negative samples learns knowledge by identifying positive and negative sample pairs.The positive sample is the enhancement of the current training sample,and the negative sample is randomly sampled from the data set.Therefore,there are some noise data of the same class as the positive sample in the negative sample.The existence of noise data will limit the improvement of model performance,and the model is more sensitive to noise in small sample scenarios.To solve the problem of noise in negative samples,this paper proposes a noise filtering method based on self-attention mechanism,and implements a self-supervised small sample learning model which can filter noise based on Mo Co[1].Experiments show that the proposed noise filtering method can effectively improve the classification accuracy of the model on small sample data sets.Compared with traditional unsupervised small sample learning methods,the classification accuracy of self-supervised small sample learning methods is significantly improved,and even exceeds some supervised learning models.（2）A dynamic momentum updating method is proposed for siamese networks with partially shared parameters in self-supervised learning.Siamese networks play an important role in self-supervised representation learning,and the momentum encoder structure,which shares some parameters,can enhance model performance.The current self-supervised methods using momentum encoder structure mainly use constant ratio to update network parameters.In this paper,it is considered that the ratio of momentum encoder parameters update should be gradually adjusted with the training process to adapt to the improvement of encoder performance.To solve this problem,this paper proposes a dynamic momentum updating method and implements a self-supervised small sample learning model based on Byol[2].The experimental results show that the self-supervised method with zero negative sam-ples has better performance in small sample learning,and the dynamic momentum update method can improve the classification accuracy of the model to a certain extent.In this paper,the optimized model achieves the best classification accuracy of unsupervised small sample learning on the mini Image Net data set. |
