Research On Automatic Image Annotation Based On Transfer Learning And Convolutional Neural Network

With the advent of the mobile Internet era and the development of artificial intelligence technology,how to retrieve massive digital image resources on the Internet in a quickly and efficiently way is a very important and practical research topic.The purpose of automatic image annotation is to learn and establish the mapping relationship between image visual features and labels,and let the computer automatically annotate the images with their semantic content.It has been widely used in image retrieval,scene understanding and other fields.It has become research hotspot in the field of computer vision.Based on the deep convolutional neural network and the application of transfer learning technology in the field of computer vision,this paper discusses the automatic annotation of images and focuses on three aspects: the extraction of deep convolution features,multi-label annotation,and annotation model fusion.The main work and innovations of this paper are summarized as follows:(1)In order to solve the problem of network over-fitting caused by insufficient sample dataset,a convolutional neural network model based on transfer learning is proposed.The deep network is pre-trained by using large public image datasets on the Internet to obtain the general visual features of the image,and then Sigmoid cross entropy loss function is introduced to make the network model capable of the multi-label annotation task.Then the target dataset is used to fine-tune the network parameters to obtain more advanced and more abstracted features and the mapping of visual features to labels.(2)Aiming at the imbalance of label distribution in image annotation dataset,a multi-label smoothing unit based on label smoothing strategy is proposed.Based on the transfer learning model,the multi-label smoothing unit can automatically smooth the high-frequency labels in the dataset during the training process of deep convolutional neural network,so that the network appropriately raises the output value of the low-frequency labels.The validity of the multi-label smoothing unit for image annotation task is verified on the Corel5 K and IAPRTC12 datasets,and the annotation performance of low-frequency labels is improved.(3)In order to further improve the annotation performance,and effectively combine the different advantages of different models in the field of automatic image annotation,an image annotation model based on model fusion and multi-label selection algorithm is proposed.Based on transfer learning,this model uses pre-trained deep convolutional neural networks to extract features which are the input of discriminative models and nearest neighbor models respectively.Then introduce a multi-label selection algorithm to select the final set from the candidate sets generated by the two annotation models.The labels in final set are the result of the annotation in model fusion.The experimental results show that the proposed model can effectively integrate the advantages of different annotation models.And the average recall of the proposed model is significantly improved.