| The essence of the image annotation problem is the visual-to-language problem.Simply put,the image can generate natural language sentences that describe the image content.This is a very easy task for humans,but it cannot but be said to be a challenging task in the field of computer vision.Because image tagging issues require "translation" between two different forms of information.With the rapid development of deep learning,the combination of Deep Convolutional Neural Network and Recurrent Neural Network has made great progress in image annotation.Due to the success of this method,the research on image tagging based on this method has been rapidly heated up.Now many methods in this field have reached a good level,but these methods seem to be close to human beings,but the gap is still large.This article takes the multimodal Recurrent Neural Network(namely m-RNN)model proposed by Baidu Research Institute as the research object.The main research work consists of the following two parts: The first part analyzes why the model can be applied to the image annotation problem.It succeeded and analyzed the reason why the model had very poor results on some image annotation problems.For example,the generated description sentence was very poorly correlated with the image.In the second part,this paper proposes the use of migration learning methods,using well-known convolutional neural networks(such as vgg,google-net)to fine-tune(Fine-tuning)to generate image feature vectors,and use better semantic concepts to Expressing the eigenvectors of an image and then inputting it into a recurrent neural network will improve the accuracy rate compared to previous ones.The content of the image is not only complex and abstract,but also has the characteristics of vagueness and multi-sense in the semantic concept.Therefore,this article mainly improves the work of image tagging feature learning and semantic learning,and realizes automatic image tagging and improves Performance,this article finally based on the improved model,using the open source deep learning framework TensorFlow,completed a simple project based on the input image,producing a natural description language. |
