| Generating a natural language description from an image is an emerging interdisciplinary problem at the intersection of computer vision and natural language processing,which forms the technical foundation of many important applications,such as image retrieve,childhood education,and aid for visually impaired people to perceive surrounding visual content.With the dramatic increase in computer hardware performance,massive memory space and GPU's superior computing power,deep neural networks have rapidly emerged,and breakthroughs have been made in the areas of picture classification,target detection,and target segmentation,laying a foundation for the study of image description algorithms.In this paper,the deep neural network is used to design and model the image description task,and the web server-side program is implemented.The corresponding text description can be automatically generated for any image.This paper proposes an image description algorithm based on regional attention.First,the candidate regions are generated using the RPN in the Faster R-CNN model,then the RoI pooling layer is used to extract the picture features from the last shared convolutional layer,and finally the image features are provided to the LSTM to generate the description statement through the attention mechanism.By visualizing the description generation process,the attention changes in the description generation process are shown.Experiments show that the algorithm has reached the state of the art performance under multiple indicators.Based on this,a server-side program for image description generation is built based on Flask.The user can upload a picture through the browser,view the machine description and give a rating.The administrator can view the user's uploaded picture as well as the given score to the description.Besides,the distribution of attention during the description generation process is also provided,which is convenient for checking the performance of the model and further improving the algorithm. |
