| Dense image caption is a task based on two research fields of Natural Language Processing and Computer Vision,and it is a cross-modal subject from image to language.In the dense image caption task,there is the problem of incomplete deep semantic mining,that is,the image features extracted by the neural network contain less semantic information than humans perceive from the image.At present,how to identify objects in a large number of images and describe the logical and semantic relationship between objects has become a difficult problem to be solved.In this paper,a deep neural network is used to extract image features to fully mine the hidden visual semantic information in the feature matrix,and combined with related algorithms to achieve dense image caption.First,this paper studies a gated dense caption generation method based on regional image patches.The method uses deep neural network to obtain image visual features,and proposes an inter-regional information extraction module to extract the contextual relationship between image blocks more completely.In addition,in order to better combine the current description image region information,contextual information between regions and global information,this method proposes a semantic gating mechanism,which can realize the adaptive integration of the three kinds of information.This paper verifies the effectiveness of the proposed method through experiments.The dense image description text obtained by this method not only highlights the context information,but also ensures the correctness of the current region image block description.Second,this paper proposes a method for generating dense image caption based on a multi-attention structure.This method effectively integrates image features of multiple resolution scales through a multi-scale feature loop fusion mechanism,and further integrates image visual features,thereby providing the back-end network with a visual feature matrix containing high-quality geometric details and semantic information.Moreover,this method designs a multi-branch spatial step-by-step attention module at the decoding end,so that the output caption text of the network model can reflect a certain target spatial position relationship.On the Visual Genome dataset,the method proposed in this paper achieves the state-of-the-art experimental results,and the generated dense descriptions are further in line with human language habits.Finally,this paper implements an image retrieval task based on dense image caption.In order to achieve the purpose of retrieval through image area blocks,this method processes the retrieval data set to obtain a new retrieval area database.At the same time,in order to make the retrieval result set consistent with the retrieval image in both visual and semantic aspects,the retrieval basis of this method is composed of the image visual feature and the corresponding description text vector.The method proposed in this paper has obtained better retrieval results on the Visual Genome test set. |
