Research On Image Metaphor Understanding

Due to its powerful visual description capabilities,images can contain very rich semantic information.However,the current research work on image understanding mainly focuses on understanding the literal meaning of images.In metaphoric images,the semantic information that the image wants to convey is not limited to the literal meaning but also the implied meaning behind it.Image metaphor understanding is to model the metaphorical phenomenon in the image,so that the computer can also form a coherent and appropriate understanding of the metaphorical image,and then fully and correctly understand the meaning of the image.In order to achieve this goal,this study imitates the process of human understanding of metaphors and proposes a three-stage research framework:(1)Literal concept detection,which aims to detect concepts that appear explicitly in images;(2)Implied concept inference,which aims to infer the appropriate implied concept for each detected metaphorical concept;(3)Metaphor description generation,which aims to generate an understandable image description based on the inferred implied concept and other basic information in the image.Focusing on the second stage,this research has carried out the following two aspects:(1)Metaphor dictionary construction based on search and entity recognition.There are two difficulties in metaphor dictionary construction.One is that the manual collection method is time-consuming and labor-intensive.The other is that the complex relationship between the literal concept and the implied concept is difficult to capture.In view of the above difficulties,this study uses keyword search to obtain a large number of sentences containing the implied concepts,and after a small amount of manual annotations,the original problem is converted into a problem of implied entity recognition.It has been experimentally verified that this method can construct a metaphor dictionary for metaphorical concepts,and supports new metaphorical concepts to be added to the dictionary without the need for retraining to update the dictionary.(2)Implied concepts inference based on metaphor dictionary and compatibility analysis.This study provides a two-step solution for inferring the implied concepts: first,construct a reference graph;then,activate the implied concepts according to the reference graph.The former uses expert knowledge base and word co-occurrence to introduce a lot of external knowledge,which enriches the connection between concepts;the latter uses the compatibility of the overall concept in the reference graph to determine the activation object.Experimental results show that it is very necessary to consider the overall compatibility of multiple concepts simultaneously when inferring implied concepts,and this method can effectively infer the implied concept of each metaphorical concepts in images.