Research On Discovery Of Object Relations In News Image Based On GNN

With the development of the internet,the internet has become a critical part of people's daily lives.The method of people to obtain news has been transformed from paper-based media to online news.Generally,online news includes news text and news image.News image directly reflect current events and news text describe current events.However,in order to obtain high news flow and cater readers attention,some media give the news a image which is irrelevant to news text.If such news are not detected in time,it will waste readers' time and energy,moreover cause the public to misunderstand the facts,mislead the trend of public opinion,and destroy the ecology of online news.Therefore,it has become a tricky problem that detect the consistence between news image and news text.Detecting the consistency of image and text by using the image caption method has attracted increasing attention in recent years However,there are many named entities in news text,and existing approaches are unable to directly generate named entities in the news image caption.It leads to a semantic gap between text and news image caption.Moreover,the existing methods lack the analysis of indirect relations between named entities.Therefore those approaches easily leads to relations error when generating news image caption.To generate the news image caption with named entities by analyzing the indirect relations between named entities,we research on discovery of object relations in news image based on graph neural network.The primary contributions of the proposed model are summarized as follows:Firstly,we propose the TopNews dataset,which contains news text and images,additionally includes news related articles,and their publication dates.Secondly,in order to accurately discover the relevance between named entities in news story,this article will construct a news knowledge graph to accurately reflect the relations between named entities in news story.Secondly,we develop the news knowledge graph by connecting named entities in the TopNews datasets.News knowledge graph precisely demonstrates the relations of names entities in a news story.Thirdly,we propose the NKD-GNN that aims to analyze the whole relations between the named entities in the news knowledge graph,and choose which named entities are related to news images.Finally,we conduct extensive experiments on the TopNews dataset.The result shows that our method can effectively detect the consistency between news image and text and our model outperforms the image-text matching model for news.