A Research On Graph Retrieval Based On Scene Graph

With the rapid development process of the Internet and information technologies,a huge amount of digital image data had been accumulated.,and these image data may contain much information of high value.it has become one of the most important research fields in this era to find out how to retrieve the image data needed by users from a huge amount of them on the Internet both quickly and accurately.This paper constructs a retrieval system of scene graphs,and realizes image retrieval based on semantics on the basis of scene graph retrieval.The retrieval system of scene graphs constructed in this paper includes recall,rough sorting and accurate sorting.In the phase of recalling,this paper offers a recalling algorism of scene graphs on the basis of the motif,with which we can recall scene graphs from the scene graph database that are relevant to the query scene graph,thereby getting a candidate set of scene graphs.This algorism mentioned above uses motifs with a simple and stable structure to represent scene graphs,thus the recalling of scene graphs can be achieved by retrieving motifs.Also,word2 vec has been introduced in this paper,we need the help of word vector to calculate the similarity in semantics between motifs.Meanwhile,three matching thresholds have been defined in this algorism to control its recalling requirements,and these matching thresholds can be adjusted according to different needs.In the phase of rough sorting,this paper comes up with a function,SGMD,to measure the matching distance between scene graphs,which enables us to quickly sort the candidate set of scene graphs that we get in the first phase.After this,we keep the results ranked at the top while the results in the bottom should be filtered to further downsize the candidate set of scene graphs,and then we can get preliminary retrieval results.In the phase of accurate sorting,we use the graph neural network to precisely sort the reduced candidate set to get the final retrieval result.The Graph neural network can be trained with data and learn to represent complex scene graphs in a low-dimension vector space.As a result,the complicated calculation of similarity between scene graph has turned to a simpler one of similarity of vector.The graph neural network designed in this paper mainly includes three parts: information coding part,information transmission part and information aggregation part.The function of the information encoding part is to encode the object information and the relationship information in the scene graph,and extract the high-order features from it,so that the graph neural network can better represent the information in the scene graph.The information transmission part allows information to be transmitted within and even between graphs,so that each object can learn first-order or even higher-order information around it.The information aggregation part aggregates the representation information of all objects so that the graph neural network can finally represent the structural information and semantic information of the scene graph with a vector.In the experimental part,through the comparison with other algorithms and visual analysis,it is proved that the scene graph retrieval system constructed in this paper has excellent retrieval performance.It can not only retrieve these scene graphs accurately matched with the query scene graph,but also can retrieve these scene graphs which is similar to the query scene graph,It can well meet the actual retrieval requirements of scene graphs.