Research And Implementation Of Scene Graph Retrieval Method Based On Graph Theory

Image retrieval has always been an important task in computer and vision.With the gradual maturity of deep learning methods and scene graph generation technology,scene graph-based graph retrieval technology has also received research and attention.Scene graph retrieval technology refers to the technology that achieves the goal of image retrieval by reversing the matching result of scene graph to the matching result of image.In addition,most of the scene graph datasets used in the current research are scene graph generation datasets,and few scene graphs similarity data labels,which brings great obstacles to the research on scene graph matching.In this thesis,the scene graph matching dataset was constructed by perturbation,and a scene graph matching model based on graph theory was built based on scene graph fingerprint vector and graph neural network.An in-depth study was carried out from the following aspects:(1)Generation of scene graph matching dataset: Based on the currently public scene graph classification dataset,this thesis interferes with the original scene graph from both structural and semantic perspectives through perturbation algorithm,generates perturbation scene graph and constructs a scene graph pair with controllable similarity.In this way,similar graph matching positive samples and dissimilar negative samples can be generated according to the requirements.Based on the data of scene graph generated by disturbance,the study of scene graph matching can be carried out.(2)Research on the Fingerprint Vector Algorithm of Scene Graph: Human fingerprints are unique,and a person can be uniquely identified and designated through fingerprints.The same ideas,by the same token,the vector diagram fingerprint by fingerprint vector algorithm will each figure data into a high dimensional vector,the only figure data and the similarity between the positive correlation between the distance between the vector and relationship,so graph matching problem was converted to vector to calculate the distance between the problems,reduce the complexity of the matching task and the time,improve the computational efficiency.In this thesis,the node degree is taken as the threshold value to extract the core subgraph of the scene graph,and the feature information and structure information of the core subgraph are extracted,calculated and transformed into vectors using vector generation tools.In the process of scene graph retrieval,the scene graph fingerprint vector can be used for screening calculation.(3)Graph matching neural network model: Due to the characteristics of non-Euclidian space of graph data,traditional convolutional neural network cannot operate on graph data.The emergence of graph neural network is to solve the above problems,so that graph structure data can participate in the convolution operation.In this thesis,a double-flow graph matching neural network model is designed.Two graph neural networks are used to transform a pair of input scene graphs into graph embedding vectors,and the matching degree of scene graphs is measured according to the similarity of graph vectors.Finally,in the experimental part,this thesis tests the scene graph fingerprint vector and the neural network model of picture matching on the proposed scene graph matching dataset.Based on the comparison of the experimental data of scene graph matching accuracy,the combined use of scene graph fingerprint vector and double-flow graph matching network is proved to be effective.In addition,this thesis also tests the retrieval results of a variety of different actual images,and the retrieval results of actual images also show that the retrieval results of the model are basically consistent with the scene of the query scene graph,which meets the task requirements of this thesis.