Research Of Scene Graph Generation Method Based On Object Relation Enhancement

Scene graph generation is an important task in computer vision,which aims to infer the objects and their mutual relationships in a given image.In this way,a given image can be generated as a graph representation with nodes and edges,thus obtaining a deep understanding of the scene.By visually understanding and modelling the relationships between different objects in a scene,the generated scene graph can be used for advanced visual tasks such as autonomous driving,visual quizzing,image description generation,etc.Existing scene graph generation methods mostly combine visual information such as appearance,location,and structure of objects,as well as semantic information to obtain contextual features for joint inference.They further enhance the features using knowledge embedding to generate more accurate scene graphs.However,these methods ignore the redundant cross and internal implicit relation among various information.To address the issue,this thesis proposes a novel method based on enhancing object relation.The main research contents are as follows:(1)Scene graph generation algorithm based on dynamic perception of object relation.The complex and diverse scene information leads to redundant fused features and difficult to accurately capture relational representations.This thesis propose a scene graph generation method for dynamic perception of object relation.The method uses a self-attentive mechanism to capture the internal relation of features and obtain the refined relational features.Afterwards,the learning strategy of the model is dynamically adjusted by setting a sparse structure-aware factor in conjunction with the structural connotations of the scene graph.Thus,the performance of multiple relational inference is improved.Finally,this paper designs multiple experiments and compares them with mainstream methods to verify the feasibility of the proposed method.The experimental results show that in the predicate classification task and the scene graph generation task,the m R@100 metric of the proposed model has been improved by more than 10.76%and 0.95% on the VG dataset,respectively.(2)Scene graph generation algorithm based on visual semantic relation fusion.Existing methods directly fuse prior knowledge for feature augmentation,resulting in real visual information being easily suppressed.This thesis proposes a scene graph generation method for visual semantic relation fusion.This method first defines the type of prior knowledge and constructs the corresponding knowledge graph to enhance the object semantics.Then,it uses the gated recurrent mechanism of GGNN to propagate node information and jointly learns the visual and semantic relation of the scene graph.Thus,improving the model’s understanding and capturing ability of complex relationships.Finally,by designing multiple experiments and comparing them with mainstream methods,the feasibility of the proposed method is verified.The experiments show that in the predicate classification task and scene graph generation task,the m R@100 metric of the proposed model is improved by more than 0.21% and 13.57% on the VG dataset,respectively.(3)Design and implementation of a visual information representation system for real-world scenes.This system integrates the scene graph generation methods based on dynamic perception of object relationships and visual-semantic fusion of object relationships.The objective is to test the model performance,support real-world scene understanding tasks,and provide visualized effect presentation.