Research On Scene Graph Generation Method Based On Deep Learning

Scene graph generation is an image understanding implementation method that can be used in intelligent monitoring,image retrieval and other application fields.Its work product is a structure graph with object as vertex and relationship between objects as connecting edge.In order to achieve high-precision scene graph generation method,a new scene graph generation model,SGiF(scene graph in feature),is proposed in this paper.SGiF solves the two problems of serious overlap of elements and low confidence of connection edge generation in px2graph,which is the scene graph generation model.At the same time,SGiF also fixes the model over fitting problem caused by the experimental data sVG.Faced with the problem,the specific solutions adopted by SGiF are as followsFirstly,SGiF uses multi-scale feature map to detecting scene graph elements.By using ResNet+FPN as image feature extraction network,SGiF can not only increase the number of samples of receptive field(the original image area corresponding to the value on the feature map),but also implement layered detection of elements with different area sizes,so as to reduce the impact of overlapping elements on scene generation modelSecondly,SGiF modifies the low confidence connection edge results in the way of relationship readjustment.On the premise of knowing the structure of the scene graph,SGiF can retrieve the connection path which is approximately equivalent to the low confidence connection edge,and take this path as the input of the connection edge adjustment model which is weakly dependent on the visual features,so as to obtain the adjustment factor which can be used to modify and rewrite the low confidence connection edge,so as to ensure the correctness of the final scene graph resultsFinally,SGiF uses multiple penalty parameters to prevent over fitting of classification model.When the distribution frequency density of sVG samples is known,SGiF sets a number of penalty parameters related to specific sample types,and takes these parameters as factors in the calculation of classification error,so as to balance the learning efficiency of classification model on each sample,prevent the model from overfitting to some samples,and improve the credibility of the generated modelIn order to verify the effectiveness of SGiF solution,the paper also proposed the validation analysis experiment.The final experimental results show that the three solutions proposed by SGiF are helpful to solve the corresponding problems,and can improve the performance of SGiF in scene map generation task to a certain extent.