Large-scale Visual Relationship Detection Based On Hierarchical Training Strategy

Computer vision is a cross-disciplinary discipline that involves getting computers to get a high level of understanding from digital image or video.Visual relationship detection is a bridge between computer perception and higher levels of semantic understanding.The difference from object detection is that computers need to focus on higher-level features.In Visual Genome,the imbalance problem among the numbers of various relationship classes is solved.In this paper,the research on relationship detection method mainly adopts the top-down processing strategy.Visual relationship detection is of great significance.Our work mainly includes three parts:(1)The design of a large-scale complex visual relationship detection network.MSDN can now realize the efficient relationship detection at a faster speed with a high accuracy.On the basis of this network,the structure of the network is modified from the three levels of tasks originally to a model that is only applicable to object detection and relationship detection.The three branches become two branches,and the message passing process only retains the object detection branch to the relationship detection branch and the relationship detection branch to the object detection branch.At the same time,the message passing process in the network is optimized,and the message delivery strategy based on region overlaps is proposed,which effectively improves the speed of the network training.Compared with the same training strategy based on MSDN network,the average processing time of each picture is reduced from the original 25.42 seconds to 10.35 seconds.At the same time,a parallelization strategy was added to the network to reduce the processing time to 5.07 seconds.(2)For large-scale complex visual relationship detection networks,a hierarchical training strategy based on hierarchical auxiliary loss is proposed.Traditional training methods can lead to inefficient detection network when the dataset is small or unevenly distributed.In order to alleviate the frequency gap between the classes in the relationship dataset,a hierarchical training strategy based on hierarchical auxiliary loss is proposed for large-scale complex visual relationship detection networks.It alleviates the long-tail problem in the dataset and provides the possibility for the computer to understand complex scenes.From the experimental results,the hierarchical training strategy can effectively help the complex relationship detection.(3)The experiment and verification of the proposed network and the hierarchical training strategy for large-scale complex visual relationship detection.Two cleaned highquality large-scale complex relationship datasets are generated in this paper.Based on WordNet,two predicate trees are constructed for two datasets,and the cleaned relationship datasets are hierarchically described.Through comparative experiments,the network and the hierarchical training strategy of the large-scale complex visual relationship detection are tested and verified.