Research On Panoptic Segmentation Technology Of Street Scene Images Based On Convolutional Neural Networks

Street view image understanding is an important and challenging task in the field of autonomous driving perception.Traditional street view image understanding tasks only solve partial perception of target categories and locations,refined boundaries,or scene environmental information,while the panoptic segmentation technology proposed in recent years can achieve unified perception of the entire scene.It combines semantic and instance segmentation tasks,not only for semantic classification of each pixel in the image,but also for distinguishing different instance individuals in the image.This technology will be widely used in the fields of autonomous driving and intelligent robots.However,the existing panoptic segmentation methods generally start from the global features,complete the subtasks of semantic segmentation and instance segmentation,and then simply fuse the results,while ignoring the importance differences of different features and the information connection between sub-branches.The research object of this thesis is street view images.Firstly,it starts from the branch of semantic segmentation,and then conducts in-depth research on the above problems,and finally realizes the panoptic segmentation method of this paper.Specifically,the main research contents of this paper are as follows:1.In view of the fact that the current semantic segmentation method generally only considers global features,but ignores the importance difference of different channels and spatial positions in the feature map,this thesis designs a lightweight convolution based on two-dimensional convolution and spatially separable convolution.The mixed attention module MAM(Mixed Attention Module),and combines this module with the feature extraction backbone network and the empty space pyramid pooling module,and finally realizes a new semantic segmentation method.This method can improve the learning of semantic and spatial features with fewer network parameter increments,and at the same time realize multi-dimensional feature fusion and increase the receptive field size of the feature map,so as to improve the segmentation accuracy of the semantic segmentation network.2.The focus of semantic segmentation is on the background,and the focus of instance segmentation is on the foreground,but there may be rich information links between the background and the foreground,which are mostly ignored by the existing panoptic segmentation methods.Based on the semantic segmentation method proposed above,this thesis designs a shared feature extractor and an instance-assisted semantic module ISAM(Instance Assisted Semantic Module),in order to strengthen the information exchange between semantic and instance branches,and better integrate background semantics and foreground Instance features;and a spatial sorting algorithm is introduced in the branch fusion module to alleviate the pixel allocation conflict problem among foreground instances;finally,a semantic-based panoptic segmentation method is realized.The experimental results show that the method in this thesis can achieve competitive results with the existing mainstream methods.3.A panoptic image segmentation system is designed and implemented,and the panoptic segmentation method proposed in this thesis is applied in practice to demonstrate the effectiveness and feasibility of this method.