Research On Real-time Semantic Segmentation Based On Lightweight Encoder-decoder

As a pixel-level classification task,semantic segmentation needs large computational cost with enormous parameters to obtain high performance.Recently,due to the increasing demand for autonomous systems and robots,it is significant to make a tradeoff between accuracy and inference speed in the limited computing mobile devices.Given this dilemma,this paper proposes a lightweight U-shaped network(LUNet).Firstly,to ensure the accuracy,the network is built with a mainstream encoder-decoder structure,so that the decoder can capture sharper object boundaries information by gradually recovering the spatial information to refine the segmentation results.Secondly,to speed up the inference speed,the feature extraction network in the encoder is built with ESPNet V2 to efficiently extract the semantic features of the image.The decoder is built using the classification module proposed in this paper.This module first uses pointwise convolution to reduce the channel dimension of the feature map of the encoder to be fused,thereby greatly reducing the amount of calculation and module parameters during module feature fusion.Then,The feature of the encoder feature map after the dimensionality reduction is fused with the feature of the score map of the decoder using element-wise addition to refining the object boundaries of the score map.Finally,the number of decoder parameters built using this classification module is only 8264,which only accounts for 1.1% of the overall network parameter amount.The extremely small amount of parameters greatly reduces the calculation amount of the decoder,which speeds up the network inference speed.Experimental results demonstrate that the proposed LUNet achieves a balance between speed and precision,it achieves 64.1% m Io U on the Cityscapes test set with only 0.764 million parameters and a speed of 83 FPS on a single GTX 1080 Ti card.Compared with ESPNet V2,which uses the same feature extraction network,it can maintain the same inference speed and bring about a 2% m Io U improvement.To obtain a network with a stronger ability to identify similar objects,this paper analyzes the confusion classification problem of LUNet and finds that its decoder has insufficient ability to correct the confusion classification that has occurred.For this purpose,a new sliced connection attention module is proposed in this paper.In this module,the feature of the score map at each level of the decoder is first enhanced by using the sliced substructure,so that each category score map has three status,and these three status include all the situations when they are fused with the corresponding encoder features,where one of the statuses is the correct score map generated by them using the correct feature fusion method.To select their correct score map,the module uses the learning substructure based on the channel attention mechanism to automatically learn the important information in the three status score maps of each category,and then use this information to select the correct score map.Thus,the confusion classification problem is corrected in the decoder layer by layer.Experiments results demonstrate that the lightweight sliced concatenation attention network(SCANet)constructed by replacing the classification module in LUNet with this module,it achieves 64.6% m Io U on the Cityscapes test set with only 0.766 million parameters and a speed of 66 FPS on a single GTX 1080 Ti card.At the expense of a small amount of inference speed,it effectively solves the confusion classification problem and has a stronger similar object recognition ability than the excellent network with parameters of the same order of magnitude.