Study On Attention-assisted Multi-source Information Fusion Semantic Segmentation Methods

Semantic segmentation is a key concern in the field of computer vision,and it is also the cornerstone task of scene understanding.In recent years,in view of the excellent performance of deep learning in image processing,speech recognition and other fields,deep learning methods are also commonly used in semantic segmentation.At present,the most outstanding semantic segmentation methods usually adopt the structure of a fully convolutional neural network to predict the classification map in pixel-wise order.However,semantic segmentation is a task of dense labeling,which needs to retain the accurate position while obtaining semantic information of a target.Existing neural network models often deteriorate details of the target when extrationg features.Therefore,the key to semantic segmentation is capturing high-level semantic information and collectting low-level spatial information at the same time.From the perspective of encoder,decoder and auxiliary information utility,multi-source information fusion semantic segmentation models based on attention mechanism are proposed in this thesis.Finally,experiments are carried out on WHDLD dataset and DLRSD dataset,which proves the correctness and validity of the proposed methods.1.In terms of the refinement on coding information,this thesis aims at the balance between semantic information and detail information.and a multi-stage feature fusion fully convolutional semantic segmentation model is designed.Firstly,in order to obtain the most advanced abstract semantic information of the image,final feature map of the pre-trained feature extractor is used as the source of high-stage semantic information.Then,to retain precise position of the target,intermediate feature map of the feature extractor is used as the source of low-stage detail information,and a low-stage feature transformation module is designed to realize the refinement of spatial information.Finally,these two information sources are fully fused through a feature fusion module,and the prediction is obtained through up-sampling operation.This new model encodes semantic information and detail information at the same time with the help of the fusion of features from different stages,and realizes the comprehensive utilization of different information sources.It is worth noting that the root of detail information source is the reuse of existing feature maps,so this new model not only exploits more information of features,but also reduces the resource consumption when building detail source.2.In terms of the refinement on decoding information,this thesis aims at the problem of rough up-sampling results,and an up-sampling optimization model based on channel attention mechanism is proposed.On the basis of multi-stage feature fusion,this new model improves the up-sampling method with the help of attention mechanism and skip connection strategy,thus greatly improving the decoder.New model abandons the commonly used method which directly adopts bilinear interpolation to output the final result.Instead,it uses the coding information to guide the feature recovery process in decoder,which significantly improves the performance of decoder.Different from other models that connect the coding information across layers,this new model makes full use of the channel attention mechanism,so that the decoder can independently optimize the decoding process and reorganize decoding information.Finally,this new model is validated and analyzed on two different datasets.3.In terms of the utility of auxiliary information,this thesis aims at the problem of coarse edge,and a semantic segmentation model based on edge information source is proposed.At present,most advanced semantic segmentation models deal with edge information and other information without distinction,while this new model focuses on optimizing edge details of the target by designing an edge extraction module,so that different types of information can be fully exploited.In addition,the introduction of edge constraint also realizes the comprehensive utilization of target classification information and target edge information.Then,binary edge labels are created using Canny edge detection algorithm,so that this model can be trained with edge labels and segmentation labels at the same time,and the multi-constraint optimization can be futher realized.Finally,the effectiveness of the improved method is verified through different experiments.