Research On The Semantic Image Segmentation Based On The Deep Learning And The Conditional Random Fields Model

Semantic image segmentation task assigns a semantic category label to each pixel in the image,it is a preprocessing operation for object recognition and detection,and it can be applied to multiple tasks,including: segmentation of safe driving areas in automatic driving,target segmentation of imaging radar in defense security,segmentation of organ in medical images and so on.With the large-scale application of deep learning technology,computer vision researchers have generally begun to use Deep Convolutional Neural Network(DCNN)to solve imaging classification,object detection and recognition.DCNN achieved great success in image classification tasks.The latest neural network classification accuracy(96.43%)on this task has exceeded human recognition level(94.9%).In recent years,researchers begin to try to apply DCNN with powerful recognition function to semantic image segmentation tasks and design a segmentation network based on the encoding and decoding structure.In the encoding process,the deep features of objects are extracted,and the object contour is restored during the decoding process.However,the pooling layers and down-sampling operations in the DCNN result in a poor effect when upsampling the object borders.In this paper,we design a Probabilistic Superpixel-based Density Conditional Random Field(PSP-CRF)model based on superpixel probability distribution and Gaussian kernel function for the pixel-wise tasks.On the preliminary results of DCNN calculation,PSP-CRF is used for combining local prior information to achieve the ultimate segmentation effect.We studies the semantic image segmentation from the following three aspects:(1)Research on the dense CRF based on the probabilistic superpixel and Gaussian kernel:In order to construct the PSP-CRF model,we firstly use DCNN to process the image to obtain the score map.The size of each map is the same as the original image,and it is arranged according to the labeling category,and the corresponding pixel belongs to the class probabilities.The normalized calculation of the map can get the probability distribution of the label probabilities of all pixels of the image(front-end classifier module).Secondly,we use superpixel generation algorithm to obtain superpixels,and combine the computational results of the front-end classifier to calculate the probability distribution of superpixel.At the same time,a redefinition strategy based on entropy is designed to make adjustments to the incorrectly segmented superpixels.Thirdly,by using superpixel block as a random field node,obtaining the unary potential function from the probability of superpixel,and obtaining the pairwise potential function the from extracting superpixel high-dimensional feature,we construct a dense CRF model to combine the results of the classifier with the local information to assist in the final decisions(post-processing module).After the construction of PSP-CRF model,two kinds of solving algorithms are used in our paper: the inference algorithm based on Mean Field(MF)theory and the inference algorithm based on linear programming to minimize the model to obtain the final segmentation result.(2)Research on the mean field inference model based on the end-to-end: In order to exert the advantage of the end-to-end of the deep learning,we has explored and experimented with the implementation of front-end and back-end fusion for deep learning and CRF model,including: the computational process of modeling and solving of CRF is divided into neural network operations,then the mean filed iterative inference algorithm is redesigned into a recurrent neural network structure module(CRF-RNN),and then the module is combined with the front-end convolutional neural network to realize front-end and back-end fusion.Experimental test and analysis: For the PSP-CRF and the end-to-end model,based on the PASCAL VOC 2012 dataset benchmark,we make a detailed test and analysis of the algorithms,including: testing and analysis of the PSP-CRF model;testing and analyzing the impact of different inference algorithms;testing and analyzing the impact of different factors(Gaussian kernel,superpixel algorithm parameters,etc.)on the algorithm results;testing and analysis of front-end and back-end fusion algorithms designed in(2).The experimental results demonstrate that our algorithm based on deep learning and conditional random fields designed can increase the image segmentation results to 82%,and increase the computational efficiency by 47%.