Research On The End-to-end Semantic Image Segmentation Algorithm Based On The Conditional Random Field Model

Semantic image segmentation is to label each pixel of the picture with a object category label.It is an important branch of artificial intelligence research and has been applied in many fields,such as segmentation of scenes in unmanned driving,and segmentation of diseased organs in medical image analysis.With the Deep Convolution Neural Network(DCNN)making greater progress in object classification,object detection and recognition tasks,researchers have attempted to apply the classification ability of DCNN to semantic image segmentation task.In recent years,researchers designed Fully Convolutional Nerural network following the idea of codec structure.In the encoding process,the network constantly uses the convolution layers and pooling layers to extract high-dimensional features from the image.In the decoding process,the network uses deconvolution layers to recover the object contour.Due to the image information lost by downsampling operation in encoding,the upsampling has poor effect on recovering the details of object contour in decoding.As the back-end part of process in segmentation,the dense conditional random field can effectively correct the error segmentation of front-end.However the parameter learning of the dense conditional random field is independent.For semantic image segmentation task,we design and implement two kinds of back-end network modules based on the conditional random field on the basis of researching end-to-end trainable network.The front-end DCNN network predicts the basic pixel probability,and the back-end conditional random field network corrects the prediction results combined with the local prior information of the image.The specific research contents of this paper include:(1)Research end-to-end trainable network based on the fully connected dense conditional random field.In order to build the network,we analysis the fusion process of front-end and back-end.We use the DCNN as the front-end module classifier to generate a preliminary predicted score map through image processing,where each point value represents the probability that the pixel of that point is labeled as an object category.The back-end network module is based on the conditional random field.Firstly,we analyze the method of constructing and solving the dense conditional random field.Secondly,in order to integrate the model into the network,we analyze and decompose the steps of the iterative algorithm,and represent each step by the network layer.Finally,after analyzing the process that the overall iterative algorithm is transformed into a recurrent neural network structure module based on conditional random field(CRF-RNN),we realize end-to-end network by fusing front-end network and back-end conditional random field module.(2)Research and implementation of CRF-RNN based on two kinds of high-dimensional gaussian filtering.We implement two CRF-RNN modules by using high dimensional gaussian filtering.In order to speed up the calculation of information transfer steps in the CRF-RNN module,we apply Permutohedral Lattice method to the module.Firstly,Permutohedral Lattice method maps the input features to a high-dimensional space and disperses the input values to the Lattice vertices through the splat step.The method uses the separated gaussian convolution for the values on the vertices in the blur step.Finally,the method gathers the values on the vertices and maps them back to the original feature points through the slice step.In order to better transfer information,we transform the separated gaussian convolution into a learnable convolution kernel in blur step.In order to reduce the execution time,we implement CRF-RNN module by using the Gaussian KD-Tree method.Firstly,Gaussian KD-Tree method builds tree according to the input features.Secondly,the input values are sampled to leaf nodes through feature queries in the splat step.Finally,values on leaf nodes are aggregated back to the original feature points through feature queries in the slice step.Using the PASCAL VOC 2012 data set,we make a detailed test and analysis on the end-toend trainable network based on the dense conditional random field,which includes: testing and analyzing the CRF-RNN module with learnable kernel based on the Permutohedral Lattice method;testing and analyzing the CRF-RNN module based on Gaussian KD-Tree method.The experimental results demonstrate that our designing the end-to-end trainable network based on the dense conditional random field can increase the image segmentation results to 72.6% and also increase the computational efficiency by 20%.