Conditional Random Field Based Image Semantic Segmentation

In recent years,Convolutional Neural Networks(CNNs)have achieved great success in high-level computer vision tasks such as Image Segmentation.Fully convolutional network(FCN)makes end-to-end Convolutional Neural Network a popular way to solve Image Semantic Segmentation.However,in Image Semantic Segmentation,which means assigning a category label to each pixel in an input image,due to large receptive fields and max-pooling layers in convolutional filters,the results obtained from FCN are still kind of coarse.And this motivates exploiting new CNN structures for pixel-level labelling problems.Probabilistic Graphical Models such as Conditional Random Field(CRF)have been developed as effective methods to enhance the accuracy of this kind of tasks.Recent image semantic segmentation methods exploit CRFs only with unary and pairwise potentials terms,which means energy terms based on one pixel or between two pixels.In this thesis we managed to demonstrate that the addition of potentials defined on cliques(a subset of pixels of the whole image with more than two)can result in an improved segmentation outcome.And this potential is named as object clique potential(Potentials from an already existed method and from super pixel cliques).By including this potential in a CRF embedded within a CNN,significant improvement has been achieved over CNN-plus-CRF systems that use only unary and pairwise potentials.We applied this optimization model to Single-Image Crowd Counting via Convolutional Neural Network.In the crowd density map,since it is no good to utilize potentials from latent object segments,we apply Grid pairwise CRF with only potentials based on patches of crowd density map.To deal with the size mismatching problem of input image and output crowd density map,we follow the concept of Deconvolutional Neural Network to enlarge the output crowd density map by interpolation.The experiments show that the proposed method is superior to previous ones in term of crowd density accuracy.