Semantic Segmentation By DNN With Nonlocal Information

Semantic segmentation which aims to give dense label predictions for pixels in an image is one of the fundamental topics in computer vision.Recently,Fully convolutional networks(FCNs)have proved to be much more powerful than schemes which rely on hand-crafted features.Following FCNs,subsequent works have get promoted by further introducing atrous convolution,shortcut between layers and Conditional Random Fields(CRFs)post-processing.Even with these refinements,current FCNs based semantic segmentation methods still suffer from the problems of poor boundary localization and spatial fragmented predictions.The difficulties lie in the conflict between deep network requirement to merge spatial fragmented predictions by getting non-local semantic information and shallow network requirement to keep low-level context information as much as possible to do pixel-level classification.We decompose the task into two trivial task:seed detection which required to predict initial predictions without the need of wholeness and preciseness,and similarity estimation which measures the possibility of any two nodes belongs to the same class without the knowledge which class they are.For each task,we use a branch network to solve it.Inside the network,we apply a cascade of random walks base on hierarchical semantics to approximate a complex diffusion process which propagates seed information to the whole image according to the estimated similarities.Based on the methodology above,the proposed DifNet consistently produces improvements over the baseline models with the same depth and with the equivalent number of parameters,and also achieves promising performance on Pascal VOC 2012 and Pascal Context dataset.