Study Of The Understanding Of The Spatial Layout And Semantic Segmentation Towards Traffic Scene Images

Traffic scene understanding is a key technique for the success of many applications such as autonomous driving,driver assistance,robot navigation,etc.For traffic scenes understanding,it is of great theoretical and practical significance to study the attention feature,the visual sensitivity,and the perception ability of the computer vision system,both in theoretical and practical.The main challenge of urban traffic scenes understanding is the complexity of high-level visual information.Human vision is a high-level visual information processing system that can processes this information rapidly to guide an action.When we see a city scene image or a video,we can grasp a three-dimensional material world.We first instantly perceive the spatial structure of the scene layout,and what we further obtain is the specific contents,such as number of cars,the vehicle types,the relative positions of vehicles with respect to pedestrians and the assessment of traffic congestion.Then we process all these information to make the determination of the driving area.This prosess has guiding significances for a machine to understanding the scene automatically.Based on the human visual sense of hierarchical perception and analysis,we design spatial structure layout understanding and semantic segmentation systems for traffic scenes.Traffic scenes are not easy to understand because of the complex and varied targets,uneven illumination,large shadows and serious occlusion between traffic objects.In this dissertation,we study how to perceive the whole traffic scene by using the low,middle and high level of the scene multi-visual features while dealing with a complex traffic scene.We focus on the study of the main spatial structure layout understanding,road detection and semantic segmentation.We also present several systems that can detemine different classes in an arbitrary traffic scene image,automatically and accurately.The main research results and contributions of this dissertation are listed below:1.Scene segmentation based on the pixel classifying calculation is complicated,and they use insufficient features,thus resulting in low accuracy,so a new model is proposed to overcome these shortcomings,which is to learn these geometric classes based on multi-visual features of super-pixels.First,various features are extracted from the super-pixels of an input image.These features are used for classifying the super-pixels.Then the difference between the adjacent super-pixels is calculated to predict their consistency.The initial classification result and the consistency are synthesized to the Markov Random Field energy function,which is then minimized,based on the graph-cuts algorithm to get the final labels of the super-pixels.Experimental results prove the effectiveness of the multi-visual features and the optimization method proposed,with superior performance achieved for traffic scenes.2.A novel segmentation algorithm for road scenes based on hierarchical graph-based inference is proposed to solve the problem that object boundaries extracted by existing graph-based segmentation algorithms are not fine enough and it is hard to adapt to complex road scene layouts.The algorithm first over-segments an image into small homogeneous regions called superpixels,and then the random forest model is used to train a multi-class regressor and a consistency regressor of superpixels.Regression results are then used to calculate the energy terms in a Markov Random Field energy function.An initial segmentation of the image is obtained by using the superpixel Markov Random Field inference.A pixel-level labeling based on fully connected conditional random fields is constructed to avoid the label confusion caused by the superpixels and to get a fine segmentation finally.Experimental results show that the proposed algorithm solves the label confusion in the superpixel inference and gets fine segmentation boundaries for both the images in manually labeled datasets and the real road scenes.A comparison with the traditional MRF graph-based inference methods shows that the hierarchical graph-based inference algorithm provides improvements of 2% and 3% on the overall precision and per-class average metrics,respectively.3.Existing approaches classify the road area by using appearance-based features,which is vulnerable to the complicated imaging conditions such as extreme shadows,illumination and occlusion.A new road detection method is proposed to overcome this problem brought about by these factors,which combined the advantages of the structural knowledge and fully connected conditional random fields(CRFs).Firstly,a confidence map of road is generated based on the detection of the vanishing point and road boundaries.Two maps and appearance features are used to calculate the energy function of the CRF.Finally,road pixels could be labeled using an efficient inference method.Experimental results prove the effectiveness of the usage of structure information and a fully connected CRF,and the proposed method is robust to the shadows and occlusion in real road scenes.4.The CRF model is usually applied as a post-processing for the sematic segmentation.Most existing methods integrate the output of a deep network into the CRF model directly,with rich features extract by the network being neglected.Besides,the pairwise potentials defined by color vectors do not work well for small objects.A novel road scene segmentation method is proposed to alleviate the limitations of the traditional graph-based method.A hierarchical graph-based inference model is applied which benefits the advanced features extracted by deep convolutional networks.The proposed schemes can alleviate label confusion for small objects and reduce the computational cost in the traditional CRF inference.Experiments on two benchmark datasets prove the effectiveness of the proposed method.The above study covered scene layout segmentation and semantic segmentation,which differ in the information presentation and represent the different levels of scene understanding.The overall research on traffic scene understanding follows the gradual progress from the elementary to the profound.The achievement has important theoretical significance and practical value.