Research On Vision-based Road Environment Understanding Technology For Autonomous Vehicles

As the carrier of new theory and new technology, autonomous vehicle embodies the control science, robotics and cognitive science, intelligent transportation and other multidisciplinary, and has an important application value in military, civil and scientific project. Vision-based road environment understanding technology of autonomous vehicles is the key to realize the vehicle’s full autonomy and practicality. Now this technology is not so mature because that there are a lot of uncertainties in the dynamic images collected from the outdoor natural environment which brings great difficulties to develop the robust, real-time and adaptive road understanding algorithms.The Jinglong No.2(C30) pure electric autonomous vehicle project is the research background in this dissertation. The research contents include the main navigation mark(the lane and the virtual road center line) detection, the road obstacle detection, and the assistant navigation sign(traffic sign) detection and recognition. The key algorithms were improved from the common technology aspects of machine vision which include visual signal processing, image feature extraction and image pattern recognition and a complete road environment understanding system is constructed. Multi-scale feature extraction in frequency domain, sparse representation and pattern recognition based on feature matching technology form the main technology clue of this dissertation.For the urban road lane understanding, a self-tuning loop Lane Visual Detector architecture is proposed and the lane detection algorithm based on multi-scale edge feature and improved angle constraint fast Hough transform in wavelet domain is proposed. And the lane tracking method based on a scale-adaptive Kalman filtering is proposed. To solve the robust detection problem of shadow lane, a shadow removal algorithm based on the two dimensional wavelet analysis of frequency domain is proposed which makes full use of the characteristic of the IPM(Inverse Perspective Mapping) image. In the IPM image, the lanes generally look like the vertical parallel lines, so the algorithm only detects in the vertical sub decomposition images of multiple levels.The shadows and crack singularity of image signals make the vision algorithm not so robust when the unstructured road’s centerline is extracted. As a big data, the real-time processing of the image sequence collected from the vehicle video camera is difficult. In order to solve these problems, the single level wavelet packet approximation compressed sensing concept and algorithm is proposed. And this algorithm is combined with the adaptive genetic algorithm to realize the robust image segmentation. So a real-time road understanding algorithm system is constructed. The experimental results show that the method is superior to the traditional method at maintaining the road segmentation consistency and semantic certainty, and the adaptive Road-Non Road two classifications are realized.In order to deeply understanding the road, the road modeling and the road semantic understanding methods based on the wavelet domain semantic tree Markov model are further proposed on the basis of the above algorithm. The supervised RT-MRF model is used for semantic segmentation of road image sequence which can give the regions with semantics boundaries. The autonomous vehicle can realize autonomous mobile by tracking the virtual road center line calculated by the segmentation results. The method proposed has solved the problem of road semantic modeling, and fills the blank of lacking rigorous mathematical model in current road visual sensing field. So the road understanding result extends to the deeper degree.Two kinds of methods about road obstacles detection are put forward. In the first method, the Contourlet transform is used for image processing. The graph cut based stereo matching algorithm is used to obtain disparity map and depth features. The adaptive Sobel operator is used to obstacle’s edge feature extraction. And the multi-feature fusion method is used to determine the size and the distance of obstacles. In the second method, the human visual characteristics are simulated, a visual saliency feature extraction method using an adaptive Hessian threshold to control the feature sparsity is prposed which is used to the dynamic obstacle detection. The algorithm can be applied to the sunny day, rainy day, evening and night and other weather conditions, and shows strong robustness.Finally, aiming at solving the problems associated with the traffic signs robust detection and recognition under the complicated conditions such as illumination changes, occlusion, perspective changes, scale changes, a new traffic sign detection method with two steps from rough to detail based on the coarse-grained H feature and color-shape combination reasoning model is proposed, and a new traffic sign recognition method based on SURF feature optimization matching is proposed.