Research On Key Technologies Of Mobile Vision Based Traffic Scene Perception

Since there are so many elements in a dynamic mobile traffic scene,it is such an important task to perceive the complete scene information by using computer visual perception technology with the popularity of intelligent transportation.Mobile traffic scene perception is a comprehensive reserchtoptic,involving perceptions of vehicle,pedestrian,road and other tasks.Although the development of deep learning has provided a strong impetus for the progress of perception technologies,the complex and changeablenatural traffic scenes pose a series of major challenges to thistopic.Based on the mobile vision technology,this research constructs a traffic scene perception framework,which can effectively support three key technologies: traffic scene pre-perception,lane detection and segmentation,and universal license plate recognition.Through the combination of image processing technology and deep learning technology,the main contributions and achievements are as follows:1.A traffic scene pre-perception algorithm based on joint object detection and semantic segmentation is proposed,which can not only achieve the effect similar to panoptic segmentation,but also make up for the shortcomings of high demand and weak practicability of panoptic segmentation annotation.Firstly,a pair matching algorithm is used to get candidate bounding boxes of scene objects.Secondly,in order to get more accurate bounding boxes,an iterative K-means based contour vertex clustering method is designed to reduce the interference of adjacent similar solid objects.Finally,in order to achieve panoptic-segmentation-like perception results,image boundary tracking algorithm is used to determine contours of scene elements.The experimental results on Cityscaps show that the final perceptual effect is more susceptible to semantic segmentation results,and the theoretical upper limit of the proposed method is 95.4% of the groud-truth of panoptic segmentation.2.Based on the semantic results obtained by the joint pre-perception algorithm,a lane detection and segmentation method based on semantic understanding is proposed.Firstly,the road image to be processed is transformed to aerial view by a inverse perspective transformation method based on coordinate analysis.Secondly,a method of candidate lane detection based on two-stage clustering is designed.In the first stage,DBSCAN algorithm is used to obtain road marker clusters.In the second stage,Kalman filtering algorithm is applied to predict directions of clusters from the bottom up,and candidate lanes are summarized according to the distance among clusters.Finally,a three-stage lane filtering algorithm is carried out to optimize lane detection and segmentation results,which makes uses of other road marks,road semantic boundaries and historical lane segmentation results respectively in three stages.The experimental results from Carla G and Tu Simple show that the proposed method can detect and segment multiple lanes effectively.3.A new encoder-decoder based license plate recognition method is proposed to deal with multiple kinds of license plates in traffic scenes.As for the encoder,at the first stage,candidate license plate characters are detected and recognized directly without considering the format of license plate,and candidate regions of license plates are extracted by DBSCAN-like algorithm;at the second stage,poor regions are processed by tilt correction and scale normalization to obtain more accurate candidate characters.As for the decoder,a sequence learning model is trained to convert each unordered coded sequence into a sequence composed of marks which indicate a way to construct the final result string.The experimental results on public dataset show that the detection rate and recognition rate are 99.51% and 95.3% respectively at about 40 fps.It is proved that the proposed framework is more general and extensible than non-modular approaches.