Research On Vehicle-mounted Visual Perception System Of Road Potholes For Unmanned Driving

Environmental perception constitutes the foundation and prerequisite for autonomous decision-making by unmanned autonomous vehicles,wherein road damage condition as a significant component of environmental perception.Potholes are typical road damage that triggers intense vibration and impact when vehicles drive over them at high speed,thereby affecting passengers’ comfort,suspension durability,and cargo integrity.Nevertheless,prevailing environmental perception has emphasized studying above-pavement obstacle targets like vehicles and pedestrians,with little exploration of negative targets such as potholes.Meanwhile,these studies are difficult to fully describe the location information of potholes on roads.This paper aims to mitigate this issue by concentrating on a vehicular road pothole visual perception method for unmanned autonomous vehicles that can identify the condition of road potholes,detect their area,and determine the longitudinal distance via image analysis.The research outcomes of this paper can serve as a decisive factor for subsequent speed control and local path planning of unmanned autonomous vehicles.The primary dataset consists of road images collected through binocular cameras installed on the vehicles.To increase sample diversity to suit varying environmental perceptions,the network is leveraged to gather additional images.Following sample selection,image enhancement is performed using the Retinex algorithm.Considering the influence of road pothole conditions on the decisions of manually-driven vehicle drivers,road pothole conditions are categorized into three distinct classes: good road,heavy pothole road,and completely broken road,forming the foundation of the dataset.To improve channel representation and spatial features,this paper devises the Group Attention Shuffle Block(CASB)and enhances Shuffle Netv2 to establish a road pothole condition recognition model(CASB-Shuffle Netv2).The model is then trained and tested on the dataset,and experiments are conducted to determine the road pothole condition.To construct the pothole detection dataset,Label Img is utilized to manually label the actual pothole areas in the heavy pothole road sample images.To address the issue of missed detections arising from variable pothole sizes at different longitudinal distances,this paper replaces the normal convolution in the feature extraction network of Center Net with a pyramidal convolution with multiple receptive field.Additionally,a feature fusion module is designed in the feature extraction network to combine the high and low-order features of potholes.As a result,a road pothole area detection method called PF-Center Net is developed,integrating pyramidal convolution and feature fusion modules with Center Net.The efficacy of the improved approach is established by extracting intermediate feature maps and conducting ablation experiments.Finally,the model is trained and tested on the dataset,and its performance is evaluated using AP0.5.This paper analyzes the principle of camera imaging and camera aberration.A distance estimation model for road potholes is developed utilizing binocular vision and the Semi-Global Block Matching algorithm.The binocular camera is calibrated using the Zhang Zhengyou’s method,and image correction and stereo matching are performed based on the camera parameters to produce parallax maps.The parallax maps are then optimized using weighted least squares filters to fill in empty areas and obtain distance information.To verify the performance of the system,this paper designed the overall architecture of the road potholes perception system and completed the selection of the image processing platform and binocular camera.The road pothole perception system is then built based on the Jetson platform and the dependency environment for the Jetson platform is configured.At the same time,the perception process of system is designed.The system speed is optimized using Tensor RT.In a real vehicle experiment scene,this paper designs and carries out a test experiment of the road potholes perception system,which proved to have high accuracy and real-time performance for sensing road potholes.