Intelligent Vehicles Trajectory Prediction And Twinning Test System Construction Base On Visual Perception Enhancement

Trajectory prediction is the basis of local path planning and decision-making for autonomous vehicles.The vehicle trajectory is influenced by the historical trajectory of itself and surrounding vehicles,as well as the constraints of road traffic rules,etc.Methods based on vision sensors can accurately perceive the driving environment(static traffic scene,dynamic traffic targets,etc.)of autonomous vehicle,extract the fine description of road traffic layout and vehicle historical trajectory,which can provide input data for trajectory prediction and the basic data of three-dimensional modeling for the twin of driving scene at the same time.In view of this,this paper explores visual sensors to fully perceive and understand the driving environment of autonomous vehicles,so that it get accurate vehicle historical trajectory and rich road semantic information for trajectory prediction;Secondly,a trajectory prediction method integrating hybrid attention mechanism is proposed to realize the long-term accurate prediction of vehicle future trajectory;Finally,based on traffic scene and dynamic data of trajectory flow,a virtual-real test system for autonomous driving is constructed by using digital twin technology.The main innovative work includes the following aspects:(1)A method of road region semantic segmentation and parameter extraction based on fusion domain adaptation model is proposed,which not only provides basic road modeling data for the twins of driving scenes,but also solves the problem of lack of road semantic information in trajectory prediction.To deal with the defects of traditional semantic segmentation models that are limited to local contour features and poor translational scalability,a high-precision and high-robust road area semantic segmentation network is constructed by fusing visual space prior information and introducing domain adaptation technology.To eliminate the interference of the foreground target on the semantic prediction of the whole road and predict the fine-grained road structure layout,by looking around the target and using the joint supervision of multi-category road source data,a road semantic reasoning and parameter extraction model is constructed.Experiments on the KITTI traffic scene data set demonstrated that the proposed road area semantic segmentation model achieves 94.96% accuracy,the effectiveness of multi-source joint supervision is verified,and the road structure layout can be correctly predicted.(2)A target detection method combining cascaded attention network and semantically guided hierarchical tree is proposed,which can provide basic data of traffic target model and dynamic traffic flow data for the twin of driving scene,and provide accurate historical trajectory coordinate points for trajectory prediction.By embedding location information into the attention mechanism to design a cascaded attention network based on parameter sharing,selectively focus on important target areas,quickly extract significant feature information of traffic targets,and accurately detecting small-scale traffic targets;Construct a hierarchical semantic tree based on traffic semantics through hierarchical clustering algorithms,learn the potential relationships between hierarchical classes,and accurately classify traffic targets that contain different semantic information;Map visual features to word vector spaces to construct a semantic mapping network,convert visual information into semantic information,detect and identify traffic targets that have subtle differences from the training data set.Experiments on the KITTI traffic target detection data set demonstrated that the proposed target detection method reached the 85.0% in precision and 92.5% in recall for small size.At the same time,it can achieve cross-domain detection of traffic targets.(3)A visual perception method of multi-task joint driving environment fused with temporal information is proposed.Through multi-task supervision and joint optimization,the paper constructs a multi-task joint perception algorithm to realize the rapid detection of traffic participation targets and obtain the information of the passable area at the same time;Use continuous image frames as input,mine the time-series correlation information between continuous image frames,eliminate the influence of spatial displacement caused by object motion on feature fusion,and consider the non-local correlation information of different image frames;Through the backbone network of shared parameters,the method of generating key point heat maps is used to detect the positions of pedestrians,vehicles and traffic lights on the road,and the semantic segmentation sub-network is used to provide road-driving area information for autonomous vehicles.(4)A trajectory prediction method based on traffic constraint and hybrid attention mechanism is proposed.The method uses the predicted vehicle and its surrounding vehicle states,such as position,speed,acceleration,through the traffic force module to robustly learn the dynamic dependencies in vehicle motion and the driver’s subjective intention to change lanes,and input the long and short-term memory network(LSTM)as traffic-force constraint,which is combined with the time series motion state of the predicted vehicle to improve the accuracy of vehicle trajectory prediction;The hybrid attention mechanism analyzes the correlation between historical information and predicted information from the two dimensions of time and feature,and establishes a hybrid attention matrix to guide the model to selectively reuse historical information to solve the problem of information loss in LSTM recursive cycles.Accumulated errors in small time series predictions to improve the long-term prediction performance of the model for vehicle trajectories.(5)Based on the Autonomous Driving Closed Field Test Base recognized by the Ministry of Transport of Chang’an University,the paper overcomes the limitations of real-vehicle road testing and software simulation testing by building a virtual simulation test environment that reflects real dynamic and static traffic scenes,and introduces digital twins to design a perception-enhanced vehicle virtual-real combination autonomous driving test system,which can adapt to the rapidly changing testing needs of autonomous driving technology and make up for the existing the shortcomings of existing testing methods.