| As the key technology of autonomous driving system,behavior-decision largely determines the safety,rationality and stability of the vehicle during autonomous driving.The behavior decision method based on end-to-end learning can directly map the sensor information into the behavior decision quantities of speed and steering angle,which simplifies the structure of the traditional behavior decision system.Prediction inaccuracy caused by the single input data type of the existing end-to-end behavior-decision model and cannot take into account the spatial and temporal information of the driving scene.To solve this problem,an end-to-end autonomous driving behavior-decision model combining spatio-temporal features and attention mechanism is proposed,which can predict the behavior-decision parameters of speed and steering angle.The main research work of this paper is as follows:(1)An end-to-end autonomous driving-behavior decision model based on multimodal multitask of spatial-temporal convolution(MM-STConv)is proposed,RGB images,depth images,and vehicle historical continuous motion state sequences are selected as multi-modal inputs,and semantic information is utilized simultaneously to obtain speed and steering multitask prediction parameters.The main methods are: First,the spatial features of the scene are extracted through convolutional neural networks with different complexities,and the spatial feature extraction sub-network is constructed to accurately analyze the scene target spatial features and semantic information.Meanwhile,an LSTM encoder-decoder architecture is used to capture the temporal context features of the scene,and the temporal feature extraction subnetwork is constructed to understand and memorize the scene temporal series information.Finally,a multi-task prediction sub-network is constructed with the hard-parameter sharing method,which outputs predicted values of speed and steering angle to predict vehicle behavior.The experimental results show that the training error of the MM-STConv driving-behavior decision model is 0.1386,the prediction accuracy reaches 82.9%,compared with other stateof-the-art existing models,the proposed model has obvious advantages in predicting vehicle speed and steering angle.(2)In order to improve the performance of MM-STConv behavioral decision,an attention module is introduced on this basis,and an end-to-end autonomous driving behavior decision model combining spatio-temporal features and attention mechanism is proposed.First,the model introduces parallel spatial attention branch network and channel attention branch network with the same structure into the semantic segmentation network and the convolution sharing structure of the spatial feature extraction sub-network respectively to improve the ability to extract key spatial information in the scene.Second,a temporal attention branch network is introduced into the LSTM encoder-decoder structure in the temporal feature extraction sub-network to model the relationship between historical motion state sequences to make it more focused on the temporal context features of the driving scene.Finally,the weighted and updated scene spatial features and temporal context features are obtained through the multi-task prediction sub-network to obtain the predicted values of speed and steering angle.The experimental results show that the training error of the end-to-end autonomous driving behavior decision model with an attentional module is 0.1185,and the prediction accuracy is 85.8%.Compared with the behavior decision model without the attention module,the training error is reduced 0.0201,the prediction accuracy is improved by 2.9%,indicating that the introduction of the attention module improves the prediction performance of MM-STConv on steering angle and speed.(3)By constructing an end-to-end autonomous driving decision system in-loop simulation test platform,the effectiveness of the end-to-end autonomous driving behavior decision model combining the spatio-temporal features and the attention mechanism is tested and verified.First,the Pre Scan virtual driving data set is collected through the platform.Second,the trained behavior decision model is fine-tuned and finally it is tested and verified.The experimental results show that the model can well predict speed and steering angle,and the introduction of a decision system in a loop simulation platform further improves the performance and generalization of the model.Specifically,the training error is reduced to 0.1218,and the prediction accuracy is improved to 86.1%. |
