Research On AR-HUD Environment Perception Technology Based On Multitask Convolutional Neural Network

To improve the safety and convenience of driving,HUD are widely used in automobiles.As cars become more intelligent,AR-HUD emerges in response to the proper time and conditions.AR-HUD integrates virtual information about vehicle conditions with real environments,which enable the driver to look directly at the information on vehicle condition so as to reduce the occurrence of accidents.As one of the important components of AR-HUD system,environmental perception is accountable for analyzing scene information in front of the vehicle,which generally contains target detection and lane segmentation.The computer platform used in vehicles is generally embedded platform.Compared to the PC machine,its calculation is much lower.Especially,neural networks need to call a large number of GPU resources in the process of forward reasoning.How to run target detection algorithm and lane segmentation algorithm in vehicle embedded platform for real-time detection of targets and segmentation of lanes is one of the difficulties in current research.Aiming at this problem,a lightweight multi-task convolution neural network proposed in this thesis.The main research contents are as follows:1.In view of the low computational power of embedded platform,this thesis presents a lightweight multi-task convolution neural network,the network is built based on the hard parameter sharing mechanism of multi-task learning framework,and it has target detection task branch and semantic segmentation task branch.In order to improve the feature extraction ability of lightweight backbone network,CSPDense Net layer was added to the second half of the backbone network.Thus,the backbone network can not only improve the ability of image feature extraction but also there is less amount of parameters and calculation.2.In view of the difficulty of multi-task convolution neural network training,this thesis puts forward a linear weighted summation algorithm for dynamic loss weight.This algorithm can effectively suppress the fast convergence of branch networks so that the convergence speed of the two branch networks tends to be the same.The multi-task convolutional neural network was tested in the test datasets,and the experimental results showed that the m AP and m IOU of the network model in the test datasets are 67.17% and78.43% respectively.3.Multi-task convolution neural network model is deployed to vehicle embedded platform while Tensor Rt is used to accelerate the network model so that multi-task convolution neural network can realize real-time detection in vehicle embedded platform.Then,the embedded platform and AR-HUD system are jointly deployed to the experimental vehicle for real vehicle experiment.