Research On Key Technologies Of Scene Understanding For Autonomous Driving

Artificial intelligence,as a new driving force leading the development of the world,has been given great attention.As an extension and application of artificial intelligence technology in the automotive industry and transportation,autonomous driving has received extensive attention from academia and indus-try.An autonomous driving car engages various domain of knowledge such as computer,sensor,telecommunication,AI and automation.Researchers have proposed to use low cost cameras to complete the environmental perception tasks.Therefore,in the current research on autonomous driving system based on computer vision,a data-driven computing framework composed of percep-tion,planning,decision-making,and control modules is widely used.And it has gradually extended to open traffic scenes,and aims to built a basic computing framework that conforms to the driver's brain cognitive process of attention,reasoning,learning and decision-making mechanisms,and establishes a fully functional autonomous intelligent system inspired by biological intelligence.This study focuses on the key technologies based on computer vision tech-nology involved in the scene understanding of autonomous driving,and groups the related works into five branches,namely rule-based methods,end-to-end learning,intermediate approaches,future frame prediction methods and brain-inspired cognitive model.Based on the five paradigms,this paper starts from the motion states exploration of the driving agent in the road environment,then goes beyond by further analyzing and understanding the driving context infor-mation and predicting the relationships among multiple driving objects.Finally it combines the data-driven approaches together with the brain-inspired cogni-tive methods to explore the relationship between video data and brain cognitive data,which improves the performance on visual saliency field detection task and further broadens its research area.The main contributions of this paper are as follows,1.This paper proposes two end-to-end deep learning models at the frame level and event level to classify the driving states of vehicles,collects and labels the real driving videos.The effectiveness of the two proposed meth-ods for processing real-time multi-event detection tasks is verified.This task provides data and benchmark methods for the future research direc-tion of this subject..2.Based on the physiological mechanism of human brain and the experience of video sequence prediction based on deep learning,this paper proposes a multi-task prediction method that simulates the layer-by-layer information feedback of the human brain and a future frame prediction method that simulates the human brain to predict by memory,separately.The research completes the prediction of the future frames of the driving scene video and the corresponding steering wheel angle.3.This paper proposes a two-stream architecture based on visual interaction.By adaptively analyzing and feeding back the spatio-temporal information of the image sequence,the optical flow-like information is extracted by this method and the future states(positions and speeds)of multiple targets in the driving scene are adaptively predicted.The experimental results on the public UDACITY dataset show that the proposed method can effec-tively predict the states of the targets in a real driving scene.4.This paper proposes a method to convert the single-channel EEG sig-nals into a two-dimensional attention feature map which can reflects the drivers'concentration intensity.An end-to-end deep learning method is constructed to extract driver attention information from EEG signals and their corresponding video data.The experiments on the self-built dataset demonstrate the correlation between the visual representations and the EEG signals induced by them.The experimental results show that the vi-sual saliency feature maps obtained by the EEG assisted prediction model are closer to the salient maps from the drivers' attention,and their feature distribution is more reasonable and interpretable than the baseline.