Visual Attention Mechanism Inspired Saliency Detection Models In Driving Scenes

Driving in traffic is a complex and tridimensional scene with multiple information sources,which changes dynamically and requires the instant processing of our brain.Visual selective attention is an important neural mechanism for the visual system that can extract key scene information and filter redundant information.In recent years,as the design of intelligent driving vehicles has gradually become an important research area in today's artificial intelligent era,the driver's visual attention mechanism and its computation model have become two research hotspots.Therefore,based on visual attention related mechanisms and saliency detection models,understanding and predicting the information related to driving tasks in traffic driving scenes will provide useful theoretical basis and related image processing technique for future self-driving vehicles designing,driving training and assisted driving systems,etc.In this dissertation,the mechanisms of both bottom-up and top-down attention in static or dynamic driving scenes have been deeply studied.The eye movement experiments of visual cognitive psychology were designed to establish a static and dynamic traffic driving eye tracking dataset.The development of corresponding saliency detection models can precisely predict the main areas and secondary objects that drivers paid attention to during the simulated driving status.The contents are divided into four parts shown as follows.Firstly,the visual attention mechanism of static driving scenes was studied in Chapter Two.We designed a psychology-based eye movement experiment and collected the eye tracking data of 20 non-drivers(free-viewing)and 20 drivers(simulated driving tasks).The differences of their eye movement properties were analyzed and it demonstrated that top-down attention driven by driving task was able to extract and process driving information more effectively.In addition,an eye tracking dataset of static driving scenes was established,which would provide experimental data support for the saliency detection model in driving scenes.In the second part,a framework of top-down based saliency detection model in static driving scenes was proposed in Chapter Three.Based on the results of eye movement experiments in cognitive psychology,the image feature information driven by driving task was discussed.The vanishing point of the road was innovatively proposed and could be regarded as valuable top-down guidance in a traffic saliency detection model.Subsequently,we built a framework of a classic bottom-up and top-down combined saliency detection model in traffic scenes,which managed to improve the performance of traditional ones effectively.In Chapter Four,the image features related to driving task in static driving scenes were further analyzed.The color,intention,orientation and multiple bottom-up saliency maps were regarded as low-level features of traffic images.Meanwhile,vanishing point and central bias were used as high-level traffic image features.Based on the theory of Random Forest in the traditional machine learning method,we proposed a saliency detection model by integrating the low-and high-level traffic image features.The experimental results on real traffic images indicated that our model was able to predict a driver's main fixation location and secondary target outperformed the state-of-the-art bottom-up saliency models(e.g.GBVS,SR,AIM,SUN,Itti).In Chapter Five,a psychology-based eye movement experiment in real-time dynamic traffic driving scenes was designed.This work collected 16 traffic-driving videos and recorded 28 drivers' eye tracking data while they were viewing the videos in the simulated driving task.A dynamic traffic driving scenes eye tracking dataset was built in this dissertation.Moreover,this work explored a convolutional-deconvolutional neural network(CDNN)for saliency detection of dynamic driving scenes based on the deep learning method.The proposed model has integrated the exogenous visual information(the key and related to current driving information,e.g.traffic signs,pedestrians,vehicles,etc.)and drivers' endogenous attentional information(top-down).This model was able to predict the drivers' primary attention area and driving-related secondary objects quickly and accurately.In conclusion,based on the visual attention mechanism,the saliency detection models of driving scenes were proposed in this dissertation by top-down framework,traditional machine learning and deep learning method.These models would provide the related technical support for the design of the future intelligent driving vehicle.Furthermore,this dissertation managed to build two eye tracking datasets for static and dynamic driving scenes,which would be able to provide the available experimental data for future research of visual attention mechanisms and computational models.