Research On Spatial-temporal Enhanced Network And Its Application In Train Driver Behavior Recognition

Ensuring the safe and stable operation of railway locomotives is the core work of the railway transport department.Improving the level of locomotive operation safety monitoring and management is the most important task.Driving norms of locomotive drivers are directly related to the safety of train operation.Therefore,real-time monitoring and intelligent evaluation of locomotive drivers' driving behavior and state are of great practical significance.Aiming at the problems of real-time performance,high false alarm rate and weak robustness of common behavior recognition methods in train driver behavior recognition,this paper proposes a Convolutional LSTM Networks with Spatial Temporal Enhancement Networks(CLSTE),which is based on train driver monitoring video data sets and open data sets,and combines the idea of spatial-temporal attention mechanism.Identify and understand the behavior of train drivers,master the operation status of trains and the working status of drivers,assist train drivers to drive safely,and better play the role of equipment in supervising the operation of personnel.This paper based on the application background of locomotive driver behavior recognition,data preprocessing,network model design,open standard data set experiment,locomotive driver simulation data set experiment to carry out the research work:1.Make experimental data set.Firstly,in view of the fact that there is no public data set for train driver behavior recognition and there are few abnormal behaviors in the existing train driver driving data,we simulate and record driver behavior videos in the laboratory environment,and combine the existing driving data to make the data set Driver-dataset.Secondly,in order to verify the rationality of the network structure in this paper,the behavior recognition open data sets UCF101 and HMDB51 are obtained.Finally,the video dataset is decomposed into continuous RGB picture frames,and images are extracted to produce RGB dataset based on the principle of sparse sampling.The Optical-flow data set is produced by converting RGB images into optical flow images using optical flow method,which is used to prepare data for subsequent experiments.2.Proposed an improved Convolutional Deep LSTM Networks(CDLN).Convolutional Neural Network(CNN)is used to extract the spatial features of time series pictures.Long Short Term Memory(LSTM)is used to extract the temporal features of time series pictures.By fine-tuning the network model and parameters,the experimental results of UCF101-82.40% public dataset for behavior recognition and Driver-dataset-86.48% are achieved.3.Proposed a spatial-temporal enhancement network.According to the characteristics of train driver's behavior and the task of video classification,a spatial-temporal enhancement network CLSTE based on CDLN and attention mechanism is proposed,which enhances the ability to extract spatial and temporal features of motion changing regions.CLSTE network structure design is relatively shallow,but it can achieve similar experimental results with other excellent deeper networks.It achieves the experimental results of UCF101-93.80%,HMDB51-69.56% and Driver-dataset-94.29%,and the test speed can reach 29 fps,which proves the effectiveness of this method.