Driver Gesture Recognition Based On Depth Images

Driver pose recognition is a hot research topic in the field of vehicle assisted driving and an important practical application of human pose recognition.Studying the human pose recognition method can provide reference for this actual problem.Traditional human pose recognition is based on the visible image or video sequence as the research object,extracting the traditional artificial features and training pose classifiers or regressors.However,the visible images are easily affected by human body appearance and the change of ambient light,and the manual setting of features is time-consuming and laborious,so it is hard to be well applied in the actual scene such as the cab environment.In this paper,we mainly study the driver pose recognition based on depth image,make full use of the color irrelevant characteristics of depth image and the included depth information,study the characteristics of learning hierarchical features automatically of the convolutional neural network,and realize the driver pose recognition method based on depth image and depth convolution network.The main contribution of this paper is as follows:(1)A method of driver's joints recognition based on depth image and depth convolutional neural network is proposed.Due to the characteristics of local connection and weight sharing,the convolutional neural network is invariant to the change of ambient light,complex human appearance and pose,and the depth images are color-irrelevant and can provide depth information of scene targets.Therefore,this paper uses a ToF camera to obtain the driver's pose depth image,designs a deep convolutional neural network based on LeNet network to extract the image hierarchical features and realizes the driver's joints recognition.Experiments show that the network can effectively locate the driver's joints and has higher accuracy than other networks.(2)A method of constructing the virtual driver's pose depth image dataset by computer animation software 3DSMAX is designed to solve the problem of lacking a large number of driver's pose depth images with human joints annotation.In this paper,we use 3DSMAX to simulate the cab environment,render the virtual driver pose images in bulk,and construct the virtual driver pose depth image datasets,color image datasets and corresponding label information.This dataset construction method not only saves manpower,but also reduces the error caused by human subjective factors.The dataset constructed in this paper can be used in many visual tasks such as driver's body part detection,pose recognition and head analysis.(3)A driver's head analysis framework based on depth migration learning is implemented,which is used to solve the problem that the characteristics of a large number of virtual driver's image data are greatly different from the real scene.Utilizing the self-learning characteristics of deep network,finding the common feature subspace of source and target domains,minimizing the source domain and target domain in the network distribution difference of the highest output layer,updating the depth migration network according a small number of target domain samples,and the resulting model has good migration effects for the target domain.(4)A driver's pose automatic annotation system based on depth image is designed.Taking driver's pose depth image as input,this system can automatically recognize driver's joints,head pose and facial landmarks.