3D Pose Estimation Of Drivers Based On Joint Networks

Driver pose estimation is an important application of human pose estimation,which plays a key role in advanced driver assistant systems.As an intermediate information,driver pose estimation can help the driver status detection system identify the driver status and judge whether the driver behavior is appropriate and safe.Generally,Driver 3D pose estimation provider more information than 2D pose estimation.Therefore,we choose driver 3D pose estimation as our research direction.The traditional human pose estimation approaches are based on visible image or depth image.For driver pose estimation,visible image is susceptible to the factors such as day-night cycle and environment,so it isn't suitable for driving room.Therefore,we choose To F camera as the sensor to provider data.To F camera which is small size and low cost can capture infrared image and depth image simultaneously,so it is the suitable camera application for vehicle system.In this paper,based on infrared image and point cloud inverted from depth image,we design the deep neural network architecture,take advantage of the characteristics of IR image and point cloud,and realize the driver 3D pose estimation.The main contribution of this paper are as follows:(1)A 3D human pose estimation method based on point cloud is proposed.A deep neural network based on Point Net is designed,which can directly process point cloud to estimate 3D human pose.For the problem of point cloud with vast noise in the real scene,we adopt two strategies: data preprocessing and network module design.On the Driver dataset and ITOP dataset,experiments have verified the high accuracy of our method.(2)A driver 3D pose estimation based on joint 2D-3D network method is proposed.To make full use of information captured by To F camera,a two-branch network is designed to extract the features of IR image and point cloud by two separate neural network modules,and then the 3D pose estimation is obtained through the fully connection layers.In order to meet the requirements of real-time feedback in real scene,we have taken a series of measures to improve the running speed of the model.Compared with other approaches,experiments show that our method achieves more efficient and competitive performance.(3)A driver 3D pose estimation based on time series method is proposed.The 3D pose obtained from joint 2D-3D network group by 5 frames on time series to construct dataset.A network architecture based on recurrent neural network is designed to fix the driver 3D pose estimation.Experiments show that the driver estimation after the model processing is higher than joint 2D-3D network and the model running speed can meet the real-time feedback requirements as well.