Research On Head Pose Estimation Based On RGB-D Data

Augmented reality is designed to embed computer generated virtual objects into real environments in real time,allowing users to observe the virtual-real fusion scenes,thereby enhancing the perception of the real environment.The primary task of augmented reality is to align the virtual object with the real environment,which is to align the coordinates of the virtual object with the world coordinate system.Therefore,3D registration is an important means to achieve geometric consistency between virtual world and real world,and it needs to meet the requirements of real-time,accuracy and robustness.Because of the importance of human activities,human head pose estimation is the key technology in 3D registration.It has important applications in augmented reality,human-computer interaction,medical rehabilitation,game entertainment and so on.Head pose estimation needs to locate its position and orientation in 3D space,and augmented reality technology is challenging for the online real-time requirement of 3D registration.The traditional head pose estimation methods are more based on color images,because color images are easy to acquire and can guarantee high resolution.However,image based head pose estimation method is very sensitive to illumination change,partial occlusion and feature point deficiency,which leads to low precision of the algorithm.In recent years,with the development of depth acquisition technology,such as the emergence of Microsoft Kinect,the depth map has increasingly demonstrated its superiority in visual problems.As the depth map contains 3D information,it can effectively overcome the ill posed condition of color images in reverse problems.At present,the depth map is very noisy,and also has low resolution and serious information loss,which cause the poor robustness of the algorithm.The method combines color image and depth map has the superiority.However,the core problem is how to combine color and depth information.Head pose estimation still faces many technical challenges.In this paper,head pose estimation based on RGB-D data captured by Microsoft Kinect sensor is studied.First of all,the optimization problem of objective function cannot expressed analytically in 3D registration is studied;secondly,head pose estimation based on depth map is studied;finally,head pose estimation based on the combination of color image and depth map is studied.The main contributions of this paper are as follows:1.An optimization strategy for objective function cannot expressed analytically is presented.Existing model-based 3D registration methods need to solve the problem of the correspondence of feature points,and the computational complexity is large.In the optimization,the objective function is usually expressed analytically.However,when the objective function cannot be expressed analytically,the algorithm results are poor.As to the registration problem of irregular objects with 3D model but missing texture information,this paper proposes a 3D registration optimization method based on contour matching.The method uses the contour matching of the object in the image to estimate the pose,and the matching error can be expressed as a function of position and orientation parameters.Since this function cannot be expressed analytically and solved,this paper proposes to compute derivatives and objective function values by discrete sampling.Experimental results show that the proposed method can converge quickly and has high accuracy and robustness.2.A real-time head pose estimation method based on depth map is proposed.Thetraditional head pose estimation method based on color images is very sensitive toillumination,shading and feature loss,and faces severe robustness and accuracy problems.Depth maps have higher dimensional information,which can improve the accuracy and robustness of parameter estimation.In order to guarantee the real-time performance of the algorithm,the head pose estimation method based on depth map is proposed.The method combines Kalman filter and random regression forest method to estimate head pose accurately.The prediction model of Kalman filter is used to improve the speed of the random regression forest,reduce the classification error rate and improve the accuracy of pose estimation.The correction model of Kalman filter guarantees the robustness and accuracy of the algorithm.3.A real-time and accurate head pose estimation method combined with color images and depth maps is proposed.Depth maps usually have low resolution,and serious noise.Head pose estimation algorithm directly with the depth map leads to poor accuracy.To solve this problem,an accurate 3D head pose estimation method combined with color images and depth maps is proposed in this paper.This method can make use of the advantages of high resolution of color images and 3D information of depth images simultaneously.Based on the 3D feature points defined in this paper,the pose estimation problem is transformed into point cloud matching.The qualitative and quantitative experimental results show that compared with other methods,the proposed method can significantly reduce errors and has high accuracy.