3D Model Based Head Pose Tracking From A Monocular Image Sequence

Head pose tracking, a basic research topic in the field of computer vision and intelligent human computer interaction, is becoming more and more attractive recently. The main objective of head pose tracking is estimating the 3D pose parameters from an image sequence with human heads. The technology of head pose tracking can be widely used in face recognition, expression recognition, gesture understanding, video conference, intelligent surveillance, fatigue detection, virtual reality, game and entertainments.This dissertation deals with the case when the monocular image sequences are captured by a single camera with introducing the 3D head model into tracking system to provide the initial depth measurements. Our tracker is consisted of three steps, they are: building of head model, interframe motion estimation and continuously robust processing.In the step of building head model, the tracking system separately use stereo camera and ellipsoid geometric model to provide initial model depth measurements. We named the model which obtained by stereo camera as ground-truth model and the one by ellipsoid as geometric model. The head models built above provide the depth data for interframe motion estimation. In the step of interframe motion estimation, region-based correlation algorithm is employed for finding correspondence between 2-D points of head regions. The correspondence then will be viewed as input for Brightness Change Constraint Equation (BCCE) which is used for estimating pose parameter.When our tracker undergoes long-time tracking, the pose estimation may difficult to recovery due to drift accumulation, head scale change, face expression change or occlusion. In order to solve these problems, we employed SIFT-Based multilayer appearance model. The model, at the very beginning, will select multiple key frames online when the head undergoes different motions, for each selected key frame the SIFT feature will be extracted and stored. Whenever the current frame needs to use the appearance model for registration, our system will use 2NN heuristic method for finding two closest key frames in the appearance model whose SIFT features match current one mostly. The key frames selected will be used as base frames for registration with current one.Compared with previous work, there are two innovations in our work and can be presented as follows: Firstly, through SIFT-Based multilayer appearance model, the tracker is able to provide robust pose estimation even in the case of head scale changed, face expression changed or occlusion occurred. Secondly, this paper compared the tracking results between ground-truth model and geometric one and found the ground-truth achieve 5°RMS while geometric one 7°RMS. The result can be viewed as a proof for selecting head 3d model in tracking monocular image sequences.