Research On Face Pose Estimation

The problem of determining the 3D orientation of a human face in an image sequence is known as face pose estimation. Face pose estimation plays an important role in many areas such as video conferencing, virtual reality, human computer interaction(HCI) and face recognition. For example, in interactive environments, like public kiosks or airplane cockpits, face pose estimation can be used for direct pointing when hands are otherwise engaged. In addition, this technology can be important as a hands-free mouse substitute for users with disabilities or for control of gaming environments.Conventional methods for face pose estimation can be classified into two main categories: model-based and face appearance-based. Model-based approaches assume a 3D model of the face and estimate the face pose by first establishing 2-3D features correspondences and then solving for the face pose using the conventional pose estimation techniques. Face appearance-based approaches, on the other hand, assume there exists a unique causal-effect relationship between 3D face pose and certain properties of the facial image(e.g. image intensity, color, image gradient etc.). Their goal is to determine the relationship from a large number of trainning images with known 3D face poses.Recently, a new face pose estimation approach has been proposed, which is known as the classification-based method. In fact, the classification-based method hopes to get a better result which is similar to regression algorithm using classification-based method. By training a classifier with labelled positive sample set and negative sample set, the classifying process is equal to finding the direction of the changing parameters. Here, we present two classifiers which are Support Vector Machine and Discriminant Analysis Methods.In this paper, we conclude the advantages and disadvantages of the three methods above respectively and discuss the applied situations of them. By comparing the three methods, we proposed a new kind of model-based methods which is elliptical model and position of the feature point-based method. We use it to get roughly estimating results. We use Synthetic Discriminant Function which comes from optical pattern recognition to improve the precision of the estimating results. According to the results of the roughly estimating, we put the input image into corresponding linear correlation filters to get better estimating results. A process of face detection and feature point location is done at first which is used as preprocessing of the face pose estimation.Firstly, because of the distribution of human face color concentrating in a small region in chromatic color space, nearly Gaussian Distribution, we build 2D Gaussian skin color model. In order to make skin color distribution more concentrated, we use weight functions to compute the mean and the covariance of the model. Thus, the main color in the face region gives more influence in the mean and the covariance of the model and the influence of the non-skin color in the face region will decrease. Secondly, we compute the distance between the pixel of the input image and the center of the Gaussian skin color model from which we can get a likelyhood image of the input image. Then, under some rule, we binarize the likelyhood image to get a binary image which only includes skin color and non-skin color. We get the rectangle region of the face by doing statistic of the number of skin color pixels row by row and column by column. Lastly, we take the skin color pixel nearest to the center of the rectangle region as the seed point and fill the binary image with the seed point to get the real face region. Due to the difference between eye color and skin color, we can use Gauss-Laplace operator to detect edge in the binary image. Thus, we get the edge detected image. The rectangle region of eye is located by using the horizontal and vertical projection in the upside of the edge detected image. After finding the two eyes, we use lip color model to locate the rectangle region of mouth in a certain underside of the edge detected image.We propose a new face pose estimation method based on elliptical model and position of the feature point. It combines the position of the eyes and mouth in elliptical model with Artificial Neural Network to estimate the complex angles of yaw and tilt rotations. Firstly, we use a fast edge extracting algorithm to extract the face edge in the binary image. Then, we use least square fitting method to fit ellipse to the edge of human face and to get the elliptical model. At last, 4 position parameters which come from the position of eyes and mouth in the elliptical model are used to get the estimating results. In order to build the relationship between the 4 position parameters and 2 pose parameters, we use a three-layer back-propagation neural network which has been trained beforehand to fix the weights of the network by using large of pose-labelled sample images. We describe how to make sure the pose parameters of the sample images in detail. With regard to the situation that only one eye was located in former location process, we use the same three-layer back-propagation neural network but different training samples. Thus, the estimation results of yaw rotations can be getted in larger range. Experimental results show that our method is able to estimate the complex angles of yaw rotations from -80 to +80°and tilt rotations from -40 to 40°. In the corresponding part of this paper, we introduce face edge extracting algorithm, ellipse-specific fitting algorithm and back-propagation algorithm which is used to train the forward neural network.We use composite correlation filters and neural network to improve the precision of pose estimation. The idea is combining several known views of the face into a composite correlation filter. Different weights are assigned to the views in such a way that the value of correlation peak between the filter and the tested image depends on the face pose parameter under study. By using two such filters, it is possible to estimate two pose parameters which are angles of yaw and tilt rotations. Some literatures assume the relation between the correlation peak value and pose parameter to be linear. However, it is not the truth. Therefore, we use a three-layer back-propagation neural network to approximate the relation between them. We use artificial images of a computer-generated plane viewed from different orientations to evaluate our pose estimation technique. We show with experimental data that this three-layer network improves the performance of the pose estimation. We also get better results as far as face images are concerned.We analyze the cause of the error in the end and the paper ends in a discussion of future work. Firstly, we should improve the face detection algorithm and face edge extracting algorithm to make our face pose estimation method fit the images with different backgrounds, different lighting conditions and different occlussions. Secondly, we should increase the accuracy of the locating algorithms of eye and mouth. Thirdly, we should combine the skin color model, hair color model with image gradient to fit ellipse only to face edge which does not include the region of neck whose color is the same with the skin. Lastly, if the 3D data collecting device is acquired, precise pose-labelled face image will be generated and thus the more precise results of pose estimation will be achieved.