3D Faces Reconstruction And Application Research

With the development of computer vision technology,we are increasingly demanding the image processing and analysis capabilities of computers.In pattern recognition,face recognition has an extremely important position and application value.Traditional face analysis and recognition techniques are based on 2D face images,but under the natural conditions,the 2D facial image of the same person have large difference due to changes in pose,expression,illumination and other factors.In order to solve the challenge in face recognition and analysis caused by the facial rigid and nonrigid variation,the facial representation and feature extraction methods are proposed to represent the invariance of the biometric information in the face and achieve the purpose of reducing the dimension of the face image.Compared with the 2D face,the 3D face has an intrinsic pose and expression invariance.Moreover,the 3D face models can be used to analyse and estimate illumination condition for 2D face images.With the improvement of the accuracy of 3D face data acquisition devices and the development of 3D face reconstruction algorithms,the research on 3D face has received extensive attention from researchers in recent years.Compared with 2D faces,3D faces contain more information and have a better development prospect in biometrics.In this paper,we have done some researches on 3D face shape reconstruction,3D face model application in 2D face image normalization and 3D face texture reconstruction.Starting from the widely used 3D face model and 3D face reconstruction algorithm,the main contributions of this paper are divided into the following aspects:(1)In theory,the basic idea of the traditional 3D face reconstruction model and the principleof face reconstruction are introduced in detail.We performed 3D face reconstructionexperiments on different face databases to intuitively analyze the advantages anddisadvantages of traditional methods.(2)A new 3D face reconstruction algorithm based on face feature points weighting isproposed.The 3D Morphable Model(3DMM),which is widely used today,canreconstruct the corresponding 3D face from a 2D face image.With the development ofthe face feature point localization algorithm,the traditional 3D face reconstructionmethod uses 2D feature points for 3D face reconstruction,which improves theefficiency of the algorithm.However,the improved reconstruction algorithm does notconsider the semantic information of face feature points and the different influences ofdifferent feature points on 3D face shape reconstruction.Therefore,we proposed thefeature point weighted 3D face shape reconstruction algorithm.We introduce featureweight information to weight different feature points.This allows our algorithm todynamically adjust the weight of individual face feature points according to itsreconstruction accuracy in the 3D faces shape reconstruction process.Considering theweight of each feature points individually enhances the reconstruction accuracy offeature points which represent different semantic,thereby further improving the overallreconstruction accuracy of the all 3D model.We performed 3D face reconstructionexperiments on different face databases,analyzed the overall error of the algorithm atfeature point level and 3D face.And verify the effectiveness of the proposed algorithmin different facial expressions and poses.(3)We have improved the existing normalization algorithm for 3DMM-based 2D faceimages.The 2D face normalization is an important step in face recognition.The initialface normalization methods include face detection and localization,face grayscale,histogram equalization,etc.but these traditional face image processing methods cannotnormalize the pose and expression of 2D face images.The 2D face image normalizationmethod based on 3DMM estimates the shape,pose and expression parameters fromimage.Then,the corresponding 3D face shape is obtained according to these 3Dparameters.Finally,according to the pose and expression regularization of the 3D shape,the posed and expression of the 2D face are adjusted accordingly to obtain a normalized2D face image.However,the existing 3DMM-based face normalization algorithms arenot effective in reconstructing the face self-occlusion region.The reconstructed self-occlusion fill area is unsmooth and unnatural in visual,the texture of the filling area hasdifferent texture compared with the surrounding area.In order to deal with this problem,we a new texture reconstruction method for self-occlusion regions.With theimprovement of algorithm,the reconstructed filled area has real illumination intensityand face texture details and the face normalized image is more natural.(4)A 3D face reconstruction algorithm based on face normalization is proposed.Theexisting 3D face reconstruction algorithms estimate the 3D shape and texture based on3D model.In the fitting process,the model is deformed and texture rendered accordingto the 3D parameters and the 3D face reconstruction result is finally obtained.However,the face texture reconstruction of these methods estimates the texture by establishingillumination parameters by establishing the light model,and then change the averagetexture of the model according to the illumination information to obtain thereconstructed texture.Since the traditional method only uses a few texture parametersto represent the real face texture and the average texture representation ability of theface model is very limited,the fitting result of face texture loses a lot of face detail information,such as: beard,wrinkles,pupil color,beard color,etc.But these details textures have important biometric information.In order to reconstruct the more accurate and natural 3D face texture,we combine the 3DMM-base face normalization method to extract the face texture information from the frontal view face images.The face normalization process estimates the 3D shape parameters for the corresponding 2D face image also,so we can get the final 3D face reconstruction result by rendering the extracted texture onto 3D shape.Compared with the existing 3D face reconstruction algorithms,our proposed method can get more accurate and detailed 3D face results.We have done the 3D face reconstruction experiments on different face database.The experiment results show that the proposed method has a significant improvement in the reconstruction of facial detail texture features.