Research On Non-Rigid Image Deformation And Its Application In Face Editing

Image deformation is an important branch and research focus in image processing and computer graphics.It is widely used in many fields,such as film and animation,virtual reality,medical imaging,meteorological prediction and many other fields.The existing image deformation methods have achieved remarkable effect for rigid image deformation.In this thesis,we focuses on the problem of non-rigid image deformation and its application in face editing.To solve the deformation problem of non-rigid objects with specific structure such as face,we introduce local neighborhood structure(LNS)constraint and non-rigid model,and propose a non-rigid image deformation algorithm with local neighborhood structure constraint.Then,by combining the proposed non-rigid image deformation algorithm with face landmark alignment algorithm,we achieve automatic face deformation.Further,the Cycle-GAN model are applied to solve the problem of face deformation,which is defective in mouth deformation.Finally,the automatic face deformation algorithm is used to produce image driven facial animation.Our main contribution in this paper are as follows:(1)Non-rigid image deformation with local neighborhood structure constraints.To solve the deformation problem of non-rigid objects with specific structure such as face,we propose non-rigid image deformation with local neighborhood structure constraints.The algorithm regards the image deformation as a map from one image to another,and each pixel in the image has a unique mapping function,that is,the deformation function.The deformation function is non-rigid and modeled in reproducing kernel Hilbert space.The solution of the deformation function is based on the moving least square method.To obtain more natural face deformation results,the local neighborhood constraint is introduced into the process of solving the deformation function.Experiments show that the proposed algorithm performs better than the state-of-the-art image deformation algorithm for face deformation.(2)Automatic face deformation based on local neighborhood structure constraints and face alignment algorithm.Considering it's difficult and inconvenient to chooseproper control points and corresponding target points for face deformation,we propose automatic face deformation based on local neighborhood structure constraints and face alignment algorithm.This method can automatically transfer the shape and expression of one face to another face.First,a face alignment algorithm is used to precisely locate landmark points of the two faces,and then a global similarity transformation is applied to align the two set of points roughly to eliminate their difference in orientation,size and position.Finally,deforming the original face image with the proposed non-rigid image deformation method,namely fit a mapping function for each pixel of original face.Note that our method can be full-automatic or semi-automatic by combining a face alignment algorithm and a non-rigid image deformation method for conveniently tuning a better result.Experiment results show that our method can produce realistic,natural and artifact-less facial shape and expression transfer.(3)Face transformation based on automatic face deformation and Cycle-GAN model.To solve the problem of automatic face deformation having defects in some specific cases,we introduce Cycle-GAN and train a model to transfer mouth between two state of close and open.When automatic face deformation appears defect,we first use the pre-trained Cycle-GAN model to transform the mouth and replace the mouth of the source face with that of the transformed face,and finally use the source face image and the target face image to deform the original face.In addition,the Cycle-GAN model can also be used for other types of transfer,such as hair color,skin color,and so on,to enrich the conversion effect.(4)Facial animation based on non-rigid image deformation algorithm.We make use of the proposed non-rigid image deformation algorithm and the automatic face transfer method to produce facial expression animation.