Deep Learning-based Progressive Face Dynamic Morphing

In this paper,progressive dynamic face morphing algorithm based on deep learning is proposed for the first time.This algorithm seeks for a smooth morphing process between the source face and the target face,and the dynamic facial expression information of the source sequence is accurately retained in this process.Recently,there exists many face editing works based on deep learning.Previous works only considered the change of appearance or only the expression transfer.For face morphing,previous works often interpolated in the hidden space or found the one-way mapping between the images,but the hidden space is difficult to control,and the one-way mapping is not accurate,which leads to poor results.For expression transfer,previous works only used RGB images to guide the network,which caused it fail to work properly with large object pose differences.Our work focuses on learning the two-way mapping between faces to guide appearance morphing.Furthermore,we use 3D information to train models for expression transfer based on the 3DMM model.We seek for both the path of the appearance morphing and the dynamic expression information.We use deep learning neural networks to predict the shared vector field between face images,and use the two-way mapping of shared vector fields to accurately define the matching relationship between pixels.In order to assist the network to capture complex two-way mapping,we design Features Alignment Layer,to weakly align feature maps in the feature space.Then,appearance morphing process can be obtained through vector field.For the expression information,our work is based on 3DMM model,which introduces 3D prior information to reconstruct the face and analyze the expression parameters.In order to avoid the insufficiency of 3D label,we introduce an adversarial discriminative network and a semi-supervised training way to learn the 3D information distribution.By the expression transfer module,the appearance morphing sequence can retain expression parameters of the source sequence and obtain the output.Experimental verification is performed on the mainstream face datasets MMI,CK(CK +),300W-LP.Comparison and analysis with previous works proves that the algorithm is novel and robust.