Research On Face Replacement In Video Based On Three-dimensional Information

With the hot application of photography and photography,video face replacement technology has become a research hotspot in the field of computer vision,and has been widely used in social entertainment,privacy protection and so on.At present,there are two kinds of face replacement methods in video,one is based on two-dimensional information and the other is based on three-dimensional information.Because the traditional video face replacement based on two-dimensional information completes the operation of video face replacement by capturing the video that the source face imitates the target face.However,when the target face has angle deflection,the source face can not exactly imitate the corresponding deflection angle,resulting in the replacement of the face jitter phenomenon.In order to solve the above problems,we use the threedimensional information of face to complete face replacement in directional video.The methods in this subject are mainly divided into two stages: three-dimensional face reconstruction based on sequence images and face replacement based on video.(1)In the phase of three-dimensional face reconstruction based on sequence images,an improved SIFT(Scale Invariant Feature Transform)combined with cosine similarity is proposed in the process of face feature extraction and matching.By changing the rectangular feature descriptor of the original SIFT operator to a descriptor composed of four concentric circles,the 128-dimensional feature descriptor of the original SIFT algorithm is reduced.Two-way matching is used to eliminate one-to-many mismatching points,and cosine similarity is used as constraint to eliminate a large number of cross mismatching phenomena.The algorithm reduces the time complexity while ensuring the accuracy of face matching,improves the efficiency of the algorithm by 2-2.5 times,and improves the reliability of the whole source face 3D point cloud reconstruction.(2)In the phase of video face replacement,we use the Seetaface face face detection module for face detection,and use the convolution neural network to estimate the angle and pose of the detected face.The generated source face model is adjusted according to the angle parameters of the target face and mapped to the corresponding two-dimensional image.Supervised Descent Method(SDM)is used to locate facial feature points and determine the face contour and Five-Sense pixel blocks.Then,the method of video face registration combined with alignment criterion is used to perform face registration between adjacent frames,and face contours between adjacent frames are aligned.Finally,Poisson fusion algorithm is used to fuse the sequence image of the source face and the video clip of the target face,and the frame-by-frame face replacement of the target video orientation is completed.In this paper,an improved face matching algorithm based on SIFT and cosine similarity is experimentally studied in FEI face database,and compared with traditional face matching algorithm from multiple perspectives.The experimental results show that the face matching algorithm in this paper reduces the computational complexity and improves the time efficiency of the matching,and improves the real-time performance of face replacement in video based on three-dimensional information.In the part of video face replacement,this paper uses the method of face replacement in video based on three-dimensional information,realizes the directional video face replacement function,and evaluates the multi-segment video using Richter test.The experimental results show that the video clips processed in this paper are robust and realistic.