Research On Face Swapping Technology Based On 3D Morphable Model

In Computer Vison field, Face Swapping meaning switching source face and target face in image or video. Face Swapping can be applying to privacy protection, entertainment, virtual reality, film and television special effects, video chats etc. In this paper, I design and implement an improved video based face swapping algorithm and system. The results are better in terms of visual effect and processing speed. The main contents and innovation of this paper are as follows:Firstly, with fully research and compare the algorithm of traditional face landmark, an improved CLM-based feature point matching algorithm is proposed,in order to solve the sensitive problem about the initial shape. In additional, in order to enhanced the effect of subsequent face swapping, a new feature point is added to the original CLM algorithm. Head pose estimate by strengthening the estimation of the head posture.Secondly, in view of the traditional face replacement in different face of the replacement effect is not good, put forward to join the head posture adjustment of the face replacement method to enhance the effect. Method is to the source face and the target face of the head of the movement to do the calculation, access to its rotation parameters. In the replacement of the face, adjust the two head posture, do face fusion, so that the replacement effect more realistic.Thirdly, face to face replacement when the difference between the face of the problem, put forward in the face replacement, automatically adjust the face of the algorithm. The method is to adjust the target face area to be similar to the source face for the result of the feature locating, and to replace the face. In the video, when the larger area is adjusted to a smaller area, the inter-frame value is used for compensation, and the area that cannot be compensated is stretched. In order to improve the face of the face of the differences in the problem. This paper realizes the real-time and video face replacement system. Experiments show that the head posture estimation method proposed in this paper is effective and the expression transfer function in face replacement has a good effect. This article deals with the traditionalFourthly, I use the face segmentation algorithm to solve the surrounding context or occlusions confusing face area problem. FCN-8s network is used to solve the problem. Using up sampling and skip strategy, the result of the face segmentation will be more accuracy. Significant effect of face swapping is showed because of face segmentationFinally, I design and implement an expression-based replacement system based on video with face segmentation, and optimize the processing flow and processing speed of the system. Will be different modules can be reused algorithm to extract the acceleration, and achieved a good speed effect.In summary, this paper analyzes the processing of human face swapping process, summed up the comparison of the commonly used algorithm, and selected for the face replacement algorithm. Experiments show that the robustness and real-time performance of the algorithm are optimized.