The Research And Application Of Face Tracking And Replacement In Movies

The technology of video face tracking and replacement is very potential in both art and entertainment industries. The technology can not only satisfy the special requirements of shooting, but also bring audiences much interesting viewing experience. It is easy for users to replace the character’s face by their own faces so that more pleasure is obtained during this process.So far, frame-to-frame face replacement in video is always time-consuming and labor-intensive. However, our system is more efficient and easy to operate. The system tracks the face and aligns the contour in target video first, and then reconstructs the 3D face according to the frontal photo. The face after replacement has the similar pose and light with the target one.We applied Mean-Shift algorithm as tracking algorithm and proposed a new similarity measure to deal with the drawbacks in traditional MS algorithm. We also applied verification mechanism to improve the efficiency and robustness of tracking algorithm. In the stage of face replacement, we applied the 3D morphable model and an expression database. After expression matching, pose estimation, face relighting, the result is ideal. The input information of the source subject face is reduced to one frontal photo. The face replacement procedure requires little user interference. To demonstrate the effectiveness of our system, we test our implementation on several videos...