Research On Virtual Try-on Based On Feature Points Extraction

With the rapid development of Internet and e-commerce technology, people have payed more and more attention to online shopping. The conflict between illusory characteristic of online shopping and personal experience of traditional shopping concept promotes the emergence of virtual try-on system. Interactive virtual try-on online shopping system is convenient, friendly and intuitive, which benefits both buyers and sellers. Generally speaking, a 2D automatic virtual try-on system contains: object detection, feature points extraction, object matting and image synthesis. They lie on wearing position, the wearer's posture, background light and try-on authenticity. Precise facial feature points extraction and seamless image synthesis are the keys to solving these problems.In this dissertation, algorithms of facial feature points extraction and image synthesis were profoundly researched. Based on the classical algorithms, novel and creative improvement was proposed to complete the virtual try-on of glasses, necklace and watch. The main innovative work of the dissertation is listed as follows:1) Improvements of facial feature point extractionThe paper makes some improvements to Active Shape Model (ASM) algorithm against its drawbacks based on the study of face organ detection and ASM algorithms. We increase the training set using horizontal mirror transformation and change the way of local texture modeling process. Firstly, change the sampling direction of some landmarks to reflect their local variations more accurately. Secondly, Sample from a rectangle area with a landmark as center instead of a normal line across it to get richer information describing the landmark. Thirdly, we adopt multi-resolution search method to find new objects in order to improve search efficiency and robustness. The improved ASM algorithm is used in facial feature point extraction and gives more accurate results, providing positioning precondition for the subsequent application of virtual try-on.2) Single image based virtual try-on methodThe paper proposes a variational model for image synthesis in gradient field according to the study of image matting and image synthesis. Combining with the facial feature point extraction algorithm, we apply the method to glasses virtual try-on and achieve automatic and natural results.3) Design of single image based necklace and watch virtual try-on systemThe glasses virtual try-on framework is properly modified and applied to necklace and watch virtual try-on, giving vivid synthesis results, thus accomplishing the system design. The system mainly contains: facial feature point extraction, palm detection and image synthesis in gradient field. The virtual try-on framework can also be used to other accessories like clothing, earring and scarf, and multimedia applications in film and television production as well.The paper has achieved accurately facial feature point extraction results and single image based virtual try-on of glasses, necklace and watch after profoundly research. We have solved low real-time, high cost and poor authenticity problems in virtual try-on and have achieved good research results.