The Key Technology And Application Platform Of Online Eyeglasses Fitting Based On Hand Gesture Recognition

In recent years, with the rapid development of mobile Internet, online shopping becomes the popular way of consumption. The concept of "Internet+", has greatly promote the traditional industry thinking transformation using the Internet way of thinking and create new profit growth point. Traditional way of online eyeglasses sales is the seller took various angle's photos and upload to the Internet, consumers know about the eyeglasses though these photos. However, this kind of sales way cannot meet consumer's need of wearing. In the same time, in the traditional eyeglasses sales mode, because of the limitation of merchant store, they cannot cover various eyeglasses and cannot offer the personalized eyeglasses service. Virtual try-on system can make up for the shortage of traditional online shopping and make it possible for try-on eyeglasses at home and buy online.For online eyeglasses sales, this paper construct an online eyeglasses fitting system combined with hand gesture recognition based on augmented reality, including face detection, face feature points location, head pose estimation,3D eyeglasses modeling, hand gesture recognition and online eyes distance measuring. The innovation of proposed system is displayed in following:1. Computer-human interaction based on hand gesture recognition. This paper proposed a method of skin detection based on Cb-Cr color difference. We design a simple and effective gesture detection and identification method. The users can select eyeglasses though hand gesture, making the interaction more natural.2. Implement of the accurate and stable face feature tracking and head pose estimation. This paper use CLM algorithm for face feature detection. At the same time, combining with POSIT algorithm and 3D face model, we realize the head pose estimation.3. Online eyeglasses fitting based on 3D model. Automatic loading eyeglasses model, adaptive matching different face shape, display the multi-angle try-on result.4. Online eyes distance estimation. We design a method of eyes distance measuring method based on reference. Users can easily get the reference, and the measuring process is simple.In this paper, we design an online eyeglasses fitting platform combined with hand gesture recognition. This platform solves some problem in online try-on methods such as real-time absence, poor human experience, and high cost. The result shows that our system obtained good performance.