Research On Air Flip Book Technology Based On Depth Camera Gesture Recognition

With the rapid development of science and technology,human-computer interaction methods have become more diversified,natural and comfortable,such as newer touch,voice,gesture and other interaction methods.Among them,the use of gestures to achieve humancomputer interaction has become a hot topic at home and abroad.Recognition technology has grown from the original data glove-based,2D camera to 3D camera that is now more popular among researchers.Gesture recognition-based air flip book allows users to use gestures to control e-books with rich data(including animations,videos,pictures,etc.),to achieve the effect of flipping and viewing the contents of the book.Its novel mode can make the exhibition hall,library and government and other units increase their influence in notification,science popularization and publicity.Not only can they display rich information,but they can also attract people to stop and watch.It has the effect of science popularization and good application prospects.At present,most of the book flip technologies use infrared sensing or gesture recognition methods based on 2D cameras.The disadvantages of those methods are limited functions,low interest,and easy to be affected by the surrounding light environment,resulting in false recognition.In order to solve the above problems,this paper proposes a method to detect the start and stop of continuous dynamic hand gestures based on the idea of FSM(finite state machine),and an improved DW-DTW(distance weighting dynamic time Warping)method for calculate the distance between the reference gesture sample and the test gesture sample,which improves the robustness of the dynamic gesture recognition method in low light,strong light and complex backgrounds.In order to enhance the interactivity,real-time,and accuracy of the air flip book,combined with the Intel SR300 depth camera,and using RealSense dynamic gesture recognition technology,this paper designs and implements an air flip book system that controls e-books by recognizing dynamic gestures.The research contents of this paper are as follows:(1)After fully investigating and analyzing the common requirements involved in the air flip book system,a set of gestures including six types of dynamic gestures is defined.A RSDHG(RealSense Dynamic Hand Gesture)database is established,which contains 960 samples of dynamic gesture data recorded by Intel SR300 depth camera.It mainly contains color image sequences,depth image sequences of dynamic gestures,and three-dimensional position information of 22 joint points of the hand.(2)On the basis of the general method of gesture start and stop detection,a method of continuous dynamic gesture start and stop detection is proposed by combining the displacement of adjacent frames of hands and the idea of Finite State Automaton FSM.Feature extraction is carried out from two aspects: the joint trajectory and the hand shape change.The extracted feature vectors are related to node motion trajectory,Euclidean distance,angular features,etc.,which enhances the effectiveness of feature extraction.(3)Based on the traditional Dynamic Time Warping algorithm,an improved DW-DTW algorithm is proposed to calculate the distance between the reference gesture sample and the test gesture sample,and then use the K-nearest neighbor classifier to classify and recognize the dynamic gesture and obtain the recognition result.The best value of K in KNN(K Nearest Neighbor)classifier is selected through experiments.When K is the best value,the recognition rate of DW-DTW algorithm and the traditional DTW algorithm in real-time environment is compared,and the robustness of the dynamic gesture recognition method under low light,strong light and complex background is verified.On the basis of the realization of dynamic gesture recognition,the construction of the air flip book system was completed by stimulating mouse and keyboard events.The system runs smoothly and the interaction experience is good.