Research On Gesture Recognition Technology Based On Convolution Neural Network

With the rapid development of computer industry and artificial intelligence,the human-computer interaction technology,as an indispensable part of it,has attracted more and more attention.Among many means of human-computer interaction,gesture interaction is the most efficient,natural and comfortable way of interaction.Dynamic gesture recognition technology can be applied in smart home,virtual reality and other different occasions,with a very wide range of application scenarios.In the scene of gesture recognition based on vision,because of many problems,including hand self occlusion,gesture movement diversity and complex background interference,dynamic gesture recognition technology is not popular in people's daily life yet.Therefore,finding a recognition scheme with high efficiency is an important topic in the field of gesture recognition.Based on above-mentioned background and difficulties,this dissertation designs a dynamic gesture recognition model combining key points of gesture with the application of convolutional neural network theory.The main work contents are as follows.The extraction network of gesture keypoints,CPM-gesture,is designed and implemented.For the input RGB gesture image,the thermal response map,Heatmap,of hand key points is generated by CPM-gesture integrating the texture features and spatial constraints of hand joints,which is used to represent the position of each hand joint.Then an efficient CPM-gesture model is obtained after conducting experiments to test the parameters in two open datasets.The experimental results show that the average accuracy of the final model to predict the key points of hand gesture is 86.2%,which can completely extract the spatial features of hand gesture and lay a good foundation for the dynamic gesture recognition stage.After analyzing the characteristics of the information needed in the process of dynamic gesture recognition,C3D(Convolutional 3D)is selected as the network prototype in the stage of dynamic gesture recognition.In order to improve the performance of the model,the model of dynamic gesture recognition,which is combined with gesture keypoints,is designed and realized,with the design of a key frame extraction method based on gesture motion information and taking two modal data of joint thermal response map and RGB images generated by CPMgesture as the model input.Then,the fusion strategy of multimodal data features is analyzed,and the two data features are fused in series to complete the final dynamic gesture recognition.In order to verify the reliability of the model,first of all,the above improvement strategies are used to compare with other methods in the selected open dataset Iso GD,and the experiments show that all of the above improvements can improve the performance of the model.And then,the algorithm proposed in this dissertation is compared with the mainstream gesture recognition algorithm of using the same dataset in Cha Learn Isolated Gesture Recognition Challenge Round.The results show that compared with the algorithm used by the champion team of Cha Learn Isolated Gesture Recognition Challenge Round,this method improves the recognition accuracy rate by nearly 3%,which proves the feasibility of the algorithm.