Deep Human Action Recognition In Natural Human-Computer Interaction For Web3D Engine

Motion recognition,as one of the main technologies of computer vision and image processing,has also been valued and widely applied in the fields of human-computer interaction and virtual reality.Due to the problems of inconsistent coordinates,difficult bone dynamic modeling and different amplitude rhythms and amplitudes of the same action,it is difficult to achieve high accuracy in motion recognition.At present,extracting skeletal joints from temporally related video actions can provide a good representation for human motion description.Among them,how to obtain the skeleton joint points,analyze the dynamic correlation of the joint points in time,and obtain a good effect of motion classification has become a problem.This paper mainly proposes a method based on the main key points of the human skeleton to construct an optimized deep learning for skeleton-based Action Recognition using Temporal Sliding LSTM networks to complete the task of action classification.Finally,the above parts are applied to the interactive process of Web3D engine to realize the application of motion recognition in the field of early childhood education.The main research work in the paper includes:(1)Aiming at the problem of extracting the human skeleton from the camera,we first processed the video frame image through the histogram equalization method to ensure the key points in its fluency and stability frames.In order to ensure real-time performance,all operations of video processing are implemented in CUDA to speed up,and then use Openpose which is open source library to extract human skeleton information.(2)Aiming at solving the problem of motion classification,we chose a new integrated time sliding LSTM network for motion recognition.First,we transformed the coordinates of the input skeleton sequence to make the data have strong scale,rotation and translation capabilities.Then we use the time difference of the motion features to make the network focus on the actual skeletal motion,and use multiple LSTMs to process the motion features to maintain short,medium,and long-term LSTMs.Finally,multiple LSTMs capture multiple motion dynamics and integrate through integration Network model.The experiment proves that the model has achieved good results on its own data sets and other public data sets.(3)We have designed a set of deep action recognition system for Web3D enginefor online children's education.We take fitness teaching as the application scenario,creatively apply motion recognition to children's fitness enlightenment and teaching,and use the researched human skeleton key determination method and time-sliding LSTM motion classification technology to connect with the Web3D engine to form a complete closed-loop interaction.Finally,experiments prove the feasibility and reliability of the system.