Human Motion Evaluation Method Based On Multimodal Information

Human motion analysis has long been an important research direction in the field of computer vision.With the rapid development of artificial intelligence technology,human motion analysis has been widely used in areas such as security monitoring,medical rehabilitation,and sports.In vision-based human motion analysis,the main input modal information is image information and bone information.Each modal information can reflect a certain characteristic of human movement.Human motion analysis based on multi-modal information combines the advantages of different modal information to improve the accuracy of the analysis.How to use multi-modal information to model and analyze human movement is the current research focus.This paper uses multi-modal information to conduct research from four aspects: human pose estimation,human pose recognition,human movement recognition,and human movement evaluation.The main work and innovations of this article are as follows:(1)Aiming at the problem of inaccurate detection of human key points and misconnection between key points in human pose estimation based on traditional Open Pose,an Open Pose method based on pose flow optimization is proposed.The human pose stream information in the image sequence is used to optimize the human pose estimation results of Open Pose,which makes up for the shortcomings of traditional Open Pose that ignores the image information between frames.The experimental results show that the spatio-temporal map convolutional network based on attention mechanism can emphasize the important action parts of human visual attention,and the recognition rate is improved by 8.8% compared with the traditional network model.(2)Aiming at the problem of ignoring human visual attention strategy in human pose recognition based on traditional Spatial Temporal Graph Convolutional network,Spatial Temporal Graph Convolutional network based on attention mechanism is proposed.Improved weight function in graph convolution operation based on skeletal point division method based on fusion attention mechanism.In order to simulate the human visual attention mechanism to analyze and recognize human posture.The experimental results show that the Spatial Temporal Graph Convolutional network based on attention mechanism can emphasize the important action parts of human visual attention.Compared with traditional network models,it improves the recognition accuracy.(3)Aiming at the problem of insufficient expression of context information in human motion recognition based on Two-stream Long Short-Term Memory networks,a Two-stream Spatial Temporal Bidirectional Long Short-Term Memory network is proposed.A bi-directional long-term and short-term memory network is constructed using a bi-directional long-term and short-term memory unit.The dual channels of the network analyze the skeletal time characteristics and skeletal space characteristics of human movements respectively,and fuse the decision to obtain the human movement recognition results.Experimental results show that the model fully considers the relevance of the context information of human action sequences.Compared with traditional networks,it improves the network model performance and recognition accuracy,and improves the accuracy of human action recognition by 3%.(4)Aiming at the action evaluation of eight-style Taichi evaluation system based on multi-modal information was developed.Firstly,the Kinect2.0 depth camera was used to collect multi-modal information of human motion,that is,color image information and bone information.Secondly,the human motion characteristics are constructed based on multi-modal information.A human motion evaluation model based on dual-channel support vector machine is proposed again.Finally,Taichi evaluation system was developed based on software and hardware platforms.Experimental results show that the system can accurately evaluate Taichi movements in real time,and has both intelligence and interaction.