Deep Learning Based Human Body Keypoints Detection And Application

With the development of deep learning and mobile Internet,more and more deep learning applications have begun to be deployed on mobile devices to provide services to people.Among these applications,the analysis of the pose and movement of the person in the image is the fo-cus of human-computer interaction,so the detection of human key points has become a research hotspot as a key technology.In current human body keypoint detection methods,more compli-cated network structures are often designed due to the emphasis on improving the accuracy of the model,so there are usually problems that the model calculation consumes a lot and takes up a lot of space.However,due to the limited computing power and storage of mobile devices,such models are difficult to deploy to devices.This thesis aims at the detection of human keypoints,with the goal of eventually deploying the model to mobile devices.It is dedicated to researching human keypoint detection models that can balance computing consumption and model accuracy and can be deployed and used on mobile terminals.It implements the function of human body similarity analysis algorithm in various scenarios.The main work and innovation achievements are as follows:1.Starting from optimizing the network structure,based on OpenPose,a human keypoint de-tection network that balances computational consumption and model accuracy is obtained by modifying and optimizing.The original OpenPose network is divided into two modules:feature extraction and detection optimization(Refine).This article has modified the con-volution module and redesigned the structure of the Refine module.The resulting network model calculations(GFLOPs)are only one-fourth of the original network.The accuracy of the model is only about 6 percentage points lower than the original model.The model file size is only 8MB.It can better handle the situation where multiple people have overlapping limbs,a large number of people,and complicated situation in the picture.After obtaining the optimized network model,it has the characteristics of low computing consumption and small storage consumption running on the mobile terminal.Based on mobile deep learning framework TensorFlow Lite,the model is deployed on Android devices.The final model has good accuracy,and as a multi-person human key point detection model,on an Android device equipped with Snapdragon 855,the GPU is used for inference to achieve a frame rate effect of 18 fps,which has good real-time performance.2.For practical applications,it is necessary to analyze the similarity of actions between differ-ent characters based on the key points of the human skeleton in the image.Based on cosine similarity and dynamic time warping methods,this thesis proposes a motion similarity anal-ysis algorithm.For fitness scenes,it can count the number of fitness movements.For dance scenes,it can analyze the similarity of the dance movements of the two characters on the video of the duo dance,and analyze the joint angles that are quite different in some intervals,providing a reference for the learning and improvement of movements.