Research And Application On Lightweight Human Pose Estimation

The goal of human pose estimation,also known as key point detection,is to accurately locate human joints in an input human image to construct a skeletal representation of the human body.In recent years,the field of human pose estimation has grown rapidly,but many approaches have pursued higher accuracy while neglecting the need for models to be deployed on the ground.In some specific application scenarios,models are required to be sufficiently lightweight while maintaining high inference accuracy due to limited computational and storage resources.In addition,in some practical application scenarios,the accuracy of key points such as wrists and ankles is required to be high,but in existing human pose estimation models,the prediction accuracy of these limb key points is much lower than that of key points such as heads(these human key points are called hard inference key points in this thesis).Based on the current development status of human posture estimation,this thesis starts the research from the following two ideas for the demand of landing in mobile.The first is to improve the operation efficiency of the model while keeping the accuracy of the model high;the second is to improve the prediction accuracy of the difficult inference key points on the basis of ensuring the model is light enough.Finally,a human posture estimation system for motion counting that can be computed and processed in real time on the mobile side is designed and developed.Specifically,the main work of this thesis is as follows.1.A lightweight network model for human pose estimation is designed.A lightweight module is first designed based on deep separable convolution and attention mechanism,and then a simple and effective lightweight feature fusion model LFFNet(Lightweight Feature Fusion Network)is designed based on this module.The number of parameters in LFFNet is only 3.1M,which is 1/11 th of a large model with comparable accuracy.LFFNet achieves a lower number of parameters while maintaining a higher accuracy,improving operational efficiency and making it more suitable for mobile deployment.2.To address the problem of low prediction accuracy of difficult inference key points in existing human pose estimation models,a loss calculation method DF-Loss(Dynamic Focus Loss)based on dynamic weights is proposed.This method can effectively focus on the feature information of the hard inference key points and improve their prediction accuracy without additionally increasing the number of parameters and floating point operations of the model.In this thesis,we have conducted experiments on various mainstream models and lightweight models to demonstrate that this method can effectively improve the accuracy of hard inference key points and the overall accuracy of the model.3.In this thesis,we design and develop a video streaming-based human pose estimation system for real-time motion counting.A lightweight human pose estimation model LFFNet is used as the underlying model combined with DF-Loss loss calculation method for key point detection,and then inference calculation is performed based on the relative positions of the detected key points to output whether the motion action is standard and the number of actions.The video processing speed of this system in mobile can reach 12 fps,which can be deployed to land on mobile devices for real-time operation and achieve high recognition accuracy while maintaining low computational overhead.