Lightweight Neural Network Research On Human Pose Estimation

In computer vision,human pose estimation is a task that locates a person's joint points based on images or videos.Human pose estimation is the basis and prerequisites for a number of important computer vision tasks,such as action recognition,person re-identification,human-computer interaction,behavior recognition,animation modeling,image retrieval,intelligent video surveillance,etc.With the rapid development of deep learning technology in recent years,some scholars have begun to use deep learning technology,especially convolutional neural networks,to estimate human poses,which has improved the accuracy of human pose estimation.However,there are still some difficulties in deploying convolutional neural networks based human pose estimation algorithm to actual scenes.One of the key problems is that because the neural network models have a large amount of parameters and calculations,reaching or even exceeding the limited storage and computing capability of mobile terminals such as mobile phones,making it difficult to deploy the human pose estimation network to the mobile terminals.To deal with above problems,this thesis studies the convolutional neural network algorithm in human pose estimation,and proposes several lightweight human pose estimation network structures,then verified the accuracy of these networks through experiments.Under the premise that the accuracy of the human pose estimation does not change much,the running speed of the human pose estimation network is greatly improved and the space required for storage is also reduced at the same time.The main contributions of this thesis are summarize as follows:(1)We introduce feature extraction modules commonly used in image processing tasks such as RFB(Receptive Field Block),HDC(Hybrid Dilated Convolution)etc.,and made it suitable for human pose estimation by adding skip connections and designing new branchs.At the same time,compared to the original modules,the new modules' s key point information extraction ability has been improved,while maintaining lightweight.(2)We introduce several commonly used networks in human pose estimation tasks such as hourglass network(HRG),high-resolution network(HRNet)etc.,Designed some human pose estimation network structure such as deep fusion network,multi-scale fusion network,inverted pyramid network,encode-decode network,S-net,according to ideas such as fuse different scales information,increasing contextual information,and continuously refine features.(3)Based on the networks designed in(2),we use data augmentation,Online Hard Keypoints Mining mechanism and other mechanisms to enhance the robustness of the networks,and test the networks' s performance in the COCO Keypoint dataset.The experiment results show that the proposed networks greatly reduce the calculation and storage costs while accurately locating key points.The best performing deep fusion network is close to the state of art results on the COCO Keypoint dataset.Finally,the real-life scenario test also proved that the designed networks have good robustness in complex environments.