Human Skeleton Keypoints Detection Based On Deep Learning

The detection of the human skeleton keypoints refers to the process of locating the main joints of the human body in the images or videos,and its function is to serve the classification or recognition of the human action.Safety monitoring,human-computer interaction,digital entertainment,sports analysis and other fields are inseparable from the analysis of human action.Therefore,the in-depth study of human skeleton keypoints detection has broad application prospects.The localization of human skeleton keypoints is one of the most challenging tasks in the field of computer vision.On the one hand,the diversity of human pose and observation angle,joints occlusion and self-occlusion,the complexity of illumination and background environment increase the difficulty of locating joints;on the other hand,many application scenarios require not only high accuracy of locating joints,but also good real-time performance.Traditional visual algorithms,with so many complicated factors,have been difficult to meet the high accuracy requirements of the application.The current method based on deep learning improves greatly in accuracy,but in most cases they are offline.Especially when the application requires three-dimensional skeleton keypoints detection,the processing speed of single frame or single picture is very slow,the demand for hardware performance is very high,and the real-time performance demand is difficult to meet.To solve these problems,this paper improves and optimizes the Stacked Hourglass network,which improves the speed and accuracy of single person skeleton keypoints detection.The mainstream object detection algorithm is improved and optimized to improve the accuracy of human detection.Combined with improved and optimized object detection algorithm and single person skeleton keypoints detection algorithm,multi-person skeleton keypoints detection based on image and video is realized.And the three-dimensional multi-person skeleton keypoints detection is realized by the binocular camera.Among them,video-based multi-person skeleton keypoints detection and three-dimensional multi-person skeleton keypoints detection not only have high accuracy,but also meet the real-time performance requirements of application.The main work is summarized as follows:(1)Using the Stacked Hourglass network,the single person skeleton keypoints detection is realized,and the effect of different order Hourglass networks on human skeleton keypoints detection is studied.The Stacked Hourglass network was improved and optimized,and the accuracy of single person skeleton keypoints detection for the network was evaluated on the MPII Human Pose Database and AI Challenger Human Skeletal System Keypoints data sets.(2)Accuracy and speed of human detection for mainstream object detection algorithms are evaluated.For human detection,Faster R-CNN and YOLO are improved and optimized to improve the recall rate and accuracy of human detection.Combined with improved and optimized object detection method Faster R-CNN and single person skeleton keypoints detection method Stacked Hourglass(N-4OSH),high-precision image-based multi-person skeleton keypoints detection is realized.The optimized object detection method YOLO and single person skeleton keypoints detection method Stacked Hourglass(FN-3OSH)are integrated to achieve real-time detection of multi-person skeleton keypoints based on video under high-precision joints localization.(3)On the basis of real-time detection of multi-person skeleton keypoints based on video,the binocular camera is selected.According to the principle of stereo vision,the least-square method is used to realize the three-dimensional multi-person skeleton keypoints detection,which meets the real-time performance requirement of application.