Research And Application Of Human Action Recognition Technology Based On Skeleton

At present,the importance of human behavior recognition in various fields such as intelligent nursing,intelligent traffic pedestrian warning,and human-assisted medical care has been continuously highlighted.Its huge development prospects have also attracted more and more researchers to invest in this field.However,due to the influence of various realistic factors such as occlusion,lighting,and the diversity of human behavior,it is still very challenging to accurately and quickly recognize human behavior.At present,compared with the behavior recognition method directly based on the video stream,the human behavior recognition method based on the key points of the human skeleton can shield most of the interference of background factors and filter out the huge amount of redundant information in the RGB image in the video stream.However,the bone sequence is non-Euclidean data.Traditional CNN is powerless to extract the features of key points of bones and non-European structures,and the existing graph convolutional network has the second-order information of key points of human bones,such as the length and direction of the bones.And the use of human bone movement is less.In order to solve the above problems,the research content of this thesis includes three parts:(1)Propose a three-streams human behavior recognition algorithm based on a human skeleton composed of key points of human bones.The algorithm uses graph convolutional networks to learn the topological structure of the human skeleton's action characteristics,respectively model the key points of the human bones,bone vectors,and bone motion information,and merge the results,and use an adaptive mechanism to construct a common for each action sample.Global map and unique map.And compared to CNN,the adaptive graph convolutional network has a stronger ability to extract topological structure data,and includes the learning of second-order information such as bone length,direction,and bone motion information.Compared with ST-GCN,this algorithm has achieved a significant performance improvement in the main data sets Kinetics-Skeleton and NTU-RGB+D.(2)An improved Deep SORT algorithm(I-Deep SORT)is proposed to reduce the number of ID Switches of the original algorithm.Add HOG(Histogram of Directional Features)feature extraction and matching to Deep SORT algorithm to achieve accurate tracking.On the basis of human target detection,I-Deep SORT is used to track a specific human body.In addition,the trained DNN behavior recognition model is used to perform real-time behavior recognition of the human body on the basis of the human body posture estimation and acquisition of the human skeleton sequence,and combined with the YOLOv3 detection model and The I-Deep SORT algorithm realizes the recognition and tracking of specific people's behavior.Experiments show that the number of ID Switches has dropped by nearly half.(3)Design and implement a behavior recognition control and display system based on ROS(Robot Operating System).The system includes human skeleton and behavior recognition,human detection and tracking modules on the Turtle Bot2 robot,and mobile App control and display modules.On the Turtlebot2 robot,the human bone sequence is obtained through the pose estimation algorithm and the behavior recognition based on the human bone sequence is realized,and the human behavior recognition result is merged with the human body detection and tracking result to realize the behavior recognition and tracking of the specific human target.Build a mobile app to control and display the results of human behavior recognition and tracking on Turtle Bot2 in real time.