Research And Software Development Of Human Action Analysis Method For Family Environment

China is gradually entering an aging society.With the increase of age,the elderly will have problems such as inability to move their limbs,and poor ability to live independently.They need special nurse from the society.Therefore,research and development of intelligent nurse systems that can provide services for the elderly has important practical significance.This dissertation mainly focuses on daily life scenarios,researches human body action analysis methods and develops related software for home environment,which has good practical application value.The main work of the dissertation is as follows:(1)To reduce the high redundancy between video frames,a clustering-based video key frame extraction method is designed.First,the video is decomposed into a single frame,and then the features of each frame are extracted.Then,the features are used for clustering analysis of all frames of the video.Finally,some frames are selected from the cluster as key frames.Using keyframes instead of video for subsequent processing can significantly reduce the computational complexity.(2)A method of skeleton base action recognition based on video sequences is designed.First,perform pose estimation on the people in the video,then generate a skeleton spatiotemporal sequence diagram based on the results of the pose estimation,and finally use a graph convolutional neural network to identify the skeleton spatiotemporal sequence diagram.Different from the existing action recognition methods of offline pose estimation plus online skeleton sequence recognition,this method can learn human pose and human action end-to-end.Although the designed skeleton action recognition method based on video sequences can directly predict human action in video sequences,errors in pose estimation itself have an indirect effect on the accuracy of action recognition.In order to make up for errors in pose estimation,a human action recognition network based on multi-task learning is further designed.The network is divided into two branch networks,a video sequence and a skeleton sequence.The two branches share the feature extraction network and output a set of feature maps.Finally,the main network fuses the feature maps from the two branch networks and uses a fully connected layer to output Classification results.(3)The software of human action analysis based on Python and Kivy framework was developed,including video decoding,action recognition,early warning and display function modules.The video decoding module obtains the input video address,resolves the correct video type from the input video stream,and decodes the video stream into video frames.The action recognition module uses the designed method to analyze the human action of the video frame obtained by the decoding module.The warning module counts the number of dangerous actions obtained by the action recognition module,and sends the warning message and video clip to the guardian in the form of email for the dangerous actions whose occurrence number is greater than the threshold.The display module outputs video images.In addition,the software also sets the basic parameters of the software through the interactive interface.