Action Recognition Of Human Skeleton Motion Sequences Based On Deep Learning

Human action recognition is a key research direction in the fields of computer vision and artificial intelligence.Its research results are widely used in intelligent monitoring,humancomputer interaction,motion analysis,virtual reality and other fields,has important academic research significance and great potential for market application.Aiming at the task of human action recognition under the video of daily life,in order to preserve the more characteristics of the time and space dimension of the action in the real scene,through the spatiotemporal motion characteristics of human action in the video,the fusion is designed with human skeleton,RGB graph and optical flow as input,Behavioral recognition neural network framework TS-IC3D-LSTM with timing.The validity of the proposed algorithm is verified on the self-built database,and the superiority of the algorithm relative to other related algorithms is verified on the UCF101 dataset.The main research contents are as follows:1.In order to reduce the influence of external illumination,occlusion and other factors on feature extraction.Firstly,the characteristics of human skeleton in target video sequence are extracted by feature extraction module,and the motion sequence of human skeleton is obtained.To normalize the coordinates of the skeleton joints in order to eliminate the influence of the absolute spatial position of the human body on the recognition process.Then the skeleton joint point is filtered by a simple smoothing filter to improve the signal-to-noise ratio.Finally,the characteristics of the human skeleton are characterized and output in the form of a characterization matrix.2.In order to solve the problem that the single feature has insufficient characteristic representation when describing human action in video.The spatial information is characterized by fusion of RGB graph and human skeleton motion sequence,and the time information of optical flow characterization is added.In order to solve the problem of computational redundancy in video data,it is also proposed to serialize the image frame of video data and extract the keyframes.Minimize redundant calculations while ensuring the correlation between image frames.3.In order to solve the slow operation speed of large input data flow algorithm,the IC3 D neural network is constructed by the expansion of convolution core and the improvement of structure depth based on 3D CNN.The use value sharing strategy reduces the parameter expansion problem in the neural network,and through continuous convolution and lower sampling,the problem of the next working parameter is reduced,in order to improve the computational efficiency of the whole algorithm.At the same time,adding LSTM constitutes the sub-network module in the TS-IC3D-LSTM Neural network,which solves the problem of gradient explosion and gradient extinction which may occur because of the large input flow through the structure.Then,the optimized network structures such as BN and Dropout are added to prevent the over-fitting in model training.Finally,the validity of the proposed algorithm is verified on the self-built behavior database.4.Based on Two-stream and LCRN,a TS-IC3D-LSTM neural network for human action recognition tasks in video is designed.Optical flow input is used to characterize the time dimension between samples,and the human skeleton motion sequence and RGB graph are used to characterize the spatial dimension.The network model training is carried out by the method of pre-training weight migration plus fine tuning and retraining.The robustness and superiority of the algorithm are verified by comparing it with the relevant algorithms on the UCF101 of human action video dataset.