Research On Multi-view Individual Action Recognition Based On ResNet And LSTM

Research on multi-view individual action recognition is a frontier topic in the field of computer vision.It aims to extract relevant features by detecting multi-view action data,and then understanding the action features to realize the process of action classification.The diversity of individual actions,the similarity of different types of actions,and the differences of the same types of actions all increase the difficulty of recognition;the complexity of shooting scenes,the intensity of light and the variability of perspective also affect the recognition effect.In order to improve the effect of multi-view individual action recognition,this paper builds two types of network models based on Res Net and LSTM,and uses them in the IXMAS and i3 DPost datasets.A relatively ideal recognition result has been achieved.The main research contents of this paper are as follows:(1)Preprocessing of IXMAS and i3 DPost datasets.In order to improve the recognition effect,two kinds of multi-view action datasets need to be preprocessed.The processing process includes image enhancement,unified size,data enhancement,sequence reconstruction and dataset partitioning,etc.Two kinds of datasets are obtained: a two-dimensional image dataset and a three-dimensional image sequence dataset.(2)Use Keras framework to build 3DRes Net-LSTMs cascade network model.3DRes Nets can not only obtain the appearance and timing characteristics of the action sequence at the same time,but also increase the depth of the network model;LSTM is used to solve the gradient problem in the training process of the action sequence.The pre-processed 3D picture sequence is input into the cascaded network model for training,and good results have been achieved.In the experiment,the two views with higher accuracy in the IMAXS data set are also fused,and the recognition accuracy of various actions is improved after the fusion of views.(3)Use Keras framework to build a 2D/3DRes Net-LSTMs fusion network model.The fusion network model uses the Added function to fuse the feature vectors extracted from the 3DRes Net-LSTMs cascade network model and the2 DRes Net-LSTMs cascade network model.The features extracted by the fusion network are more comprehensive,and the recognition effect on the two datasets is not only better than the single cascade network model,but also better than most current mainstream network models,thus verifying the effectiveness of the fusion model for multi-view individual action research.