Human Action Recognition Based On Multi-features Fusion

With the continuous increase in the number of videos and the development of deep learning,the research on human action recognition has stepped up several gears.Human action recognition technology based on video content has important applications in fields such as intelligent surveillance,motion prediction,and video target tracking.A successful human action recognition method is the two-stream convolutional neural network model,which uses two networks of temporal stream and spatial stream to combine the appearance and motion features in human action videos to realize the recognition of human action.The information processed by the model(optical flow image and RGB image)is easily affected by factors such as background clutter,viewing angle changes,and object occlusion.Considering that the Wi-Fi signal can overcome the constraints of environmental factors in the task of action recognition,and it has the advantages of low cost,simple equipment deployment and wide applicability.Therefore,based on the two-stream network model,this paper further integrates Wi-Fi signal feature to identify human action in videos.This paper first extracts the appearance information and motion information in human action videos based on the two-stream network model,merges these two kinds of information and inputs them into the LSTM network to obtain the video long-term motion feature.At the same time,the Butterworth low-pass filter and the principal component analysis method are used to denoise and reduce the dimensionality of the Wi-Fi signal corresponding to the video action.Then,use statistical algorithm to extract the CSI signal feature.The motion feature and the CSI signal feature are fused by two different methods,namely the direct average method and the weighted average method.Finally,a linear SVM classifier is used to recognize human action.In order to verify the multi-features fusion human action recognition method is effective and feasible.We first established a basic human action data set,that is,synchronously collecting human motion video and Wi-Fi signal information corresponding to the video action under two different angles,front and side.Then,we design comparative experiments on this data set.The experiment results show that the feature fusion method using weighted average has higher recognition accuracy than the direct average fusion method.It also proves that the multi-features fusion method proposed in this paper improves the accuracy of human action recognition by4.2% than two-stream network model method,which alleviates the deficiencies of two-stream network that is susceptible to environmental factors.