Interactive Behavior Recognition Via Multiple Features Fusion

With the increasing popularity of cameras and storage devices,a large amount of videos are produced every day.We want computers to be able to analyze video automatically with as little manual intervention as possible.Behavior recognition is an important part of video analysis.It has a wide range of applications in public safety,autonomous driving,virtual reality and so on,thus,it becomes a research hotpot in video analysis.Interaction behavior is a basic human behavior in daily life,which is more complicated than general individual behavior,since it has complex background and occlusion problems involving a human body.Not only that,there are often large differences in behavior between interactors of the same behavior.These problems increase the difficulty of interactive behavior recognition.This dissertation studies the problems of two-person interactive behavior recognition through CNN(Convolutional Neural Network)and Recurrent Neural Network algorithm in deep learning.Firstly,based on expounding the basic principles of Neural Network and CNN in detail,the role of each layer in the hierarchical structure of the network and the updating process of parameters in the backpropagation process are introduced respectively.Then,VGG16 is adopted to classify video frames.It is found that if VGG16 is used to classify the BIT-Interaction dataset directly,the classification accuracy is very low,since the sample number of this dataset is very small.To tackle the problem,this dissertation uses mirror flip and rotation method to expand the dataset,and thus a higher recognition rate is obtained.In order to further improve the recognition accuracy,the original video frames are enhanced in the preprocessing stage.Experimental results show that the enhancement weakens the background features while highlighting the character features,and thus obtains higher recognition accuracy.Finally,CNN and LSTM(Long Short-Term Memory)are combined to classify the interaction behavior.The features extracted from the CNN are trained in LSTM,wherein the memorizing and forgetting characteristics of the hidden state in LSTM is taken into consideration.Experimental results show that the recognition accuracy is higher than single CNN without increasing data set.Further,features of different scales are extracted and merged by two different CNNs,within the different semantic information of the two scale features is fused,then,the network parameters of LSTM are trained for classification,and a higher recognition accuracy is finally obtained.In addition,experiments with different layers of LSTM are discussed for effect of number of layers on recognition rate.