Human Action Recognition Modeling Spiking Neural Networks In Visual Cortex

Human action recognition from video sequence is a challenging research mission in computer vision. It mainly includes two reasons. On the one hand it is the human action inherent complexity, such as its uncertainties and anthropometric differences between individuals, makes the research area face enormous challenges. On the one hand it is intensely vulnerable to the environment, such as clothes, background and camera movement. In recent years, researchers have carried out a large number of action recognition research and made the fruitful research achievements. However, because of particularity and complexity of the human action, some achievements are limited.As the brain science research constantly goes deeper, the understanding to the information processing mechanism in visual cortex is continually enriched, and brain-heuristic action recognition becomes a hot Up to now, a large part of models focus the feedforward architecture of primary cortex(V1) and middle temporal area(MT) properties, while ignoring some important characteristics of neuron in V1 and MT region. These systems not only lack biological plausibility but also have an imperfect recognition performance. In this paper, combining with the latest research results of visual neuroscience, we carry out research about the simulation of the V1 and MT spiking neural network information processing mechanism to deal with temporal and spatial video information and human action the spatial-temporal characteristics extraction and recognition, and this research has obtained corresponding results.Firstly, this paper constructs the computational model of simulating the visual cortical information processing mechanism for spatial and temporal information detection and action recognition. The model simulates the spiking neural network of V1 and MT region dedicated to motion information processing mechanism in the ventral stream to attain the effective spatial-temporal video information.Secondly, this paper also proposes a connection map computing method between V1 and MT neuron based on previous neuroscience studies. Meanwhile, we give out the computing method of surround inhibition in V1 and MT region, in order to satisfy information processing method about speed and direction selectivity and implement the complete and effective motion information.Furthermore, we augment the operator of focus on the action to model visual attention mechanism. This operator obtains the most effective spatial and temporal information of the video object and ensures the scale invariant features of video object.Finally, combining with spiking neuron model, we propose a new analytical method based on the mean spike rate for spike train of spiking neuron. Using the mean spike rate of neurons on the local time, the method constructs the mean motion map to extract human action feature.Our system takes support vector machine(SVM) as classifier. We evaluate our approach for action recognition on Weizmann and KTH datasets and obtain a great quantity of data. Experimental results show that our approach achieves video object detection and action recognition and the performance is competitive as compared with current state-of-the-art methods. At the same time, we also find the feature representations of MT region outperform that of V1 region, which has biological rationality.