A Study Of Human Action Recognition Based On Spatio-temporal Features

With the rapid development of pattern recognition and computer vision, the research of human action recognition has been becoming an important research area. Its applications include surveillance systems, patient monitoring systems, human-computer interfaces, smart home applications, content-based video retrieval, athlete assistant training which has a very broad application prospects and potential economic and social value. However, since the difference between inner and outer action classes, the larger perspective diversity, mutual occlusion and other factors, have brought a lot of difficulties to the study of human action recognition. Therefore, finding relatively robust descriptors that adapted to the changing environment is becoming an urgent problem. The proposed space-time local features are scale- invariant and shift- invariant which make it robust and practical. Space-time local feature-based approach has become a very proming research area in human action recognition research area. A lot of work has been done in this paper to find efficient low dimensional image descriptors, reduce feature extraction complexity and improve the robustness of action recognition. In recent years, since human action recognition system has been gradually used in commercialization, and made some progress, many people have joined this research. The main contributions of this thesis are as follows:1. Propose a human action recognition method based on space-time gradient features and motion features. By using space–time auto-correlation of gradients(STACOG) extracts the local relationships, such as co-occurrence, among the space–time gradients by means of the auto-correlation functions. Then, add the human motion feature, that is, instantaneous frame rate of image sequences in video. Finally, the concatenated features are as the input of the classifier and has obtained a goog performance.2. Propose a human action recognition method based on non- negative matrix factorization(NMF) and sparse reconstruction classification. First, the spatial-temporal interest points are detected and construct cuboids around the points, then local motion pattern features are extracted. Second, class-specific dictionaries are obtained by using NMF. Then, the features are reconstructed and classified by the residual. In order to ensure combining elementary object features only additively, a new framework is proposed by applying local space-time features and NMF.3. Prospose a human action recognition method based on multi- scale spatial-temporal local features. First, the spatial-temporal interest points are detected and then by suppressing the background interest points and adding local and temporal constraints to the spatial interest points, the selective interest points are obtained. Finaly, the local motion features are extracted around the selective interest points. The proposed method that extracted the selective local motion features around selective interest points is superior to those traditional spatial- temporal interest detection methods which always detect unwanted points in the background, so the recognition rate is improved.