Exploring Feature Extraction Methods For Action Recognition With Complex Environments

Action recognition (AR) has been a hot spot of computer vision research. AR has promising application in virtual reality, augmented reality, smart surveillance, motion analysis and video search based on content. AR of simple action is getting mature. How to improve the recognition rate and efficiency for AR with unconstrained video has been a very important topic.For improve the recognition rate, there are two main research directions for AR which are feature extraction methods and classification methods. As the input of classifier, the performance of feature impact on the classifier performance directly, especially on the efficiency and recognition rate. In this dissertation, the feature extraction methods with complex environment are studied, aiming to improve recognition rate and efficiency. The main achievements are as follows:1) Motion boundary sampling is proposed to reduce memory, computation cost. And explore how this sampling method influence means average precision (mAP) with improved dense trajectories (IDT). The proposed methods not only assure the recognition rate, but also improve the efficiency. It takes only half of trajectories compared to improved dense trajectories for realistic datasets.2) A linear dimension reduction method is proposed in AR which uses dimension reduction method repeatedly in the process of feature extraction. The dimension of local descriptor is reduced before extract FV. The dimension of FV is reduced before it fed to classifier. Through these processes, it can improve recognition rate and efficiency. The experiment results show that this method significantly reduces computational cost and improves accuracy. And the training and testing time decrease 1-2 orders of magnitude, the accuracy of UCF11 increases to 92.51% and HMDB51 to 56.47%.3) Extreme learning machine (ELM) and fusion strategy for all kinds of descriptors is designed. Fusion strategy includes early fusion strategy which has placel fusion and place2 fusion, and late fusion strategy which is place3 fusion. To improve efficiency and recognition rate with large-scale AR, six rules of late fusion are used to analyze and compare recognition rate. ELM is an efficient classifier which the training and testing time decrease one order of magnitude compared to SVM. What’s more, place2 fusion, the sum and product of place3 fusion are a stable fusion and have the hightest mAP.Based on the above research, a framework of AR based on the combination of motion boundary sampling with IDT and dimension reduction techniques is proposed. The experiment results show that this method not only significantly reduces computational cost but improves accuracy. The method to determine the number of principal components from principal components analysis is designed. Five rules are analyzed to determine the number of principal components which are power rate analysis, median_eigvalue, median_oneth, broken_stick, and modify_broken_stick, respectively. The experiment results show that the modify_broken_stick and broken_stick methods achieve the best recognition rate and about 0.5% accuracy increased compared to the full power rate. And the efficiency is also improved.