Research On Human Activity Recognition With RGB-D Large-scale Dataset

Entering the 21st century,with the development of information technology and the constantly intellectualization of human life,computer vision is increasingly affecting all aspects of people's life.Human activity recognition and analysis,because of its wide application prospect and practical value,has been a hot spot in the field of computer vision in recent years.Human activity recognition' is analyzing the original video image sequences,extracting relevant activity feature information,and finally explaining the information in order to realize the human activity recognition and learning.Despite the rapid development of computer technology and image processing technology has greatly promoted the research in the field of activity recognition,and with the popularity of big data technology,algorithm performance has increasingly depended on the dataset to a large extent.However,due to difficulties of choosing effective activity features;occlusions,scene simplification and lacking a large number of samples in most existing datasets,human activity recognition technology research based on big data under complex nature scenes still remains a very challenging problem.RGB-Depth(RGB-D)sensor can provide color and depth images simultaneously,3D depth information will be obtained directly with no extra computational cost.This provides great convenience for the application of depth information in human activity recognition.Human activity recognition and analysis is relying on the activity datasets,in the process of research on activity recognition,a variety of datasets have appeared successively.The existing public RGB-D activity datasets are difficult to be used for human activity recognition on big data under complex natural scenes because of the limited action category and sample size as well as the simplifying background environment.As a consequence,this paper proposes a RGB-D large-scale activity dataset for advancing the state-of-the-art in human activity recognition under complicated natural scenes,at the same time,we studied 3 feature extraction algorithms based on this dataset.The main research work of this paper is as follows:First,this paper analyzes the background,significance and purpose of human activity recognition research,and reviews the current status of human activity recognition technology from three aspects:dataset,feature extraction,classifier.We describe the current problems of activity recognition based on RGB-D and introduce the main content of this paper and its chapter arrangement.Second,advantages of RGB-D sensor and the importance of depth information in human activity recognition are described.We introduce some of existing RGB-D datasets in detail,and compare their advantages and disadvantages.Third,we choose five typical and commonly used RGB-D datasets,and make preprocess and statistical analysis to them.A comprehensive RGB-D large dataset is created by integrating the five sub-datasets.We re-label all activity samples in the comprehensive RGB-D large-scale dataset and unify the data storage format.This section mainly gives a specific description to the work of building the comprehensive large-scale dataset.Meanwhile it also introduces the data information,superiority and significance of this comprehensive RGB-D large-scale dataset.Fourth,we extract three features based on the comprehensive RGB-D dataset,which are Depth Motion Maps(DMMs),Depth Cuboid Similarity Feature(DCSF)and Curvature Space Scale(CSS).DMMs feature is obtained by accumulating the absolute difference between two consecutive projected maps across an entire depth video sequence.DCSF describes the similarity relationship between the local 3D depth cuboids around the spatio-temporal interest points(STIPs)extracted from depth videos.CSS shows the changeless feature of a human contour curve in a plane under different space scales.We test the three feature extraction methods on 5 sub-datasets and the comprehensive RGB-D large-scale dataset respectively,and apply Collaborative Representation Classifier(CRC)for human activity recognition.Through the comparison and analysis to the experimental results,we verify the applicability and effectiveness of the established comprehensive RGB-D large-scale dataset.Finally,we summarize the whole work in this paper and expect the future research work on human activity recognition.