An Improved CoDe4D Method And Its Application In Human Activity Recognition

Local Spatio-Temporal(LST)feature extraction algorithms are widely used in the field of video behavior recognition.LST feature extraction method is usually used to extract LST feature points from videos for representing the behavior trajectory of target characters in the video.The method selects the pixel points with significant changes in each video frame as the LST feature points.The LST feature points generally fall on the motion trajectory of the target person.Therefore,the LST feature points can represent the motion trajectory of the target person's activities better,which is very suitable for human activity recognition.In the LST feature extraction algorithm,the selection of feature points plays an important role for the final classification accuracy.With the advent of depth sensors,traditional LST feature extraction algorithms that only extracts RGB video features require adjustment to extract LST features from depth information.The principle that the depth sensor and the RGB sensor acquire data are different,and the difference in noise is generated,so that the two need to use different noise processing methods;how to gather the feature points on the motion track to avoid the selection of invalid feature points;and the classification accuracy rate is still Need to improve further.These are all issues that need to be addressed.In view of the above problems in the traditional LST feature extraction algorithm,this paper chooses the CoDe4D(Color-Depth Local Spatio-Temporal Features)feature extraction algorithm from the LST feature extraction algorithm to improve the CoDe4 D feature extraction algorithm.The color information and the depth information do not reduce noise separately.And the CoDe4 D feature extraction algorithm extractsfew LST invalid feature points.The improved CoDe4 D feature extraction algorithm uses different noise reduction methods for color information and depth information to reduce noise separately.In order to reduce the number of invalid feature points,the improved CoDe4 D algorithm modifies some parameters of the filter.In the classification of recognition,the support vector machine(SVM)with the generalized histogram cross kernel function is used as the classifier to further improve the classification accuracy of behavior recognition.Finally,the feature vector obtained by the improved CoDe4 D algorithm is added as a part of the input of the improved MiCT(Mixed 3D/2D Convolutional Tube)network,and the improved CoDe4 D algorithm and the improved MiCT network are applied to the dataset MSR Daily Activity 3D.The main work of this paper can be summarized as the following two aspects:(1)The improved CoDe4 D feature extraction algorithm.At the edge of the target human body,when the depth value acquired by the depth sensor may bounce back and forth between the depth value of the background and the depth value of the human body edge,the flip noise will be generated,andwill be generated due to the special material present in the scene and the rapid movement of the target character.For noise generation,a correction function is used to suppress this noise.For RGB sensors,histogram equalization is used to smooth out the noise and evenly distribute the grayscale data in the video.Modifying some parameters of the filter reduces the number of invalid feature points,and then uses the SVM with the generalized histogram cross kernel as the kernel function to improve the behavior classification accuracy.In order to verify the effectiveness of the improved CoDe4 D feature extraction method,this paper compares the feature point map extracted by the improved CoDe4 D with the feature point map extracted by several other feature extraction algorithms.And then uses DCSF(Depth Cuboid Similarity Feature)to describe the extracted feature points.The DCSF feature description method and the word bag method are used to construct the feature vector.The classification accuracy is obtained by SVM,and the classification accuracy is compared with several other feature extraction algorithms.The experiment is based on the MSR Daily Activity 3D dataset.The experimental results show that the improved CoDe4 D feature extraction algorithm has less feature points and its' trajectory is more obvious.After using DCSF feature description and word bag method is used to construct feature vectors,then SVM is applied for recognition,the classification accuracy is higher than other LST feature extraction algorithms.(2)The improved 3D/2D Joint Convolution Mixing Module(MiCT).Based on the original MiCT network,its' network structure is adjusted so that the improved MiCTnetwork can simultaneously analyze RGB information and depth information.The MiCT network framework is divided into 3D/2D concatenate connection modules and 3D/2D cross-domain residue connection modules.The 3D/2D concatenate connection module is used to extract spatiofeature,and the 3D/2D cross-domain residue connection module is used to share spatiofeature,to avoid gradient disappearance and to speed up the calculation of convolutional networks.The MiCT network is used for the joint processing of depth information and RGB information,and then the feature vector obtained by the improved CoDe4 D algorithm is added as an auxiliary feature inputting to the MiCT network.In order to verify the effectiveness of the improved MiCT network for depth and RGB.The classification accuracy of the improved MiCT network is compared with other LST classification algorithms for classification accuracy.The experiment is based on the MSR Daily Activity 3D dataset.The experimental results show that the MiCT network is constructed by using 1 to 3MiCT modules respectively.The MiCT network constructed by 3MiCT modules has the highest classification accuracy.And the accuracy of theimprovedMiCTnetwork is higher than the improved CoDe4 D method.