Human Action Recognition Based On Tensor And Canonical Correlation Analysis

In recent years, the canonical correlation analysis (CCA) is widely used in the field ofbehavior recognition, such as discriminant canonical correlation analysis (DCC), theincremental discriminant canonical correlation analysis (IDCC), because it can reflect thedegree of correlation of the two image collections, as a similarity measure. However, in DCCand IDCC algorithms, the original structure of the sample is damaged, and the largedimension caused by vectorization can result into dimensional curse problem.. And the timecomplexity is very high. In this paper, CCA-based tensor discriminant analysis method isproposed to learn discriminant correlation between different tensor samples. Experiments onthe Weizmann behavior database compared with the-state-of-the-art CCA-based methodsachieved higher recognition rate and has strong robustness for broken images. The MDCC hasthree nolvelties as follows:1. MDCC employs tensor expression, to avoid the curse of dimensional curse problem;2. MDCC takes into account the typical correlation between the tensor samples;3. MDCC not only takes into account the correlation between the pair of tensor samples,also takes into account the correlation between the homogeneous and heterogeneous samples,for making the multi-linear discriminant analysis.Incremental learning has been used to classification areas, such as the incremental lineardiscriminant analysis (ILDA), incremental discriminant canonical correlation analysis (IDCC),as well as the increment tensor biased discriminant analysis (ITBDA) with to deal with theaction recognition and tracking issues. This article will be to introduce the idea of incrementalversion of MDCC method, by adding training samples incrementally instead of the wholeinitial training samples, thereby reducing the time complexity. Moreover, the more effectivediscriminant projection matrices can be obtained by updating iteratively. Experiments areperformed on the Weizmann behavior database, IMDCC recognition performance is higherthan the-state-of-the-art behavior recognition algorithms. IMDCC can obtain higherrecognition rate regardless of broken images, so it has better robustness. IMDCC has threenovelties as follows:1. IMDCC has lower computational complexity, therefore can be used as an effective method to deal with large-scale data;2. IMDCC converges by iterative learning;3. The original data is represented as a tensor, thereby avoiding the dimensional curseproblem caused by image vectorization.Higher order tensor analysis as an extended version of the vector or matrix analysis, hasbeen widely used in the field of biometrics recognition. There are two expressions for datasamples. The first way is independent a sample to a tensor expression, so the whole specialinformation of the sample can be retained. In the external environment, however, there arealways some interference factors exist in the field of behavior recognition, for example, thecamera angle or different people causing data differences. Accordingly, the second version ofexpression is that all samples is expressed as a tensor, under normal circumstances, the tensorof each mode represents a related external influence factor. This article focuses on the seconddata expression, mainly to consider the impact of multi-angle factors in behavior recognition.In addition, in order to improve the classification ability, this paper combines tensordiscriminant method. This paper proposed a new method called tensor-based multi-scalefeature discriminant analysis method (MSF-TDA) to build the behavioral model, to performthe behavior recognition. All the behavior samples are organized into a big tensor, where eachmode represents different semantic information, for example, different view angle, anddifferent behavior executor. Multi-scale feature extracted from each sample is used toillustrate the details of the different scales of movement, then is used for the tensor spacediscriminant analysis. This iterative learning method is confirmed by the theory and practiceof this paper. The proposed method combines the nearest neighbor classification (NNC)method, compared with the-state-of-the-art methods, to increase the recognition rate andreduces the time complexity, it also has better robustness according to changing view angles.MSF-TDAalgorithm has three nolvelties as follows:1. MSF-TDAdescribes the characteristics of the behavior in different scales;2. MSF-TDA considers and processes all the external factors which may affect therecognition rate;3. The time complexity of MSF-TDA can be greatly reduced, as the multi-scale featurewith lower dimension is used.In this paper, CCA-based tensor discriminant analysis method (MDCC/IMDCC) isproposed, which transforms the behavioral sequence into a third-order tensor, in order tocalculate the discriminant projection matrices by the correlation of tensor samples. Compared to TCCA, MDCC and IMDCC get a higher recognition rate because of the inherentdiscrimination. However, all of these methods are applied only one dimensionality reductionmethod in the tensor space, such as linear discriminant analysis (LDA) or the CCA, withouttaking into account the practical significance of each mold of a tensor, for example, the timecorrelation of sequence mode and the pixel characteristics of the image mode. We know abehavior sequence can be regarded as a third-order tensor, mdoe-1and the mode-2show theimage information of a behavior, while the mold-3shows the time information. Similarly, theface image may also be represented as a third-order tensor, mdoe-1and the mode-2show theface pixels, while the mold-3shows expression, illumination or color (RGB) factors. Thesame applies to the target image, mdoe-1and the mode-2show a target image, while themold-3represents the illumination, angle or a change in color. Considering that the imagesequence can reflect the time information of the behavior, the expression, light, colorinformation of the human face, and light, angle, color information of the target image, andtherefore can use these factors to improve the recognition rate of the sub-sequenceidentification method. In this paper, a tensor discriminant fusion algorithm (fusion tensordiscriminant analysis FTDA) is proposed, which combines the linear discriminant analysis(LDA) with the discriminant canonical correlation analysis (DCC) in tensor space. FTDAfirstperformed LDAin (N-1) modes which indicate the characteristic dimension, for extracting thefeature of the image, and then performs DCC in N mode which indicates the relevantdimensions for obtaining maximum correlation in the sequence. Projection combined withLDA and DCC is different from the traditional method of tensor analysis. There are threecommon databases, the KTH behavior database, C-PASCAL database, and AR face database,used for experiment. FTDA performs better than the-state-of-the-art tensor discriminantmethods. FTDAhas low time complexity than other methods, therefore suitable to be used forlarge data calculation. FTDA can deal well with the damaged images and occlusion images,thereby improving its robustness. In addition, FTDA conveges in an iterative procedure,which has been proved in experiment. The discriminant function value is unique regardless ofinitial transformation matrices, so the discriminant function is convex. The FTDA algorithmhas three nolvelties as follows:1. Considering the different characteristic of each mode of a tensor, FTDA performsdifferent projection method on different mode;2. FTDA takes into account the correlation of discriminant information of the image sequence;3. FTDAconverges by iterative learning.