Multi-view Human Action Recognition Based On Tensor And Multi-scale Features

Intelligent Visual Surveillance System could distinguish and identify the objects from the numerous video images, and could analysis and extract useful information from the video images automatically, thereby improving the performance of the intelligent video surveillance system. With the rapid advancement of the network technology and information technology, the intelligent monitoring technology in the field of pattern recognition has become one of the most hot academic topic in recent years with more and more attention.Intelligent surveillance and analysis video system can effectively improve the accuracy of alarm, leading an unprecedented revolution in the way of surveillance with prophetic capabilities of the early-warning, and can effectively expand the use of video resources. The core technologies of the intelligent visual surveillance include these factors: moving target detection, target tracking, target classification, recognition and understanding of target behaviors and so on. Intelligent Video Analysis System is used in a wide range of applications, which provides rapid development and implementation of the surveillance performance in many field, the most important application is in the security-related areas at present. Especially after the "911" terrorist attacks, the Madrid bombings and the London bombings, the demand for such applications with many functions is growing up quickly. The main purpose of Intelligent Visual Surveillance System is to assist the security sector to improve outdoor, perimeter, large regional public security environments. In intelligent visual surveillance system, the study of human action recognition has been researched to the most extent in recent years, which plays an important role in the performance improvement of intelligent analysis technology.The main task of human action of recognition is to identify and process each frame of the sequence video images which the camera captured, divided the target area accurately and clearly, and identify the person's behavior by calculating the speed, point coordinates and other parameters of extracted features.In this paper, a number of existing human action recognition algorithms, image processing and tensor space theories have been learned and researched, by citing a large number of domestic and foreign literature, this paper has researched and improved video-based human action recognition algorithms, and a large number of experiments have been done which performed good.First of all, we detect the human targets from the video. For the interference of the external environment and the camera's reason, the video images are usually collected with noise which severely affect the result of the target detection. So we first use Gaussian filter to denoise the gray images, then using background subtraction to extract foreground objects. We process the moving target by morphology detection, by denoising the binary images by the threshold area elimination algorithm, using morphological dilation algorithm to make the subtle breaks reunion, using region growing method to fill the empty, thereby get a better binary image results.Second, the human image skeleton has been modeling, which is used as input features of human action recognition. The human skeleton has been extracted by morphological method, then the endpoints of the skeleton have been found, and traverse the chain code from endpoints to determine the skeleton bifurcation points, and then calculate the curvature of up-limb to find the corner point, finally by connecting the corresponding points, we can build the human skeleton model. Compared with the previous human models, this model could accurately describe human action, and position the critical points of human model precisely with less time-consuming.Thirdly, the extraction of multi-scale feature has been used. In general, human action recognition use the binary silhouettes (silloutte) as input features, but the silloutte connot express enough information of human action, including the speed of macroscopic features and microscopic characteristics of the point coordinates. The behavior of people contains many details of different scales at the same time, different scales reflect the different characteristics of human behavior. Large-scale features reflect the coarse characteristics of the human body, such as speed. Medium-scale features reflect the body's posture characteristics, such as bending, dance, etc. Small-scale features reflect the very exact characteristics of the human body, such as the location of the head and limbs. The three levels of multi-scale features have been compared with a single binary silhouette features, and the advantages of multi-scale features is obviously.Fourth, a series of images which constitute the feature space of tensor has been researched. In general, human face and gesture recognition methods are based on single binary silloutte to determine the feature space, but human action is a series of action sequences, so this paper determine the feature sub-space by using a continuous image sequence, and extract multi-scale features from the sequence to do the human action recognition.Fifth, a Serial Image Tensor has been defined on the base of the serial images, which has been used for the feature space of the tensor, and we could get more information of human action by this tensor model.Sixth, we make the tensor decomposition in a multi-view and different people respectively. As there may be block from different views, so the recognition rate may be various, so as to the different people, and we research the tensor decomposition from the two point of view. The results show that under the conditions in the multi-view, multi-scale features based tensor analysis has better recognition rate than the silhouette based tensor analysis, which demonstrate that the multi-scale features we proposed has better optimality. While, in the condition of different people, both the recognition rate is not much different in action recognition, which also shows that action recognition is little effected by different people, in another word, the occurrence of risk behaviors have much in common, and there is low probability of false consciousness.Finally, the clustering algorithms has been compared with the tensor analysis method from a large number of experiments, and the recognition rate, memory requirements, and other aspects such as the need of real-time have been compared and summarized. The results show that the multi-scale features based tensor analysis methods has higher recognition rate, smaller memory requiring and fewer time-consuming under the multi-view and different people conditions, which demonstrates that multi-feature and tensor based human recognition method this paper proposed has a good recognition effect.In summary, tensor analysis and multi-scale feature based the multi-view human action recognition algorithm is proposed in this paper. At the same time, we also did some work as following: first, we improved the moving objects detection algorithm, and got good binary image. Second, we proposed a human skeleton modeling method. Thirdly, the multi-scale feature has been extracted. Forth, a serial images frame has been proposed. Fifth, a serial images tensor was defined. Sixth, we did the human action recognition under multi-view and different people respectively. Finally, by comparing with the clustering recognition method, we obtained the optimization of the method this paper proposed. As the rapid advancement of the science and technology, the technology of human action recognition will be improved gradually, and I hope the work this paper researched could make a modest contribution.