The Research On Spatial And Temporal Fusion Feature Extraction Algorithm Based On Video

In recent years, with the improvement of computer hardware technology and the rapid development of the Internet, affective computing began to concern researchers, practical and theoretical research in this field has become a reality. Currently, the affective computing technology has been widely used in various areas of society, to better serve the people’s daily life and created enormous social and commercial value.The emotional feature of Affective Computing Research includes facial expression, voice, gesture, physiological signals, and the facial expression is one of the emotional feature which is the most intuitive and the easiest to get. In facial expression recognition research process, early research mainly for static images and extracting static expression feature. With the development of the study, researchers began to focus on video-based facial expression recognition research, Video-based facial expression dynamics feature contains not only the static spatial information but also the dynamic information changing with time, which more realistically reflects the changing facial expression and more conducive to accurately identify facial expression.Therefore, the focus of this paper is the facial expression recognition based on the video. In this paper, facial expression recognition system includes three modules: Feature point detection and location, Temporal and spatial integration of feature extraction, Classifier design and the analysis results of simulation. This paper select the Piotr Dollar emotional database and East China University emotional database as a source of research data, whose main purpose is to verify the feasibility and effectiveness of the paper’s design scheme and East China University emotional database,and then suggest improvements program for improving the database.The main work is as follows:(1) Facial feature point detection and location section, this paper use Piotr Dollar spatial and temporal feature point detection and localization algorithm to detect and locate the face spatial and temporal feature points of the expression video. The algorithm is specially designed for the features of behavior recognition, sensitive to changes in motion and can better get a slight change in positioning of facial movement.(2) The facial expression feature extraction, this paper proposes a video-based emotional spatial-temporal fusion feature extraction algorithm, the algorithm is an improvement of an emotional spatial-temporal fusion feature extraction algorithm proposed by Jing jie Yan and Ming han Xin. The algorithms used Piotr Dollar description operator and CBP_TOP description operator to extract the local feature of cuboids, then subjected to [-1,1] normalized and use principal component analysis (PCA) method to optimization reduce the dimension, finally, fuse directly these two local features. The algorithm reduces the computational complexity and improve the computing speed on the basis of ensuring the accuracy of the expression feature.(3) Classifier design and simulation experiments and results analysis. This paper uses K-Nearest Neighbor (KNN) algorithm as emotion recognition simulation.The simulation was done on the Piotr Dollar emotional Database and East China University emotional database, to test the feasibility and effectiveness of this paper’s design scheme and East China University emotional database. The results show that:this paper’s design scheme greatly improve the speed of recognition on the base of ensuring the recognition rate, and the East China University emotional database can be used effectively, some details need to be further improved and perfected.