Research Of Facial Expression Recognition Based On 3D Points And 2D Image Information Collaboration

Facial expressions play an important role in communication between people, and the facial expression have a more intuitive, accurate advantage in expressing human emotions relative to text, voice and other media. It can enhance the naturalness of human-computer interaction by using the interactive mode for emotion. Facial expression recognition is currently using mainly two-dimensional image collected by the device of cameras as inputs, and to achieve the cognition of expression by training the expression recognition model based two-dimensional image. However, studies have shown that three-dimensional expression information including depth information also plays an important role in the understanding of expression. So the expression recognition based on three-dimensional information is becoming a major method. To explore the effect of the method of fusion that two-dimensional images combine three-dimensional coordinates information, this paper studied the effect of facial expression recognition based on feature fusion. However, in the actual application environment, there are problems that two-dimensional expression image is easy to acquire, and the acquisition of three-dimensional expression information need specific collection equipment, so three-dimensional data collection is relatively difficult. This paper studied and modified the multi-layer extreme learning machine. To build the expression recognition model based on 3D and 2D information collaboration by knowledge transfer and transfer the model in 3D expression data to the recognition in 2D expression data, this paper uses the three-dimensional expression data as basis and uses the two-dimensional expression data to modified the local parameter of recognition model. The main research work and innovations are as follows:1. This paper builds a 3D expression dataset based on coordinates of facial marked points and a corresponding 2D image dataset. The Opti Track facial motion capture device can accurately capture three-dimensional points coordinate. Comparing with two-dimensional image data, the three-dimensional coordinate can express the facial movement information more accurately. This paper builds a 3D expression datasets based on facial points coordinate. The dataset contains three-dimensional coordinate data of 27 marker points in specific location. Meanwhile, this paper captures the 2D expression dataset for the same object. The two types of dataset contain 6 basic expressions of 7 person, and each expression contain 5 group data.2. This paper studies the expression recognition based on the fusion method of 3D expression and 2D expression data. This paper has proved that feature fusion can promote the expression recognition effect comparing with single type of feature by studying the expression recognition of the fusion of 3D geometry vector and 2D texture feature. To extract expression feature which has good characterization capabilities, this paper uses multi-layer extreme learning machine to extract feature of the fusion data which comprised of 3D geometry vector and 2D expression data, and uses the feature extracted to recognize the expression. Experimental results show that the multi-layer ELM-AE in multi-layer extreme learning machine has a strong capacity of feature extraction, and it can improve the result of expression recognition based on method of feature fusion.3. This paper proposes a transfer learning model based on modified multi-layer extreme learning machine, and studies the expression recognition based on transfer learning. The transfer learning model is comprised of multi-layer ELM-AE in multi-layer extreme learning machine and OS-ELM classifier. This paper transfer the expression recognition model based on three-dimensional expression data to the recognition of 2D expression data by using the transfer learning model and it improves the result of expression recognition in 2D expression data.