Research On Expression Recognition Based On MLBP-TOP And Optical Flow Hybrid Features

In recent years, the reasons for renewed interest in facial expression recognition are multiple, but mainly due to people have more interest about human computer interaction (HCI). Facial expression recognition is to analyze and detect the special expression state from given expression images or video frames and then to ascertain the subject's specific inborn emotion, achieving smarter and more natural interaction between human beings and computers. The study of facial expression recognition has found important applied values.In this work, we firstly discuss the background and then analyze the main existing expression recognition algorithms. After analyzing the methods currently used by others,several improved algorithms and methods for these tasks are developed. The performances of our methods are illustrated by experiment results. The main work is described as bellows:(1) Facial expression feature extraction method based on improved LBP-TOP is proposed. In this method, basic LBP-TOP feature is improved. According to differences of different planes in LBP-TOP, features of three planes are classified into two types, which differential local binary patterns and centralized binary patterns are seperately used, to extract expression time domain information and gradient information, as mixed local binary patterns features from three orthogonal panels.(2)Facial expression feature extraction method based on auto-marked points optical flow method is proposed. In this method,it detects left and right mouth corners of facial image by Harris corner detection algorithm, and then use Robert arithmetic operator and level projection to get upper lip and lower lip points, least squares curve fitting method is used to fit four new points.Take these points as orginial Lucas-Kanade Optical Flow tracking points, it computes optical flow field of mouth region, and tracks all the points, then composes feature vector by horizontal and vertical displacement.(3) Expression recognition algorithm based on hybrid features and fusing discrete HMMs is presented. Because different expression areas have different contribution to each expression. This method extracts texture variety feature using MLBP-TOP for the eye area, and extracts shape variety feature using optical flow based on auto-marking feature points for the mouth area. At the same time, we fix on the weight contributing to each expression for each expression area by using contribution analysis algorithm when training templates. In the process of recognition, we fuse the probability of each expression area with the weight of each expression area obtained by contribution analysis algorithm, and use the maximal probability as recognition result.(4) A prototype system of facial expression recognition based on image sequence is designed and implemented by using the object-oriented methods. It can be used to prove the effectiveness of above algorithms.