Person-independent Facial Expression Analysis In Video

With the rapid development of artificial intelligence,machine learning and pattern recognition,facial expression analysis has become an important research direction in the field of machine vision and has been widely used in the fields of human-computer interaction,distance education,medical treatment,daily life and security.Facial expression is a critical way to share emotional feelings,Everyone's expression of different expressions is different,and the same person has different degree of expression of the same expression.Therefore,it is of great significance to study the person-dependent facial expression analysis.At present,most of the video expression recognition is based on the existing database,and the video expression in the database is only a sequence of consecutive pictures.Expression recognition algorithm for arbitrary length of video segment is still relatively uncommon.The research content of this thesis includes not only the video expression recognition in the database,but also the recognition and expression analysis of any length video expression segment.The main contributions of this thesis are as follows:In the facial expression video preprocessing stage,this paper proposes a method of separating facial expression segment from the whole video by calculating the variance of the distance ratio between the key facial feature points.The face image to be recognized is selected from the extracted facial expression segment,and the angle of the face is adjusted so that the central point of the two eyes is in the same horizontal direction.Based on the geometric features of the face and the distance between the central points of two eyes,the face,region of eye and brow and mouth are divided from the full face,and then the scale is normalized.Finally,in order to improve the effectiveness of feature extraction,median filtering and histogram normalization are applied to the segmented image.In the feature extraction stage,a feature extraction method of completed local binary from three orthogonal planes(CLBP_TOP)is proposed.Before using this method to extract features,firstly,the three parts of the preprocessing result are divided into blocks,and then the CLBP_TOP features are extracted on each small block.Finally,the joint statistical histogram features are obtained as the feature vectors of the final facial expression video.In expression recognition stage,a nearest neighbor classification algorithm based on improved dynamic time warping(DTW)algorithm is proposed.Based on the original nearest neighbor classifier,the improved DTW algorithm is applied to measure the similarity between two vectors,and the time consistency of facial expression in video is realized.Compared with other methods in CK database,the proposed CLBP_TOP features are more discriminant,and the recognition rate of each facial expression segment is generally higher.The method is also suitable for the recognition of video segment with arbitrary length,and presents the related indicators of expression evaluation: the duration of expression and expression intensity scores,and the detailed description and analysis of the expression are realized.This thesis not only analyzes the single facial expression video,but also studies the situation that there are several facial expression transitions in the video,which can analyze which expressions they contained,the classification of each,the duration and intensity of the expression.