Facial Expression Recognition Based On Landmarks And Transfer Learning

With the arrival of the era of artificial intelligence,human-computer interaction will become an indispensable part of future life.How to make computers recognize human emotions has become an important research topic in human-computer interaction.Expressions are external representation of human emotions and play an important role in people's daily life.How to equip machines with the ability to understand emotions by recognizing facial expressions will become an urgent topic in the future.In recent years,many researchers have studied expression recognition algorithms and a large number of methods have appeared.However,due to individual differences in faces,such as people's cultural background and skin color,expression recognition algorithms cannot achieve ideal results;secondly,the accuracy of expression recognition will be weakened to a certain extent due to the different shooting angles of face pictures;Lastly,some expression databases also have problems such as the small number of pictures,difficult data collection,and large labeling costs,which disturb expression recognition process.Therefore,how to improve the accuracy and generalization ability of expression recognition models is still of great significance to the expression recognition in practical applications.This paper carried out research on facial expression recognition technology from the following two aspects.First,this article explores the facial recognition technology approach based on facial landmarks.This choice is based on the following considerations: First,using facial landmarks can exclude the influence of skin color,lighting and so on,and it is possible to obtain a more robust recognition effect;Secondly,based on the current mature facial landmarks extraction technology,it is possible to design a simpler neural network structure to improve the recognition speed;Finally,this article attempts to explore such an interesting problem,namely ”Can the methods based on geometric changes of facial landmarks reach the level of human-like facial expression recognition?” Second,facial features are crucial in facial expression recognition,which often require in-depth design and extensive adjustments.This article explores the ability of the existing pre-trained network structure for facial expression recognition through transfer learning.The main contents of this article are summarized as follows:(1)A facial expression recognition algorithm based on 68 commonly used facial landmarks' locations is proposed.Due to the relatively small amount of landmark data,after extensively trying a variety of different network structures,it is found that an artificial neural network with only 2 hidden layers can reach a fine recognition rate.The experimental results on the CK+ and KDEF databases show that the algorithm can achieve a recognition accuracy close to the representative algorithms with faster speed.The experimental results also show that the algorithm has a good generalization ability to new expression database and is robust to illumination.(2)Since pre-trained network has a good feature extraction ability,this article proposes a multi-feature expression recognition algorithm based on the pre-trained model VGG19.In this work,a variety of fusion strategies have been tried extensively,the features of the local area of face are merged with the global features of face.Experimental results on the CK+ and KDEF databases show that the fused features have a better description of facial expressions.