Research On Facial Expression Recognition Based On Model Ensemble

Many existing methods based on deep learning have good recognition performance in the expression dataset under constrained environment.However,Facial Expression Recognition(FER)in real environment still faces great challenges.The challenge mainly comes from three aspects:(1)collection difference,(2)expression difference,and(3)difference in recognition difficulty.Most of existing works focus on solving the first two problems,such as applying appropriate pre-processing methods(face registration,histogram equalization,etc.)to alleviate the problem of collection difference,and adopting network ensemble strategies to solve the problem of expression difference.To solve the problem of low efficiency in existing network ensemble strategy,this paper first proposes an automatic network selection method for finding the optimal network set with complementary performance from multiple candidate networks.In order to solve the problem of difficulty difference in recognition,this paper uses validation results of optimal network set to determine the complicate expression categories in data set.Next,to improve the recognition accuracy of complicate expression categories,this paper proposes an iterative training method for dynamically collecting complicate expression samples.Experimental results show that iterative training method not only alleviates the problem of data imbalance,but also significantly improves recognition accuracy of complicate expression categories.Therefore,the average recognition performance is improved.Considering that expression recognition is a fine-grained classification problem,this paper believes that attention mechanism can help to further improve recognition performance.Therefore,based on the updated training data set in iterative training stage,this paper further explores the impact of attention mechanism on expression recognition performance.This paper designs a Facial Expression-oriented Attention Module(FEAM),which includes a channel attention unit without any trainable parameters and a spatial attention unit with lightweight parameters.The experimental results show that compared with the latest attention module,FEAM is more suitable for facial expression recognition in real world.Finally,this paper also analyzes the role of proposed FEAM in expression recognition through visual experiments.