Facial Expression Recognition Under Challenging Conditions

Facial expressions are presented as an important form for human transmission and expression of their emotions.Although computer vision and robot technology have achieved rapid development in recent years,how to recognize facial expressions accurately remains to be challengeable.Facial expression recognition(FER)aims to predict the emotion categories of given images.In recent years,FER has drawn much attention as its important role in human-computer interaction,which has been accelerated by the opensource of various datasets and algorithms.There are two main types of FER datasets,one is laboratory conditions(that is,controlled conditions),and the other is In-The-Wild conditions(uncontrolled conditions).Although the Convolutional Neural Networks(CNNs)performs well under the laboratory conditions,its performance is not satisfactory under In-The-Wild conditions,which is mainly resulted by the existence of challenging conditions(including a certain percentage occlusion,large pose and uncertainty labeled samples)in the In-The-Wild datasets.This paper proposes two effective methods based on deep CNNs under above two challenging conditions,respectively.These methods realize significantly improvement on the robustness of CNN-Based FER under challenging conditions.Specifically,the two methods in this article are introduced as follows:1.This paper addresses the real-world pose and occlusion robust FER problem in the following aspects.First,to further the research of FER occlusions and variant poses under real-world circumstance,we collect and annotate six in-the-wild FER datasets with pose and occlusion attributes for the community.Second,we propose a novel Region Attention Network(RAN).A given image is firstly cropped it into several regions,then the regions and original image are fed into neural network for extracting features.RAN contains two attention modules,namely self-attention module and relation-attention module.These two attention modules aggregate and embed varied number of region features produced by a backbone convolutional neural network into a compact fixedlength representation.Finally,in the fact that facial expressions are mainly defined by facial action units,we propose a region biased loss to encourage high attention weights for the most important regions.We validate our RAN and region biased loss on both our built test datasets and four existing datasets: FERPlus,AffectNet,RAF-DB,and SFEW.Extensive experiments show that our RAN and region biased loss largely improve the performance of FER with occlusion and variant pose.Our method also achieves state-of-the-art results on FERPlus,AffectNet,RAF-DB,and SFEW.2.This paper proposes a simple and effcient Self-Cure Network(SCN),which suppresses the uncertainties effciently and prevents deep networks from over-fitting uncertain facial images.Specifically,SCN suppresses the uncertainties in two different aspects: On the one hand,a self-attention mechanism over mini-batch to weight training samples and perform ranking regularization.On the other hand,a careful relabeling mechanism to modify the labels of these samples in lowest-ranked group.The suppression can make CNN learn robustness feature and extract more certainty samples from dataset.Experiments on synthetic FER datasets and our collected WebEmotion dataset validate the effectiveness of our method.Results on public benchmarks demonstrate that our SCN achieves current state-of-the-art methods on RAF-DB,AffectNet,and FERPlus.