Micro-expression Recognition Based On Dataset Balance And Local Connected Bi-branch Network

Facial expression is an important window for emotional expression.Macroexpressions can be easily observed and convey human emotions,but spontaneous microexpressions can better reflect people’s true inner emotions.Therefore,micro-expression has important research and application value in many fields such as public safety,criminal investigation and clinical diagnosis.However,due to the short duration of microexpressions and weak muscle movement,there are a few public micro-expression datasets,and the sample imbalance of datasets leads to the long-tail phenomenon(that is,too many or too few categories causing a large gap in the number of categories).These problems seriously hinder the improvement of micro-expression recognition accuracy.Aiming at the problem of small number and unbalanced sample categories in microexpression dataset,this thesis proposed the assessment re-sampling(ARS)method to expand and balance the micro-expression dataset.The ARS method directly expands the micro-expression dataset through the original data,which effectively alleviates the problem of sample imbalance.Firstly,a self-training method of semi-supervised learning was used to train an evaluation model with high accuracy and high recall rate,and the apex frame of micro-expression was used as the training set for the evaluation model.Then,the well-trained evaluation model was used to evaluate the non-apex frame images in the original micro-expression dataset.Finally,the micro-expression images after evaluation were selected to directly expand the unbalanced dataset,so as to solve the problem of long-tail distribution caused by the lack of apex frames and the imbalance of dataset categories.Aiming at the low recognition rate problem due to the weak micro-expression muscle movement and the difficulty of feature extraction,this thesis proposed highfrequency filtered manual features and Local Connected Bi-branch network(LCB)to recognize micro-expressions.Firstly,the high-frequency component of the microexpression image was extracted as partial input of the LCB network,which makes the network pay more attention to the high-frequency information of the micro-expression data,that is,to enhance the attention to the facial detail contour.Secondly,in the LCB network,the global features of micro-expressions and local features with local connections were extracted through high-frequency filtered images and attention mechanism.Finally,micro-expression recognition was performed by fusing global features and local features with local connections.Extensive experiments were conducted on three micro-expression datasets,CASME,CASME Ⅱ and SAMM.The experimental results show that the ARS method was more effective in the balance of micro-expression datasets,which can adapt to different networks or datasets with robustness.Combined with the high-frequency features of micro-expressions,the LCB network proposed in this thesis achieved 90.23 % microexpression recognition accuracy in the SAMM dataset after ARS dataset balance,which proved the effectiveness of our proposed method.