Research On Micro-Expression Recognition Under The Mechanism Of Deep Information Interaction

With the needs of all parties in society,the advancement of information technology and the development of computer vision,the field of micro-expression recognition has received more and more attention,and researchers have conducted more and more in-depth research on it.When we observe the state of a person,the first thing that comes to mind is to observe the facial expression,and at the same temporal we will also communicate based on the other's expression.Often in business negotiations,criminal interrogations,psychotherapy and other occasions,people intentionally or unintentionally hide their true emotions in order to achieve their goals.Micro-expression is the human muscle language.In such cases,the result of using micro-expression recognition As a basis for judgment,the success rate will be greatly improved.Micro-expressions are different from ordinary macro-expressions,and they have the following salient features:(1)Short duration(1/25-1/5 seconds);(2)Small muscle movement range.In addition,due to the difficulty of capturing micro-expression and the complicated collection work(specialized by professional psychology researchers),the existing publicly available data sets are scarce,which has caused huge obstacles to the research in the field of micro-expression and restricted.The development of micro expression recognition technology.In response to the above problems,this article conducts in-depth research on micro-expression recognition based on deep learning methods.The main research is based on the information interaction between different modalities and different domains between networks to improve the accuracy of micro-expression recognition at this stage.By fusing different information to obtain richer knowledge from the spatial and temporal domains to optimize parameters,the network obtains more knowledge under limited data samples,and at the same temporal alleviates the over-fitting situation caused by single-label supervised training,and finally improves the network Predictive power.The main research results are as follows:(1)In order to enrich the network knowledge,a micro expression recognition algorithm based on auxiliary network information interaction is proposed.An improved deep mutual learning strategy is used to guide the interaction training between different modes of image sequence to improve the recognition rate of the network.The main network is established based on gray image sequence,and the auxiliary network is established based on optical flow.In the training phase,the supervised learning loss and the mimicry loss in the mutual learning loss are designed to optimize the training process,so that each mode can learn to correctly predict the real identification of the training sample,and can match the prediction of other modes.In the test phase,due to the mutual learning mechanism to enhance the discrimination ability of gray branch,the optical flow branch can be clipped to improve the recognition speed on the premise of ensuring the accuracy.Finally,the accuracy of this method is 75.76%on SMIC database and 60.52% on the combined database of CASME,CASME ? and SMIC.(2)Aiming at the fact that it is difficult for a single network to learn the rich temporal domain information of the micro-expression sequence,a micro-expression recognition method for the temporal-domain information interaction of the dual-stream network is proposed,and the Dual Scale Temporal Interactive Convolution Neural Network(DSTICNN)is constructed.,The network processes the micro-expression sequence,and then realizes the automatic recognition of micro-expression.The algorithm improves the final recognition rate by improving the deep mutual learning strategy to guide the network to learn different temporal domain information of the same image sequence.The algorithm builds DSTICNN32 and DSTICNN64 based on different temporal scales,and improves the loss function of deep mutual learning in the training phase.At the same time,mean square error loss is added to the feature map of the two-stream network close to the decision-making layer,and finally cross-entropy loss,JS divergence loss and mean square error loss are used to jointly supervise training,so that the two-stream network learns and strengthens each other and improves their respective prediction samples.Ability.This experiment uses the Leave-One-Subject-Out verification method,and finally the best accuracy rate obtained in the SMIC database is85.93%,and the best result obtained in the CASME ? database is 83.65%.(3)In order to enrich the network's spatio-temporal feature prediction ability,a micro-expression recognition based on pyramid-enhanced spatio-temporal information interaction is proposed.The algorithm uses the feature pyramid technology to construct a network of temporal-enhanced information and spatial information respectively,so as to obtain rich temporal and spatial features.In the temporal-enhanced network constructed by expanding the temporal dimension of the 2D-Resnet34 network,through the setting of the convolution and pooling layers,the spatial dimensions of the feature maps of the different 4stages in the control network are kept consistent,and only the temporal dimension is changed.Then,the final feature image of each stage is normalized in temporal dimension,and connected from top to bottom to form a feature pyramid,and finally micro-expression prediction is performed.In the same way,the temporal dimension of the feature image at each stage of the construction of a spatial-enhanced network control network does not change the temporal dimension but only the spatial dimension.When the network trains the micro-expression sequence,it can learn a wealth of temporal or spatial information and make predictions for the micro-expression sequence.At the same time,the method also introduces the strategy of mutual learning.During the training process,the prediction results of the spatially enhanced network and the temporal-enhanced network are exchanged,so that the dual-stream network is not only enhanced by the feature pyramid between temporal and spatial.Information,and through network interaction also increases the knowledge of spatial(temporal).This experiment adopts the Leave-One-Subject-Out verification method,and finally the best accuracy rate is 87.14% in the SMIC database,and the best result is87.31% in the CASME ? database.