Micro-expression Recognition Based On Twos-tream Fusion Network

Micro-expression is an expression that is generated when people try to hide or suppress a certain emotion with a very short duration,a very weak intensity and can not controlled by themselves.Micro-expression often hides people's real feelings,so automatic microexpression recognition is widely used in clinical medicine,business negotiation,judicial investigation,psychotherapy and lie recognition.At present,there are some problems in the traditional micro-expression feature extraction methods,such as lack of spatiotemporal information and poor robustness to noise,and it is difficult to extract feature information quickly from deep network because of the small number of database samples.In view of the above problems,this paper uses the twostream fusion network to combine the deep learning with the traditional methods.The following is the main content:(1)A local binary model based on Adjacent Double Crossover Local Binary Pattern from Three Orthogonal Planes(ADCP-TOP)is proposed.ADCP-TOP has a stronger ability to capture detailed information by integrating pixel information in the neighborhood,and codes the sampling points of odd and even positions separately,which increases the potential direction information and improves the robustness at the same time.During image segmentation,the coarse-grained regions of interest(CROI)are divided according to the facial action units to remove redundant information,and the fine-grained regions of interest(FROI)are obtained by dividing the CROI again to enhance the ability of extract detailed information.(2)A Pseudo 3D convolution of Attention Mechanism network(P3D-AM)is proposed.P3D-AM network simulating 3D convolution by 2D convolution of XY,XT and YT orthogonal planes,and convolutes images in parallel.In this way,the network can concentrate on extracting the motion change information in the corresponding direction,while improve the robustness of the system,and ensure the expression ability.The parameters of the element are consistent with 3D convolution,which guarantees the expression ability.In addition,the attention module is added to each layer of convolution,which makes the network pay more attention to the small change area,and effectively improves the extraction ability of microexpression details.(3)A new two-stream fusion network is proposed,which is divided into two flow paths to extract micro-expression features.Firstly,the traditional coding features are extracted by the proposed ADCP-TOP algorithm,then the ADCP-TOP features are input into the P3 DAM depth network for feature fusion,and finally the micro-expression recognition is carried out by the fused ADCP-P3 D fusion features.Finally,experiments are carried out on SMIC database,CASME database and CASME2 database.The proposed micro-expression recognition method based on two-stream fusion network pays more attention to motion change information and can extract micro features more accurately.Compared with other micro-expression recognition methods,this method has higher recognition rate.