Research On Feature Extraction And Fusion Algorithm In Deception Detection

Lying is very common in daily life,and it is a necessary means of interpersonal communication.Therefore,human beings have been interested in the research of polygraph.Deception detection can judge whether the person being tested lies by analyzing speech signals.Compared with traditional methods,speech deception detection is a non-contact detection,which is not limited by time and space,and has a wide application prospect.Although some research progress has been made in the field of speech deception detection,there are still many problems to be studied.The research on speech feature extraction and fusion is very important in speech deception detection.The representation ability of speech features directly affects the accuracy of deception detection.Therefore,this paper studies the speech feature extraction and fusion algorithm in speech deception detection,and the main work is carried out from the following three aspects:(1)When using long-term and short-term memory network to classify and recognize speech lies,the commonly used LSTM network label learning method is the average pooling method in time dimension.However,if there are too many blank frames and non-lie feature frames in speech,these frames will affect the final output of LSTM network,which will lead to the excessive redundant information in the final features,thus reducing the representation performance of lie features.In response to this problem,this paper combines attention mechanism with LSTM network to detect speech deception.The deep frame feature sequence can be extracted from the original frame feature through the LSTM network,which is input to the attention pooling layer later.To obtain the weight of each frame,sum the weight and the weight of each frame to obtain the final output value,and finally use the softmax classifier to fine-tune the network parameters.Experiments on CSC and killer corpus show that attention pooling learning can improve the performance of LSTM network in feature extraction,and thus increase the accuracy of lie recognition.(2)In order to improve the feature extraction and characterization ability of bidirectional LSTM network,a multi input and multi fusion strategy is proposed.At the input end of the DBL-MM model,two kinds of speech features are input into two Bi-LSTM modules at the same time,and the lie information represented by different features in speech is learned by parallel operation.At the output end of the model,in order to fully learn the emotional information contained in each frame,the proposed model adopts two fusion strategies,namely average pooling and attention pooling,and integrates the two types of output features into two types of advanced fusion features.Then,the two kinds of advanced fusion features are added and merged again,and then they are processed by batch normalization.Finally,they are input into softmax classifier to classify lies.The results show that the multi input and multi fusion strategies have better rate of lie recognition than traditional learning strategies.(3)In order to improve the feature fusion effect of hybrid network model,a multi feature fusion algorithm for attention is proposed.Firstly,considering the limited influence of label data on speech feature extraction,semi supervised DAE network and LSTM network are used to form a hybrid network model(HN-AMFF).The DAE module of the proposed model uses a large number of unsupervised data to complete the pre-training.The output part of the LSTM module adds an attention mechanism based on the time dimension,which speeds up the convergence speed of the network.Secondly,in order to make the feature sets of different feature spaces concentrated in the optimal feature space,a feature fusion algorithm based on attention mechanism is used to fuse the advanced features extracted by different modules,so as to improve the fusion effect of different types of features.The experimental results show that the proposed model has higher recognition rate than traditional feature fusion algorithm and deception detection model under the same number of labeled data.