Speech Lie Detection Research Based On Signal Decomposition Reconstruction And Deep Learning

Some lying behavior can be harmful to human society,how to detect lie is always an important question,especially for fields such as law,military and forensic science.Speech contains a great deal of information that reflects the psychophysiological changes in a person,so using speech for lie detection has scientific basis.What’s more,speech lie detection is lowcost,easy to operate,less likely to cause rejection and fear in the person being tested,and the detection result is more objective.With the advancement of technology,researchers have used machine learning algorithms for speech lie detection research,and the rise of deep learning in recent years has led to a new level of lie detection research.Therefore,in this thesis,lie detection techniques based on speech signals are studied,and the specific work is as follows:Firstly,the theoretical basis of lie detection and speech processing is introduced.For the lack of corpus in speech lie detection field,the thesis draws on foreign corpus collection methods,collects resources from Chinese video websites,and got a Chinese lie corpus which is named CWG-LD(Chinese Wolf Game Lie Dataset),enriching the Chinese lie corpus.Secondly,since speech lie detection studies focus less on the speech itself,the CC-SDR(Correlation Coefficient-based Signal Decomposition and Reconstruction)model is proposed using signal decomposition algorithms in signal processing techniques.The signal is decomposed by a signal decomposition algorithm,and a threshold value is calculated based on the correlation coefficient between the sub-signal and the original signal,so as to filter the useful components of the speech to reconstruct the speech and improve the performance of speech lie detection.This thesis verifies the effect of CC-SDR with four different signal decomposition algorithms,EMD,LMD,VMD and EWT.The results show that except for VMD,all three algorithms,EMD,LMD and EWT,can make CC-SDR work for speech lie detection.Among them EMD performs the best,on average,the accuracy and F1 scores in Real-life Trail are improved by 1.08%and 1.30%respectively,and in CWG-LD they are improved by 1.70%and 1.92%.Finally,since the temporal characteristics of speech are more neglected in speech lie detection studies,this thesis proposes a CAMT-TCN-LSTM(Channel attention and Muti-taskbased TCN-LSTM)network based on channel attention and multi-task learning for speech lie detection.Both TCN and LSTM can handle temporal data,attention is used to focus on the more important dimensions of the features,while multi-task learning is used to obtain better performance.The experiments show the best results with a 4-layer TCN,a 1-layer bidirectional LSTM and the use of the ECANet attention module,achieving 89.8%and 68.6%accuracy and 89.3%and 68.0%F1 scores under the two datasets,respectively.Ablation experiments verify the contribution of each part.The organic combination of TCN and LSTM is the core,which complements each other’s shortcomings,promotes their advantages and greatly enhances the results.The attention mechanism and multi-task learning strategy playing a role in optimizing the performance of the TCN-LSTM model.Finally,the results are compared with others,and the superiority of proposed method is analyzed.