Research On Technologies And System Of Emotion Recognition Based On Lightweight Skip-Layer Attention Convolution Neural Network

Human Computer Interaction(HCI)has been a key direction in the research field of AI(Artificial Intelligence).Emotion recognition,as an important part of HCI,has also attracted attention from researchers.Emotion recognition is a technique that can automatically recognize one's emotion by biological and non-biological signals,which can make the experience of HCI friendlier and more natural.With rapid development of robot industry and mobile Internet technology,emotion recognition researches based on mobile device are deepening continuously There are two main research interests in the domain of HCI research,lightweight model and data fusion.At present,most of the models proposed are complex and computation-costly,although with good performance.Meanwhile,emotion recognition technique based on single data source doesn't contain complete information,which makes the results unreliableGiven to these problems,this thesis conducts research on lightweight convolution neural network and emotion recognition based on facial expression,speech and data fusion.The main innovations of this thesis are as follows1.A lightweight convolutional neural network model LiSANet(Lightweight Skip-layer Attention Net)based on the Skip-layer Attention mechanism is proposed.We broaden the network to capture spatial features in different sizes,utilize depthwise separable convolution and skip-layer attention mechanism to achieve high accuracy and reduce computation cost.LiSANet achieves 90.30%classification accuracy on Cifar-10 data set,which surpasses MobileNet by 3.62%with approximate quantity of model parameter and complexity,and is lighter than VGG19 with a near classification performance2.A facial expression emotion recognition model named F-LiSANet,based on LiSANet,is proposed to improve the speed and accuracy of the model by using pre-training and fine-tuning methods.After trained and tested on Fer2013_plus facial expression data set,compared with VGG19,the classification accuracies of F-LiSANet on test set and validation set are 0.21%and 3.44%higher,respectively.Meanwhile,the quantity of model parameter is only one third of VGG19,and the model complexity is only one twentieth3.A speech emotion recognition model named S-LiSANet is proposed based on LiSANet Speech spectrogram transformed from speech signal in time domain is used as input to train the model on the speech emotion data set EMO-DB.Regarding insufficient data,we add random white noise on original signal to augment data set,which increases average classification accuracy by 20%4.Data fusion emotion recognition model and system based on F-LiSANet and S-LiSANet are studied.BP Neural Network is selected as the decision level fusion algorithm,and LiSA-BPNet is proposed in combination with F-LiSANet and S-LiSANet.After training on eNTERFACE audiovisual emotion data set,the test accuracy of LiSA-BPNet is 90.16%Compared with single data source classification models,the performance of LiSA-BPNet is much better and results are more reliable.According to the trained fusion network model,the emotion recognition system is built on the intelligent mobile robot.