Research Of Video Emotion Classification Based On Face Multimodality

This paper focuses on the research of video emotion classification algorithm based on face multimodality technology.Face multimodality refers to facial emotion recognition.In this paper,face detection and alignment,face recognition,and face emotion recognition are studied to obtain the realization of face-based multi-modality algorithm based on video segmentation and apply it to video emotion classification.At present,face multimodality and video classification technology have made some progress in research.Face recognition has achieved high recognition accuracy internationally and is widely used in security,surveillance and other fields.However,the current research of face multimodal technology is still in its infancy,and the baseline algorithm only obtains the accuracy of 49.3%.Although there are more successful improved algorithms,such as the HOLONET and C3 D algorithms mentioned later in this paper,the highest accuracy rate is still relatively low stage,and the algorithm takes a long time,and there is no classic application scenario yet.For video classification algorithms,there are many algorithms for classifying the spatial color features of videos,and they are all relatively successful.However,there is a lack of research on the classification of connotation such as video content,especially the behavior of people.Therefore,this paper will use the face multi-modality recognition technology to study the emotion level classification of video.In this paper,we study the mature algorithms of the face multi-modality and video classification.For face multi-modal recognition,this paper starts with face detection and alignment technology,then researches face recognition algorithms,and finally focuses on exploring mature algorithms for face multi-modal recognition.For video analyzing and classification,this paper studies the conventional video algorithms such as key frame localization algorithms,subsequently,deeply studies existing video classification algorithms,and deeply researches the current algorithm and the implementation process.Finally,it is found that the current algorithm still has the problems of low accuracy,poor real-time performance,and incomplete identification information.Based on the above research,this paper proposes a face multimodal algorithm based on the residual model unit,and combines the residual network unit and the traditional face recognition VGG model in order to increase the recognition accuracy.The algorithm is chosen to improve the real-time performance in the LSTM network stage by combining the key scenes of the video,and experiments are conducted in the AFEW public data set and the results are analyzed to verify its effectiveness.In the video classification algorithm,this paper proposes the video key scene selection algorithm based on the image entropy maximizing key frame extraction technology to obtain the multimodality recognition input sequence.Afterwards,according to the video emotion classification task,this paper researches the results of facial expression recognition and the characteristics of video emotion classification,and then proposes a video emotion classification framework,and conducts experiments and results analysis.This paper respectively researches face recognition and video classification,and then proposes improved algorithms.The two are combined to realize the recognition of video content emotion information and apply it to video classification.The video classification algorithm finally achieved an average accuracy of 75%.Finally,a video classification system based on this algorithm was implemented.