Research On Content-based Algorithm Of Video Affective Extraction

With the advance of technology of computer, telecommunication, wide-band network and compression of video and audio, digital video is widely coming into the life of people. the images/video in multimedia Human-Computer Interface (HCI) not only carry a large amount of image information, but also carry a lot of emotional information. It has become an issue in the spotlight to research on how to apply the technology of affective computing in the image/video retrieval system in order to make computers recognize the emotional information in images/video and make use of the emotional information to conduct video retrieval.This paper which is aiming at the video of feature films mainly studies the algorithm of affective extraction based on its styles and characteristics. We analyze both the fields of audio and video and extract the emotion features in some clips. We also categorized the video clips according to the emotion features. The main contents of this paper are as following:We propose the emotion classification method based on SVM (Support Vector Machine) with the features of scalefactors. First of all we classify the audio information into four groups which are speech, music, mixed audio and mute by extracting the features from MPEG encoded bit steam. We then classify the emotion of audio into happiness, sadness, anger, disgust, surprise and normal based on speech,music and mixed of audio of each type. Experiments have shown that this algorithm is reliable and effective.For extracting the emotion information from video image, we must first segment the shot and extract the key frame from the shots. So we present an algorithm for shot boundary detection based on SVM (support vector machine) with modified kernel function in compressed domain. It uses the features, such as the type of macroblock, the difference between DC coefficients of two co-located blocks in successive frames and the type of frame, to segment a video into the shots by classifying the frames into three classes, namely, the frames of cut change, gradual change and non-change. We modify the kernel function of SVM based on its nature, and some experiments have been done to compare with other kernel functions commonly used. The experimental results show that the classifier with the kernel function of RBF+Gaussian RBF has the best classification performance and achieved higher Recall and Precision of shot detection. Also it is about 14% higher than the best result of 2001 TREC evaluation in F1 comparison when cut and gradual changes are both considered. This paper also proposes a method of extracting the key frames from shots according to the characteristics of feature films.A support vector machine (SVM) is used to make emotion recognition and classification of key frames in feature films by adopting the features of color histogram and MPEG-7 edge histogram after segmenting the shots and extracting the key frames. We construct the SVM with RBF and make exercises and tests. Then we have obtained the rough emotion semantics of the key frames after the classification.