Research Of Multimedia Violence Fragment Detection Based On Audio-visual Channel Fusion

With the development of the film industry and the popularity of the Internet,a large amount of multimedia information, for example movies, was producedevery year. These multimedia information, often contains unhealthy contents,especially the violent plots. Watching violent content will seriously affect thedevelopment of the physical and mental health of children and adolescents. So forthe parents, they must according to the content of the multimedia to determinewhether let their children watch the movie or not. Due to the sharp increase ofmultimedia information, just rely on manual review of the multimedia contentalready can’t satisfy the needs of the present stage. Therefore, we need to studythe automatic detection technology to detect the multimedia violence scenes.Most of the previous analysis of violence according to analyzing picturecontent. It had a low detection rate, and some violent character is difficult todefine, such as terrorist screaming. In the movie based multimedia data analysis,this paper studies the multimedia violence fragment detection method based onaudio-visual channel integration.Firstly we proposed a new shot segmentation method based on the colorhistogram and spectrogram. The shots include the image data and thecorresponding audio data. This paper present a shot segmentation method basedon combining the color histogram and spectrogram. Experimental results showthat our shot segmentation algorithm based on combining audio and videoinformation can effectively improve the detection of gradual shots.For the detection of violence fragment, employing shot as the granularity ofthe data being processed, this paper explored violence detection respectivelybased on single-channel (audio features, video features) or fusion of audio-visualchannels. Through the experimental results of this paper can be seen, the effect of violence detection on dual channel better than to use any of the single channel.Finally, in this paper, evaluate the degree of violence on the violent shotswhich have been detected. We propose the violence degree evaluation methodbased on high-level semantic, the violent shots is further divided into three grades:mild violence, violence, very violence. Through analyzing the content include inthe violent shots, find out violent audio events and violent video scenes in theviolent shots, then according to the results of the analysis to evaluate the levels ofviolent shots. In the violent audio event detection work, we proposed a violentaudio event detection method based on the time delay network. Processing thecharacteristics of frames in one audio segment, we use the time integration andframes integration instead of averaging all the characteristics simple.Experimental results show that, the effect of violent audio event detection basedon time delay network is better than the averaged characteristics.