Violent Video Detection Based On Dynamic And Static Information Fusion

In an age of rapidly developed Internet, the growth and the spread of the social media makes us have full access to different types of information. However, some information contains harmful contents including pornography, horror and violence, can lead significantly adverse affects on the mental health of adolescents. Recent years have witnessed the extensive research that have been conducted to design effective webpage filters for detecting and recognizing undesirable contents automatically. Despite some successful software for filtering pornography and horror available publicly, the research of analyzing semantics of violent video is still limited. Due to massive studies of social science revealing the detrimental effect of violence information on children's physical and psychological health, an effective violent video scenes recognition algorithm is necessary in real-world web scenario.In violent video recognition, the following two problems require to be addressed:(1)How to determine the salient area in the image and extract the visual features for representing“violence”?(2) How to simultaneously combine dynamic and static information to identify the violent video? Correspondingly, our work mainly focuses on the two above-mentioned respects, which can be summarized as follows:1. We propose a path-based blood color and fire detection algorithm in this thesis.Specifically, we first use regular patch generation method for achieving effective image segmentation. To improve computational efficiency, integral images are employed to accelerate the feature extraction procedure. Then, we convert the original image from RGB color space to orthogonal YCb Cr color space and extract the patch texture features based on covariance descriptor from each small patch. Next, we extract the patch color features by computing the mean and standard deviation of patches in the Cr and Cb components respectively. The patch texture features and the patch color features are concatenated to compose the patch features, which are subsequently fed into a Support Vector Machine classifier for evaluating the decision scores.2. We propose an algorithm for recognizing violence films and streaming media by fusing the dynamic and static information. Taking into account the prior knowledge, we extract the motion features as the prior and adopt a Bayes theory based fusion algorithm to combine the recognition results from dynamic and static contents. Experimental results demonstrate that our algorithm unifying dynamic and static modalities is more robust to noises and achieves superior performance than those which use a single modality.In general, our approach, which is built from discriminative “violence” features and allows fusing dynamic and static, provides a promising paradigm for violent video detection.