The Research Of Motion Video Semantic Analysis Technology Based On Multi-modal Characteristic

With the continuous development of computer technology and network techno-logy, the trend of the scale abundance and form diversification for the internet data has emerged. As an important internet data, the proliferation of motion video poses a challenging problem for the automatic analysis, interpretation and indexing of video data. The conventional analysis, management and retrieval methods for text data are not suitable for digital video. To this end, various video-content-based semantic analysis algorithms were proposed, which will promote the improvement for the retrieval, indexing, and reference of motion video.In recent years, a wide variety of content-based semantic video analysis approaches have been proposed. However, there are not a few drawbacks for them. For example, the algorithm is always designed too complex, which leads to the large scale of computation to some extent; the knowledge and rules specific to certain domain are overemphasized, which limits their usages and constrains their generality; the multimodal characteristic of video data is not utilized fully. Under this circumstance, this article will try to explore video analysis approaches to address the above-mentioned issues. And the main contributions are listed as follows:1) A generalized video semantic analysis algorithm based on the fusion of multiple local features, including motion features, color features and texture features, is proposed. By introducing the concept of Kurtosis of motion energy, the artifacts of noise and lens jitter are reduced, and accurate motion regions are obtained. The features of motion, color and texture are extracted from in these areas, and Hidden Markov Models (HMM) are exploited to associate every video shot with a particular semantic class. Experimental results on Tennis and Football video sequence show that the proposed approach can achieve a relatively high ratio of correct semantic recognition.2) An effective sports video semantic analysis algorithm based on the fusion and interaction of multi-models and multi-features is proposed. By utilizing the semantic color ratio, the video shot is classified into global shot, in-field shot and out-of-field shot, which facilitates the HMM-based classification behind. For shot corresponding to a specific scene, by introducing image registration based on SIFT feature, the artifacts of noise and camera movement are reduced, and accurate motion features are obtained. Then, Hidden Markov Models (HMM) are exploited to associate every video shot with a particular semantic class. Experimental results on Tennis and Football sequence show that the proposed approach can achieve a relatively high ratio of correct semantic recognition.