Research About Multimodal Information Fusion Method Based On Contextual

On the Internet, there are a massive of audio-visual and other types of information resources, which have the real-time or non-real-time, determined or obscure different, complementary or mutually exclusive characteristics. And these features require information service system to provide multimodal information to meet the information requirements of users. However, the forms of multimodal information being diverse and the relationships complicated, if treated in isolation, it can result in wasting information resources. Hence, it is necessary to fuse information, so as to provide more accurate information for the user. The information fusion technology has fully used the complementary and gain of more source information, improving the quality of information processing. For the multimodal information integrated service of current Internet environment, current information fusion technology exists some limitations, which cannot effectively fuse the polymorphic semantics of multimodal information and which is difficult to achieve the collaborative requirements of multimodal information fusion.This article uses the guidance of the high-level semantics and context information of multimodal information environment, through cooperative learning methods to establish multimodal characteristics of joint and joint space, achieving the multimodal information fusion at the characteristic level and at the decision-making level.Main research contents are as follows:Chapter one, mainly introduces the background and significance of the research in this article, domestic and international developments of information fusion and the main content of this article.The second chapter describes contextual information, correlation analysis, the research and development of association method in information fusion, and the analysis of context information in fusion processing.The third chapter introduces the multimodal information fusion method.The fourth chapter describes the multimodal information fusion process, fusion method and its application of the feature level.The fifth chapter describes multimodal information fusion process, fusion method and its application of the decision-making level.The sixth chapter introduces multimodal information fusion process, fusion method and its application of the hybrid level.The seventh is a summary in this article, including research results and prospects of this article. The research method of this article mainly contains literature analysis method, literature research method and so on. This article analyzes the application of the contextual information in multimodal information fusion, summarizes the multimodal information fusion method, and makes a detailed investigation on the specific application of the information fusion method in different fusion levels in the relevant books.