Web Short Text Oriented Knowledge Association Model And Semantic Coherence Computation Method

With the promotion of Web 2.0 technique and the opening up of social media platform, ‘we media’ era arrives. ‘we media’ enables web users to freely report, reply or comment on hot events via microblogs, BBS or comments, which generates massive short texts. These short texts have properties including short content, unconstrained expression, large scale, and rapid growth. Besides, some short texts may focus on one or more topics or domains and thus generate semantic association between them. However,other properties of short text include sparse association, noise disturbance, undesirable redundance and incoherent semantics, which offer challenges for the existing short text processing which is still in the exploratory stage. The challenging issues include how to discover knowledge association patterns, how to extract core semantics of event, how to generate coherent knowledge flow and how to measure semantic coherence.In this paper, we propose web short text oriented knowledge association model and semantic coherence computation method, where knowledge association model is the foundation of semantic computation, aiming at generating coherent knowledge flow. To address the above 4 challenging issues, we focus on 4 research issues as follows:(1) To discover keyword association patterns, we propose association pattern discovery model, which discover different association patterns by divide-and-conquer pattern discovery and enrich knowledge association by schema-based optimization.(2) To obtain concise and core semantics, we propose semantic association based core short text extracting model, which obtain the whole association relation distribution by learning semantic Markov random field and cover the association distribution by discovering minimal number of short texts to maximize information gradient.(3) To generate coherent knowledge flow, we propose cognitive memory-inspired knowledge flow model. The model learns three types of cognitive logical structures and their corresponding association distributions. Different association distributions decide that knowledge flow spreads and activates toward different directions. A coherent knowledge flow is generated by the process of structure shift and the process of spreading and activation.(4)To measure semantic coherence, we propose a semantic link network based coherence measurement model for measuring semantic coherence under different organizations. The model obtains coherence state by constructing semantic coherence network from which coherence features are extracted, measures semantic coherence by learning the weight of coherence features and discovers coherence patterns by the combination of coherence features.The paper focuses on the establishment of the theory and method of knowledge association for web short text processing and semantic coherence analysis. The research promotes the development of the short text information processing, cognitive informatics and other related research domains. The research results can be applied in intelligent search system, knowledge decision system, intelligent question-answer system and so on.