| There are huge text information data on the Internet,how to find useful information or filter irrelevant information in good and bad information becomes a key issue.The key of text filtering is text similarity computation,traditional text similarity calculation is mostly based on word frequency statistics or keyword calculation method,which can not reflect the semantics,which leads to the lower accuracy of text similarity calculation.More and more attention has been paid to semantic based information filtering in recent years.And the existing semantic framework based on text similarity algorithm research in calculation of sentence or text similarity,ignoring the text similarity calculation part of the importance of long phrases,can not deal with the complex statements,and can not reflect the semantics of text very well,so the filtering accuracy is low.In order to solve the above problems,this paper proposes a network text filtering algorithm based on the semantic framework of multi predicate words.The main contents of the algorithm include:text dependency parsing,semantic framework filling,long phrase text processing and frame similarity calculation.In order to better reflect text semantics,in addition to considering the backbone elements of the semantic framework(subject,predicate,object),the composition of the frame is also inserted into adverbial,time,place,way and other elements.In dealing with the similarity computation of long phrase,we first use dependency syntactic analysis to build phrases into a tree.Then we use AHP to determine the weights of all levels,and get the similarity of long phrase texts based on the similarity of nodes at different levels.After comparing the accuracy of the sentence,short text and long text,we can see that the similarity calculation of this algorithm has reached a high accuracy.Based on the algorithm in this paper,a network text filtering system based on the semantic framework of multiple predicates is designed and implemented. |
PDF Full Text Request