Design And Implementation Of Sentence Level And Paragraph Level Semantic Similarity Algorithms

With the rapid development of natural language processing tech nology,sentence level semantic similarity algorithm has more applic ation scenarios,such as selecting the most matching answer from th e database to the user's question,selecting the best translated word sequence from the data set,and text classification and sorting.Para graph level semantic similarity algorithm also has more application scenarios and greater research value.Compared with sentences,para graphs are more complex in structure and higher in dimension,so i t is more difficult to study the semantic similarity of paragraphs.F or the research on the semantic similarity algorithm at the sentence level and paragraph level,the following three contributions are ma de in this paper:Firstly,the semantic similarity algorithm at the sentence level i s improved by using the multi-tapping attention extraction method i n the self-attention mechanism.Compared with the method of short-t erm and long-term memory network,the semantic similarity algorith m proposed in this paper has two advantages:first,it can obtain mu ltiple feature maps,so as to achieve multi-angle extraction of sema ntic features;Second,it can directly calculate the semantic correlatio n between any two words in a sentence.Compared with the basic m odel,the improved sentence level similarity model is more effective and the community q&a system is optimized.Secondly,a paragraph level similarity algorithm based on text a bstract is proposed.In textual research,paragraphs have larger text spans and more complex dimensions than sentences.In order to red uce the difficulty of semantic calculation caused by the difference o f length and dimension between paragraphs,this paper proposes a p aragraph level similarity method based on the generated abstract.A bstract can not only express the main ideas of paragraphs,but also reduce the dimensional differences between paragraphs.This metho d is easy to calculate and improves the efficiency of semantic com putation.Thirdly,the paragraph level semantic similarity algorithm is fur ther improved by introducing interactive information layer and cross-attention mechanism.In traditional semantic similarity computing,te xt representation only contains its own semantic information.The me thod proposed in this paper integrates the interactive information be tween text pairs through the interactive information layer and the cr oss-attention mechanism,enhances the connection between text pairs,combines with the sentence level similarity algorithm,and optimize s the community question answering system.