| Similar text recommendation has important applications in related fields of information retrieval and natural language processing.In the era of personalized service,the personalized recommendation of its evolutionary upgrade version provides more efficient and direct information resources for individuals.In some areas,for the users’ privacy or behavioral information is strictly required,that requires similar text recommendation only in plain text information.The most traditional method only relies on the key words based on statistical information,without considering the semantic information of words in the text and the semantic information of the text itself.With the rise of machine learning and deep learning in recent years,word vectors have achieved good results in representing lexical semantic information.Word vectors are used to represent text,and distance measure functions are used to find similar text.This kind of superposition of semantic information on vocabulary indicates that the information of text is still not good enough to summarize the information of long text.Considering the combination of statistics and neural network,this thesis proposes a similar text measurement method which combines keywords and weighted keywords to quantify text.The experimental results show the effectiveness and feasibility of this scheme.In this thesis,we mainly study Indonesian similar news recommendation based on neural network,and propose a new similarity measure on pure text.Neural network mainly uses its word vector model to effectively represent the lexical semantics.Indonesian is the language of application.The work of this thesis is as follows:(1)In the process of recommendation and screening using keyword representation text,TF-IDF,LDA and TextRank are introduced in detail.Experiments show that the recommended results of TF-IDF algorithm are not inferior to those of LDA and TextRank algorithm,which take into account more text context links and implicit topics.(2)To quantify the text and use it in similar recommendation,this thesis mainly introduced the word vector model based on neural network and the text vector model doc2 vec.The experimental results show that the method of word vector addition and average quantization of text performs worst in text quantization,and the weighted word vector average and doc2 vec model perform better.The disadvantage of doc2 vec is that it also needs training to test news.(3)The characteristics of Indonesian language are analyzed,and the corresponding changes in data processing are made according to the characteristics of Indonesian language.A similarity measurement method combining text keywords and text vectors is proposed.The experimental results show that this method is better than the keyword technology or text vector technology,which proves that the trade-off between the two methods can combine the advantages of both.(4)Experiments in Chinese and English show that the proposed method is equally effective. |
