Research On The P2P Information Retrieval For Chinese Text

Compared with the traditional centralized of information retrieval system, the information retrieval system based on Peer-to-peer(P2P) is strongly fault tolerant, well expansible, and cost less. It can fully excavate the edge resources of the network, and offer individualized information service.In the P2P document information retrieval technology, the Word Segment plays an important role. It is the foundation and the precondition of the information processing. The traditional mechanical participle has already achieved great success, however, for the Chinese documents resources, it still has the obvious insufficiency: It is unable to set the word length accurately which the algorithm needs, called long-term problem, which can initiate the contradiction between accuracy and efficiency contradiction; It can not solve the problem that how to deal with the different meanings when segmenting the Chinese words, and so on.Based on these considerations, this thesis provides a new Chinese word segmentation algorithm which improves the problems with the traditional segmentation algorithm. It changes the traditional thinking based on the relational database, using the tree-like structure. It achieves good results in the application of the document information retrieval based on the P2P technology.The thesis introduces the notion of the tree-like structure word library. The words are ranged according to the tree-like structure, causing the participle matching gets liner effects. It not only solves the long-term problem, but also ensured the efficiency of the word segmentation. In view of the problem of how to deal with the different means the word segmentation gets, it uses branch processing method and dynamic programming solution with the dictionary in tree-like structure, getting good effects.Based on results of the Chinese word segmentation, the thesis has detailed the Lucene—a full text search engine toolkit, and designed a full text index solution based on the Lucene. The solution builds the index by the word for the full text searches. It achieved good results in the test.