Research On The Technology Of Disambiguation For Chinese Word Srgmentation

Search engine is an important tool for people to search information andknowledge.And Chinese segmentation,as an important part,is also the hot researchproblem in this field in recent years.Disambiguation technique is an important partof Chinese segmentation,this technique eliminates the ambiguity generated by wordsegmentation through a certain strategy, so as to improve the accuracy of thesegmentation,the main contents of this paper are as follows:(1)study the background,significance,domestic and foreign status of the Chineseword segmentation disambiguation technology.(2)study Chinese word segmentation disambiguation technologies andalgorithms.Including reasons of ambiguity, classification of ambiguity, algorithms ofdisambiguation and the challenges faced in ambiguity processing stage.(3)propose an improved algorithm of ambiguity resolution.In this algorithm,usethe support factor as segmentation standard,resolve the overlapping andcombinational ambiguity identified by the ambiguity matrix.For overlappingambiguity,according to the distribution of ambiguity in the document,structuredifferent support factors,finally determine the segmentation from front or frombehind through the size of the support factors;for combinational ambiguity,structuredifferent support factors,finally determine the segmentation from the bonding orfrom the point through the size of the support factors.(4)design and implement an Chinese segmentation system of overlapping andcombinational ambiguity based on support factors.The system consists of fourmodules,namely preprocessing module,preliminary segmentation module,digestionidentification module and ambiguity resolution module.In the design of the dictionary,use multiple text files to improve the access speed.