Research And Implementation Of A New Concurrent Segmentation Algorithm

Word segmentation is the first and a fundamental step for Chinese information processing, and is always one of the most important topics of Chinese Processing. To raise the accuracy and efficiency of word segmentation, this paper presents a method of word segmentation. It can make the processes of each word search, word segmentation and syntactic-semantic analysis parallel, more efficient.Based on current methods of word segmentation and analysis, a method of omni-word segmentation is discussed in this paper. And particularly, describes the word-parallel search and the word-persisting searching of word Omni-Segmentation are discussed. Then, a word-parallel search & colligated- ambiguity-recognizing model of word omni-segmentation is proposed. In this way, the approach performs the segmentation while the input is entering, and exist many searching process, and at the same time the ambiguity recognition with simple syntactic analysis is achieved. In this way, the parallel of input and segmentation, the segmentation and ambiguity-recognizing is realized, and the correct segmentation is obtained at the end of input. The system architecture and the parallel algorithm for the approach are given in this paper, and a simulation system is constructed. The simulation of the method has proved that the idea proposed in this paper is both feasible and effective.The approach may reveal something of value both in theory and application. It adopts the current parallel computing technology for the achievement of the parallel implementation of word-search and ambiguity-recognizing. In this way the process of word segmentation is greatly accelerated by making full use of the hardware resources. Its implementation on parallel hardware can make it possible for the implementation of high-speed and applicable projects and give support for practical large scale and parallel Chinese information processing.