Chinese New Word Recognition Technology Research And Application Based On Flow Text

With the booming popularity of web2.0technology and network, more and more people actively play the existence of self worth and actively participate in all network topics, thus creating a lot of new words emerge in people’s daily lives. These new words of network with less vivid symbols but express more information, and verbal communication is the need of future, and because of its widespread use speed and other characteristics, it have been increasingly valued by linguists, while it is the inevitable question in Chinese information processing field. Chinese new word recognition technology is technical support of automatic segmentation, human-computer interaction, online translation, and other important areas.Therefore, this paper is committed to research on new words recognition, including traditional new words and new words on the internet. Effective approach is proposed to recognize new words and corresponding system is developed.The main work of this paper includes the following aspects.(1) A method for traditional new words recognition in single string mode is designed. Also, the maximum method is proposed to identify traditional new words in this mode quickly and effectively.(2) In terms of traditional new words in suffix mode, inductive method and threshold method are compared with each other. Inductive method is used to recognize traditional new words in this mode. Because of the difference between new words on the internet and traditional new words, new words on the internet are identified based on the track of frequency magnitude of traditional new words.(3) A system is implemented to identify new words based on the algorithms above. At present, all of the methods proposed in this paper have been applied to new words recognition system and the result is satisfying.