Research And Implementation For Chinese Lexicon Analysis System Based On Neural Network

The thesis aims to build a suite based on neural networks for Chinese lexical analysis,including CWS(Chinese Word Segmentation),POS Tagging(Part-OfSpeech Tagging)and NER(Named Entity Recognition).The research contents of this thesis including two points.One for researching a certain neural network model structure which is suitable for Chinese lexical analysis,and the other we study the implementation of the structure.Firstly,we introduce the tasks of Chinese lexical analysis from multiple aspects and investigate the existing systems.Then we divide the neural network structure of sequence labeling into input layer,representation learning layer and label predicting layer and introduce them one by one.Through experiments,we explore the model effects on every task by using different inputs and model structure.Although these structures we get are different,but they all has an Bi-LSTM layer to learn input representation,with handcraft feature or unlabled data.Finally,we introduce the framework of system implementation,and speed benchmark is conducated.The thesis has 2 contributions.First,we conduct the experiments and determine the neural networks which are suitable for each Chinese lexicon analysis task.LTP is selected as baseline,and the datasets of LTP are took as our experimental datasets.In CWS task,the F1-score of our model on development set and test set is 0.33 and 0.48 percentage points higher than that of LTP.And in POS Tagging,the optimal model's accuracy in development set is 0.2 percentage points higher than baseline and 0.22 higher in test set.In NER,our model achieves 2.57 and 0.57 percentage points of F1-score on development and test data than that of LTP.The second research result is embodied in the system implementation.The system is built with clear code structure and we get a available Chinese lexicon analysis system.