Chinese Information Extraction Based On Hybrid Neural Network

With the development of network technology,a large amount of Internet information arises and how to extract the users' needs from massive network data is a key issue in the current information discipline.Statistical method has becoming the spot right now due to the mature of machine learning theory.Currently,there are two main kind of method dealing with Information Extraction problem: base on rule and base on statistic.Each of two methods has its own limitation and advantage,therefore combing these two also can yet be regarded as a good way.Event Extraction is one of the main tasks of the Information Extraction.Its main purpose is to extract the information of event related elements in text.To solve the problem of Chinese Event Extraction,this paper transforms event element recognition into sequence annotation task and builds several deep neural network models to solve the problem based on CEC,a Chinese emergency event annotation corpus annotated by Shanghai University.One kind of network model dealing sequence annotation task is word2vec-BILSTM-CRF model,which is seen as baseline,compares with several training models proposed in this paper.(1)Training Model using BERT and Self-Attention layer: This paper introduces a new network model by using BERT pre-trained model proposed by Google Company to replace char embedding layer.The result of this method outperforms the common model in many ways,and F1 value increased by 13%.To further improve recognition result,this paper also works on self-attention mechanism and applied selfattention layer in this model.By adjusting the training parameters,the F1 value of the model recognition effect is improved by 5%,and the accuracy is up to 75%.Similarly,compared with the benchmark model with self-attention model,this model's F1 increased by 7%.(2)Hybrid neural network model based on self-attention mechanism: Besides adding self-attention layer,this paper also adds an event type classification network to combine a hybrid network.Hybrid neural network means that multiple training tasks are carried out at the same time,each training task shares the underlying parameters,and the training error is allocated according to a certain weight ratio.Based on CEC training corpus,this paper found the right weight ratio through several small batch training tests,and takes the loss obtained by adding the two networks as the loss function of the whole training network.On the basis of(1)model,the accuracy,recall and F1 value of this model are improved by 2%,and the F1 value is 78%.(3)Combined dependency parsing and Chinese Information Extraction network model: Model(1)and(2)are based on statistical method,which is slightly inferior to the rule-based automatic annotation method in event object recognition.Therefore,this paper combined rule-based method and statistical method,proposes a model that combining dependency parsing and deep neural network to deal with the task of event extraction.Dependency parsing is one kind of syntactic analysis technique to obtain the dependency relationship between words in a sentence.In this method,the LTP analysis tool is used to analyze the dependency syntax of the training sentence to get the dependency structure of the sentence,and then the training sentence is roughly segmented according to the analysis results.After segmentation,the redundant segmentation characters are added to the training text,and then input into the deep neural network model to learn the network parameters.This paper also uses the text segmented by mixed granularity as the training data.After many experiments and adjusting the parameters,we get a satisfactory recognition effect.The F1 value of the whole system is 80%,and the F1 value of the final event object element recognition is 77%.