Research On Chinese Information Extraction Based On Deep Belief Nets

Information extraction is a task that automatically gives tags to texts from unstructured or semi-structured documents, and organized them into structured texts. In this era of knowledge explosion, it is needed to quickly and accurately define important contents among huge amounts of documents.DBN is developing rapidly in decades, which is a machine learning method combines unsupervised learning process and supervised learning process. It has obtained graet achievement on image process but it is not used much on infor-mation extraction. In this dissertation, DBN was studies to complete two of in-formation extraction tasks:(1) Chinese entity detection and recognition;(2) Chinese entity relation detection and recognition.Studies of DBN were processed on the following four areas:(1) Using DBN to Chinese named entity detection and recognition task. It can be accomplished by detecting entity and recognizing entity in sequence or in combination. Both strategies of system based on DBN are established in this dissertation. Word-based and character-based features are applied respectively, to figure out which one is better in DBN system. In experiments, comparison among shallow DBN with different numbers of neural elements, comparison between DBN in shallow architecture and deep architecture, and comparison between DBN and other machine learning methods are shown. It is demonstrated to be a machine learning model applicable to information extraction.(2) To introduce post-processing of Viterbi algorithm to the named entity detection and recognition systems based on DBN. DBN neglects label limitation between current token and its context. In this dissertation, Viterbi algorithm is involved to exclude illogical tag sequence and detect the one has highest proba-bility. Real probability and binary probability are used here. Real probability is from training data whereas it is not guarantee to be suitable to testing data. Bi-nary probability take every logical states equivalently which alleviate limitation to label sequence. (3) Using DBN to Chinese entity relation detection and recognition task. Sys-tems of relation detection and relation recognition in sequence and in combina-tion are both built. Word-based feature and character-based features are also uesd to figure out which one is better in this task when DBN is involoved. Fea-ture of relation samples includes unigram, entity relative position feature, de-pendency feature and so on. Different structure of shallow DBN and DBN in deep architecture are applied to find out the best network structure. Further-more, other machine learning methods are used to indicate DBN is capable of accomplishing this task.(4) To improve training processing of DBN in two ways to optimize the re-sults of named entity detection and recognition. Supervised learning by gradient descent will experience Vanishing Gradient problem. Network error message is difficult to convey to bottom layers, making the underlying layer parameters of deep network lack of effective training, resulting in less effective result than the shallow network, or unsignificant improvement. To improve the DBN network training process is described follow:a) Whenever an extra hidden layer is placed on the top of DBN, supervised learning process is appiled to adjust parameters after unsupervised learning pro-cess.b) Place an additional output layer above each hidden layer, to adjust pa-rameters in layers which connect to the output layer.At last, these two methods are also used on entity relation extracton to test their effect.