Font Size: a A A

Maximum Entropy Method And Its Applications In Natrual Language Processing

Posted on:2006-03-22Degree:DoctorType:Dissertation
Country:ChinaCandidate:Y Q ZhouFull Text:PDF
GTID:1118360155460569Subject:Computer software
Abstract/Summary:PDF Full Text Request
Computer technology has been greatly developed in recent years. The memory has become more and more bigger, and the computational speed is more and more faster, while the prize is more and more cheaper. All these external factors make the corpus based statistical natural language processing methods has become the hotspot in the field of natural language processing. Maximum entropy method is a kind of statistics based machine learning method. During last ten years, it has been successfully applied in many fields of natural language processing, and already or nearly achieved the state of art in these fields.The so-called maximum entropy method is to find a model keep to the maximum entropy principle, which means select the statistic model that has the maximum entropy and satisfy all the constrains. The advantage of maximum entropy method is it is based on the promising philosophy principle and has solid mathematic deduction. Under the uniform framework of maximum entropy, it is very convenient to use various features furthermore there is no independent assumption. Its shortcoming is time and space consuming of training.In this dissertation, we first introduce the basic theory, mathematics deduction and algorithms of maximum entropy method. Then propose two fast methods for training and execution, Selective Gain Computation algorithm and Sparse Feature Tree, accordingly. Selective Gain Computation algorithm is a fast feature selection method, which can speed up the feature selection process hundreds or thousands times. The Sparse Feature Tree's time complexity is in direct proportion to the logarithm of feature number. At last we introduce our maximum entropy tool kit and our practice of applying maximum entropy method in natural language processing.
Keywords/Search Tags:Maximum Entropy Method, Maximum Entropy Model, Natural Language Processing, Feature Selection, Feature Matching
PDF Full Text Request
Related items