Research On Feature Selection Algorithm Based On Information Theory

With the rapid development of computer technology and information science, people become more and more urgent with computer’s being more intelligent. John Macarthy first came up with the concept of "Artificial Intelligence" at Dartmouth Academy in1956. After that, many artificial intelligence subjects got rapid development including pattern recognition.The definition of pattern recognition is classifying the input which is through computer correctly. Pattern recognition systems consist of four segments:data capturing, pretreatment, feature extraction and selection, classification decision. As new technology appear constantly, the size of data in pattern recognition become bigger with less sample and high dimension. These give great challenge to classification learning. Therefore, how to remove irrelevant or redundant features from high dimensional data in order to avoid "curse of dimensionality" and make high dimensional data be capable to use traditional training algorithm under the environment of high dimensional data. Thus, feature selection is an important part of pattern recognition system and also be necessary to design a high performance classifier.This paper firstly introduce the research status of feature selection and then give a brief presentation of basic concept about information theory. At last, a revised feature selection algorithm based on normalized mutual information is put forward, and an unsupervised feature selection algorithm based on information theory,and tested them on datasets.