Study Of Feature Selection Based On Information Theory

The emergence of high-dimensional machine learning fields such as image processing,information retrieval and bioinformatics pose severe challenges to the existing feature selection and machine learning algorithms. This dissertation mainly studies on feature selection and dynamical feature selection with unsupervised learning.In this dissertation,we firstly review the basic knowledge of information theory and feature selection algorithm,and detaily introduce some feature selection algorithms,in which ReliefF algorithm is considered to be an efficient one of Filter category.Given the shortcoming of ReliefF method,we improve it by using Kullback divergence,which is an important concept in information theory.This improving makes the feature selection result more efficient.In order to obtain more effective feature subset in a short time,we promote a new feature selection framework which can be achieved in two steps. Algorithm using this framework can remove redundant features effectively and obviously reduce the time complexity comparing with subset evaluation algorithm.We also do some study in unsupervised learning feature selection method,which can automatically amend the feature subset when there are new unlabelled instances are added.