| Most learning systems use hand-picked sets of features as input data for their learning algorithms. This is particularly true of computer vision systems, where the number of features that can be computed over an image is, for practical purposes, limitless. Unfortunately, most of these features are irrelevant or redundant to a given task, and no feature selection algorithm to date can handle such large feature sets. Moreover, many standard feature selection algorithms perform poorly when faced with many irrelevant and redundant features. This work addresses the feature selection problem by proposing a three-step algorithm. The first step uses an algorithm based on the well known algorithm called Relief [54] to remove irrelevance; the second step clusters features using K-means to remove redundancy; and the third step is a standard feature selection algorithm. This three-step algorithm is shown to be more effective than standard feature selection algorithms for data with lots of irrelevance and redundancy. In another experiment a data set with 4096 features was reduced to 5% of its original size with very little information loss. In addition, we modify Relief to remove its bias against non-monotonic features and use correlation as the distance measure for K-means. |
