Geometric linear discriminant analysis for pattern recognition systems

More often than not, the complexity of a pattern recognition system is directly related to the difficulty of the task. As the tasks become more difficult, like recognizing objects in digital images or processing human speech, the systems that successfully complete those tasks are also becoming increasingly more complex. A contributing factor to the system complexity is the dimensionality of the observation space. That is the number of input features for a pattern recognition system. As the dimension of the observation space grows, there is an increased difficulty to effectively use the statistical-based methods found in those systems. First, the run-time of a system is related to the dimension of the problem. A system that takes seconds to run is preferred over one that takes weeks to run. Second, it is unusual to have enough data to accurately estimate the parameters of the statistical models, and poorly estimated models give way to systems with sub-optimal performance. Finally, the ultimate judge of system usefulness is its performance, which is dependent by the information feed into the system. An attempt to increase system performance by adding more features to the observation space only exacerbates the other two problems.;In this work, a new method called Geometric Linear Discriminant Analysis (Geometric LDA) is presented to address the problems associated with the dimension of the observation space. Geometric LDA employs a novel idea that exploits the geometry of the problem. It uses a numerical optimization procedure to find a p x d matrix whose elements provide a measurement of feature relevance. The matrix can be incorporated into a feature analysis tool that can be used to select a subset of features or reduce a d-dimensional observation space to a p-dimensional observation space by means of a linear transformation. Although dimensionality reduction by means of a linear transformation is not a new idea, Geometric LDA offers a promising approach to the problem of finding the "best" transformation matrix.