Sparse Low Rank Bilinear Discriminative Model And Its Application In Image Classification

The dilemma of high dimensionality and small size samples, which is widely encountered in many machine learning applications, trends to cause model’s overfitting. The main reason is that, if we use the high dimensionality data, such as image data, the number of sample we can gain, is far less than the dimensionality of image data. The great gap between the data number and the dimensionality causes the model overfitting, reduces the accuracy and sometimes even damages the modeling process.As we know, low rank approximation to parametric matrix has recently been proven to be an effective method to control the complexity of models. However, in most of the current low rank methods, it is required to specify the target rank by hand beforehand (e.g., in PCA). Setting a reasonable rank number beforehand by knowing nothing about the data is barely possible, especially by hand. So, it is an interesting and challenge topic to find a way of getting the rank number inside the data without assuming the data rank. Although the topic seems difficult, we still have some good hint. Imposing the sparsity constraints on the parametric matrix can avoid setting rank.In this paper, we focus on image classification, especially two-class problem. In order to preserve the integrality and potential inside connection of the image, we use the matrix-pattern as the input pattern of our discriminative model, which is different from the original vector-pattern model. The main contribution of this dissertation is detailed as follows:We propose a novel low rank approximation discriminative model to deal with overfitting by using matrix-pattern as input data pattern. In particular, under a bilinear discriminative framework, we decompose the parametric matrix and simultaneously constrain their ranks with the sparsity-inducing regularization. The resulting problem can be efficiently solved with coordinate descent. Our method is able to take the spatial information of structured data into account, leading to improved generalization capability. The feasibility and effectiveness of the proposed method is demonstrated on several image databases.