Small Sample Based Linear Dimension Reduction Algorithm And Applications

With scientific and technological development, people need to process an increasing number of high-dimensional data, and the curse of dimensionality problems become more and more significant. Therefore, it is very important to reduce the dimensionality of the high-dimensional data. Linear dimension reduction algorithm is an efficient way to solve this problem. However, in practice, most linear dimension reduction algorithms often suffer from the so-called small sample size problem. In this thesis, we have explored some linear dimension reduction algorithms to cope with small sample size problem:1. By using Pseudo-inverse Linear Discriminant Analysis algorithm as a bridge, we prove that Direct Linear Discriminant Analysis algorithm (DLDA) and Linear Discriminant Analysis via QR Decomposition (LDA/QR) are equivalent. And thus both DLDA algorithm and LDA/QR algorithm are a type of Pseudo-inverse Linear Discriminant Analysis algorithm. That is to say, the Pseudo-inverse Linear Discriminant Criterion is a general solution to the small sample size problem. In addition, we prove that the DLDA algorithm is also a two-stage algorithm. Interestingly, the first stages of these two algorithms are equivalent too. And in some applications, the first stage of these two algorithms can also be individually regarded as a linear dimension reduction algorithm. To this end, we introduce the concept "Lightweight Linear Discriminant Analysis" algorithm to summarize the first stage of the two algorithms.2. To deal with the small sample size problem in the Locality Preserving Projections (LPP) algorithm, we propose a Pseudo-inverse Locality Preserving Projections algorithm. We develop an effective algorithm to solve the pseudo-inverse generalized eigenvalue problem via using a matrix diagonalization technique, which can diagonalize three matrices simultaneously. In addition, we also proposed an effective LPP algorithm via QR decomposition, namely LPPQR, which can be applied to cope with the small sample size problem. And also, we prove that the LPPQR algorithm and the pseudo-inverse LPP algorithm are equivalent in theory. Hence, we can regard the LPPQR algorithm as an effective implementation of the Pseudo-inverse LPP algorithm. This also indicates that the Pseudo-inverse LPP algorithm is a generalized solution to deal with small sample size problem in LPP algorithm. 3. A large number of comparative experiments verify our theoretical results. In addition, we apply the two proposed algorithms to appearance-based palmprint recognition. And also these two algorithms achieve encourage recognition accuracy.