Research On Discriminant Least Squares Ordinal Regression And Application

Ordinal regression is a special supervised learning in pattern recognition, the problem comes from the discrete and ordinal label structure widely appears in real-world. But the traditional supervised learning algorithms are difficult to guarantee the desired performance due to not using ordinal information. Ordinal regression's objective is to classify patterns by using a between-class natural order property between the labels, and it is expected to obtain higher classification accuracy, while keeping the predicted labels as consistent with the true label. In recently years, ordinal regression attracted the attention of numerous researchers, and many algorithms have been proposed, which can be roughly classified into three categories. But the least squares regression(LSR), to our knowledge, has not yet been applied to ordinal regression scenario. As a classical regression method, LSR is widely used for pattern recognition due to good theoretical support. The LSR is optimizing the mapping between samples and labels, thereby minimizing the sum of square error. The LSR has not yet been ordinal transformation partially due to ordinal information is difficult to be used directly.In this paper, we firstly develop a novel least squares ordinal regression(LSOR) by using cumulative label in order to embed ordinal information. Considering LSOR only reflects the rank, but does not reflect the discrete. We lastly proposed a new discriminant least square ordinal regression(DLSOR) by using both the cumulative labels and margin-enlarging technique. DLSOR which without constraints imposed on the regression function, only through the label transformation achieve the goal of embedding ordinal information and expanding between-class distance, as a result, a high classification accuracy associated with low mean absolute errors can be guaranteed on the premise that DLSOR's complexity of model has consistent with LSR. Through a series of experiments on benchmark datasets, we demonstrate the superiority of DLSOR.Furthermore, motivated by generalization performance will largely be constrained when facing the limited size of samples in real applications. Inspired by recently-proposed marginalized corrupted features(MCF), we propose a performance-improved least squares ordinal regression using doubly corrupted features(LSOR-DCF) which is based on LSOR is developed by corrupting both the samples using random noise from known distributions and the labels using deterministic biases. The experimental results demonstrate the superiority of LSOR-DCF in performance, especially in small data sets, to related methods without adding either noise in samples or corrupted noise in samples and labels alone.The major contributions of this dissertation are that through the ordinal transformation of LSR, expect to generate fourth ordinal regression algorithm with implicit embedding ordinal information. Moreover, the strategy of ordinal transformation and double corruption features can also be extended to other algorithms.