Novelty Detection Based On Robust Kernel Principal Component Analysis And Kernel Entropy Component Analysis

Novelty detection is regarded as an important research issue in the field of machine learning.Although the traditional novelty detection methods can achieve good classification performance on the low-dimensional data,they usually achieve poor results on the high-dimensional data.traditional novelty detection methods often get poor results on the high-dimensional data.To effectively solve the problem of‘curse of dimensionality'in the task of novelty detection,a feasible method is to eliminate redundant and irrelevant features in the high-dimensional data.That is to say,utilizing dimensionality reduction to decrease the dimension of these high-dimensional data.The existing dimensionality reduction methods can be classified into two categories,i.e.,feature selection and feature extraction.As two commonly used nonlinear feature extraction approaches,kernel principal component analysis(KPCA)and kernel entropy component analysis(KECA)have obtained extensive attentions and applications.The two methods are both based on the kernel method.They conduct nonlinear transformation on the original data and extract features that possess more discriminant ability.In this dissertation,the novelty detection methods based on KPCA and KECA are respectively investigated.The main work are as follows.1.The method of maximum correntropy criterion based KPCA for novelty detection is proposed.The traditional KPCA based novelty detection method is very sensitive to noise.If there exist noise in the given training samples,the detection performance of KPCA based novelty detection method may be decreased.To enhance the anti-noise ability of KPCA based novelty detection method,a maximum correntropy criterion(MCC)based novelty detection method is proposed.Correntropy in information theoretic learning is utilized to substitute thel₂-norm based measure in KPCA based novelty detection method.Through adjusting the width parameter of the correntropy function,the adverse effect of noise can be alleviated.The half-quadratic optimization technique is used to iteratively solve the optimization problem of the proposed method.The local optimal solution can thus be obtained after a few iterations.Moreover,the algorithm description of the proposed method is provided.Experimental results on the benchmark data sets demonstrate that the proposed method obtains better anti-noise and generalization performance in comparison with the other three related approaches.2.The novelty detection model based on KECA and the improved Cauchy-Schwarz(CS)statistic is proposed.The CS statistic is regarded as the angle cosine between two vectors in the feature space.Therefore,the classification methods based on the CS statistic often cannot obtain better performance when the differences of angles among the given data are significant.To reduce the degree of dependence for the CS statistic against the angles among the given data,an improved CS statistic is proposed.Through changing the value of its parameter,the degree of dependence towards the CS statistic against the angle difference among the give data can be adjusted.There are two stages to construct the proposed novelty detection method,i.e.,feature extraction and classifier construction.In the phase of feature extraction,KECA is utilized to conduct feature mapping upon the original data.The feature vectors are chosen according to their values of Renyi entropy.A projection matrix can thus be obtained by the achieved feature vectors.Then,the projection matrix is utilized to conduct nonlinear mapping on the original data.In the phase of classifier construction,the improved CS statistic is used as the similarity measure between different vectors.On this basis,the decision threshold can be determined.The efficiency of the proposed novelty detection method is validated by comparing with the other three related methods upon the benchmark data sets.