Research On One-Class Classification Models And Algorithms Based On Tensor Theory

In fault diagnosis, face recognition, network anomaly detection, text classification and many other fields, we often encounter one-class classification problems. The traditional vector-based one-class classification algorithms represented by One-class Support Vector Machine have limitations when tensor is considered as input data. In recent years, machine learning algorithms which directly use tensor as input data have attractive extensive attention of the researchers, and achieved some good results. The benefits can be concluded as twofold. First, the use of direct tensor representation helps to retain the data topology more efficiently. The second benefit is that tensor representation can greatly reduce the number of parameters. It helps overcome the overfitting problem caused mostly by vector-based algorithms and especially suits for high dimensional and small sample size problem. Therefore, in this paper, we focus on one-class classification problem based on tensor theoretical, and this dissertation is structured as follows:1. Nonlinear One-class support tensor machine. This part addresses one-class classification problem with tensor-based maximal margin classification paradigm. To this end, we formulate the One-Class Support Tensor Machine, which separates most samples of interested class from the origin in the tensor space, with maximal margin. Since the matrix kernel function is relatively simple, we first present the new algorithm with second order tensor. Based on Superior Tensor Learning (STL) framework and its bidirectional optimal projection algorithm, the corresponding optimization problem is solved in an iterative manner, where at each iteration the parameters corresponding to the projections are estimated by solving a typical One-class Support Vector Machine optimization problem. To evaluate the new algorithm, two kinds of datasets are considered:vector-based and tensor-based datasets. Since tensor representation is particular suitable for high dimensional and small sample size cases, we detail the performance of each classifier with different small sample sizes. We also discuss on time cost and overfitting problem on vector-based dataset. The considered tensor-based dataset is human face image. All the experimental results indicate the validity and advantage of the new OCSTM.2. Nonlinear support tensor data description. This part addresses a tensor-based data description named as Nonlinear Support Tensor Data Description. The main idea of proposed algorithm is to find a hypersphere in tensor feature space, which can involve most samples of the interest class with the minimum volume. We present the new algorithm with second order tensor for the simple and convenient calculation matrix kernel function, and the corresponding optimization problem is solved in an iterative manner which is based on STL framework and the alternating projection method. Since OCSVM and SVDD are equivalent for some certain type of kernel function, we also discuss the relationship between the two type of tensor-based one-class classifiers. To evaluate the new algorithm, we consider two kinds of datasets:vector-based and tensor-based datasets. Since tensor representation is particularly suitable for small sample cases, for vector-based datasets, we test the performance of each classifier with small sample cases. The tensor-based datasets in experiments are human face images, which can be represented as second order tensors and always be joined lines into vectors in traditional vector learning algorithms. The experiments indicate that, tensor representation can reserve the important structure attributes of the images, which vector representation cannot qualify. Beside, we evaluate the new algorithm with Gaussian-based kernel matrix and polynomial-based kernel matrix. All the experimental results indicate the validity and advantage of the new algorithm.3. Linear One-class support tensor machine. This part addresses one-class classification problem with tensor-based maximal margin classification paradigm. To this end, we formulate the Linear One-Class Support Tensor Machine, which separates most samples of interested class from the origin in the tensor space, with maximal margin. To solve the corresponding optimization problem, the alternating projection method is implemented, for it is simplified by solving a typical one-class support vector machine optimization problem at each iteration. The efficiency of the proposed method is illustrated on both vector and tensor datasets. The experimental results indicate the validity of the new method.4. Linear support tensor data description. This part addresses a tensor-based data description, which is named with Linear Support Tensor Data Description. We first detail the proposed algorithm with second order tensor for simplified calculation, and the corresponding optimization problem is solved in an iterative manner based on the alternating projection method. Then we extend it to high order tensor model. We evaluate the new algorithm on several vector-based datasets which are truly high dimensional and small sample size datasets. On the other hand, the direct use of tensor as input helps retain the data topology more efficiently. Thus we also consider two kinds of tensor-based datasets:human faces datasets (second order tensor) and gait silhouette sequences (third order tensor), which always are joined lines into vector in traditional vector learning algorithms. All the experimental results indicate the validity and advantage of the new algorithm.