Research On Multiple Respects Of Support Vector Machine

Support Vector Machine (SVM) is one of the most important derivations of the Sta-tistical Learning Theory (SLT). Because the SVM has a strong theoretical base and amaz-ing experimental performances, it has been widely used in our real-world life. Currently,the research areas that aim to make the SVM perform better in many kinds of real-worldapplications are hot, and many SLT research branches have been developed. From thispoint of view, this thesis contributes to three SVM based SLT branches. They are theSVM based Multiple Kernel Learning (MKL), multiclass classification, and unsupervisedlearning. The theoretical contributions of this thesis are further applied to the areas of theaudio signal processing. The main contributions of the thesis are summarized as follows:1. The efcient structural SVM algorithm was proposed. Specifically, the structuralSVM is a technical combination of the advanced kernel weight optimization algorithmand the state-of-the-art single kernel SVM. The new algorithm not only has linear timeand storage training complexity, and a low test complexity that is irrelevant to the trainingset size, but also can optimize diferent objectives.2. The novel Weight Optimization and Layered-Clustering based Error CorrectingOutput Code (WOLC-ECOC) was proposed. Specifically, the WOLC-ECOC containstwo novel techniques. The first one is that the layered-clustering based ECOC was pro-posed so as to inherit the advantages of the subclass technique of the state-of-the-artsubclass-ECOC, and meanwhile to prevent the drawback of the decision tree based sub-class splitting. The second one is that the optimized weighted decoding was proposed,such that if any binary-class classifier is added to the ECOC classifier ensemble, thetraining risk of the ECOC ensemble will be guaranteed to be non-increasing. Finally, theWOLC-ECOC wraps the layered-clustering based approach and the optimized weighteddecoding until the training risk converges.3. The linear time Sparse Kernel Maximum Margin Clustering (SKMMC) was pro-posed. Specifically, the SKMMC is an improved version of the advanced Cutting-PlaneMMC (CPMMC). It first introduces a adapted threshold to the CPMMC, which makesthe CPMMC insensitive to the parameter selection. Then, it employs the sparse kernelestimation techniques to the MMC, so that the MMC can have a linear time complexityin nonlinear kernel scenarios. 4. The linearithmic sparse and convex Support Vector Regression based MMC(SVR-MMC) was proposed. The SVR-MMC was further extended to the SVR basedMultiple Kernel Clustering (SVR-MKC) and the SVR based Multiclass MMC (SVR-M3C). Specifically, the SVR-MMC first introduces the SVR to the MMC so as to preventan integer matrix programming problem. Then, it constructs a convex hull on the origi-nal MMC problem so as to relax the original problem to a convex optimization problem,which has a global optimized solution. Moreover, several efcient algorithms was pro-posed to solve the convex optimization problem. Finally, the MMC problem can be solvedglobally in a linearithmic time complexity and a linear time storage complexity with bothlinear and nonlinear kernels. The two extensions–SVR-MKC and SVR-M3C inherit allmerits and advantages of the SVR-MMC.5. Several above theoretical contributions are applied to the areas of the audio signalprocessing, such as the Voice Activity Detection (VAD) and the music classification. Inthe respect of the VAD, this thesis finally proposed a novel efcient VAD framework,two new acoustic features that are based on the statistical signal processing, and an SVR-MKC based VAD that can fuse multiple features in the kernel level. In the respect of themusic classification, this thesis proposed to use the WOLC-ECOC for the music genreclassification.