Research On The Method Of SVM Hyperparameters Tuning And Application In Face Recognition

Support vector machine (SVM) is widely used in pattern recognition as a powerful classification tool. The basic idea of SVM is to map the input space into a high dimensional Hilbert feature space, in wicth the linear non-separable problem can be transformed to linear separable problems. The kernel functions play a very important role, and the different kernel functions implicitly define the nonlinear mapping from input space to the optimized Hilbert feature space, thus, the optimal hyperplane and the classification results are different. Therefore, the choice of hyperparameters which include the kernel parameters and penalty parameter can directly affect the SVM generalization capability. However, the SVM hyperparameters tuning is the bottleneck of applying SVM to pattern recognition and machine learning due to lack of theoretical analysis.SVM is a classification tool and it has been successfully applied to face recognition, however, it is known that face recognition is generally restricted by illumination and the posture. Research shows that the SVM model with the optimal hyperparameters, to some extent, can suppress the influence of the environmental change for face recognition. And the main difficulty of SVM hyperparameters tuning in practical problems is the efficiency of hyperparameters tuning, especially in high dimensionality and multiple face classification problems. Therefore, this thesis has researched into the method of hyperparameters tuning, and proposed optimization method of hyperparameters tuning for face recognition.This thesis proposed a SVM hyperparameters tuning method based on uniform design (UD) for face recognition. The method replaces the grid and gradient descent method with UD for selecting some features that spread uniformly in the whole searching region and can gain satisfying testing results. The optimum SVM hyperparameters can be obtained by UD for minimizing k-fold cross validation error bound or leave one-out (LOO) error bound.SVM hyperparameters tuning via uniform design can effectively alleviate the computational cost, but its drawback is that it adopts a single objective criterion which can not always guarantee the generalization capacity. Therefore, this thesis further proposed a Multi-Objective Uniform Design (MOUD) optimization algorithm as a SVM hyperparameters tuning tool. The sensitivity and specificity as multi-objective criteria have been proved of better performance and can provide a means for obtaining more realistic models. Because of replacing single objective criterion with multi-objective criteria and adopting uniform design to seek experimental points that uniformly scatter on whole experimental domain, MOUD can reduce the computational cost and improve the classification ability simultaneously. The verifying experiments are executed on UCI machine learning repository and real face databases. The experimental results support the evidence that the proposed method significantly improves the efficiency of SVM hyperparameters tuning in face recognition, which reveals its practicability in real face databases.