Extrinsic regularization in parameter optimization for support vector machines

A heuristic is proposed to optimize free parameter selection for Support Vector Machines, with the goals of improving generalization performance and providing greater insensitivity to training set selection. The main points of the proposed heuristic are the inclusion of extrinsic regularization to improve generalization error; the use of simulated annealing to improve parameter search efficiency in comparison to an exhaustive grid search; and an intensity-weighted centre of mass of the most optimum points to reduce solution volatility. Two standard classification problems are examined for comparison, and the heuristic is applied to protein sequence alignment quality and retinal electrophysiology classification. The heuristic is extended to univariate and multivariate regression problems, examining environmental modelling and periodic gene expression. Input variable selection and sensitivity are explored to determine the most significant segments of the electroretinography waveforms.