The Effects Of Support Vector Machine Parameter Tuning On Classification Accuracies Based On Brain Imaging Data

Background and Objective:Neuroimaging techniques are widely used for studying the human brain and there exist many different approaches for analyzing brain imaging data.In recent years,machine learning techniques,especially the multivariate pattern analysis（MVPA）approach,have become increasingly popular in this field due to its power in information detection from brain imaging data.However,it remains unclear how the parameters of machine learning algorithms should be determined when applying these techniques to brain imaging data analysis.In fact,most reseachers simply use the default values set by the used softwares.To better understand how changing the parameters in the machine learning algorithms would affect the classification results based on brain imaging data,we used support vector machine（SVM）to classify between different populations or between different experimental conditions based on structural or functional measurements extracted from magnetic resonance imaging（MRI）data and tested how changing several parameters in this process would affect the classification accuracy.The parameters that were modulated in the present study include:the size of spatial smoothing,the type of the SVM kernel,the type of penalty,and the penalty factor.Subjects and methods:To examine whether the parameter tuning would affect the classification results differently for different types of brain imaging data,two different datasets were used in this study:1、Schizophrenia data:including both structural and functional MRI data collected from 110 patients diagnosed with schizophrenia and 110 gender-and age-matched normal subjects.All parcitipants gave informed consent forms prior to their participation.The study was approved by the Ethics Committee of Tianjin Medical University General Hospital.DPASFA software was used to preprocess the fMRI data using standard procedure,and to extract ReHo and ALFF values.VBM8 software was used to extract gray matter volume from structural MRI data.SPM8 software was used to spatially smooth the data（Gaussian kernels with four different FWHM of 5mm,8mm,15mm and 20mm were used）.Finally,support vector machine（SVM）implemented in the MVPA software（an inhouse software package developed in Matlab）was used to classify between patients and controls based on these brain imaging data and different parameter values were used to test how the classification accuracy changes with the change of these parameter values.2、Pain and touch data:fMRI data collected from 50 healthy volunteers during pain and touch stimulations.All parcitipants gave informed consent forms prior to their participation.The study was approved by the Ethics Committee of Tianjin Medical University General Hospital.FMRI data were first preprocessed using standard procedure（image realignment and spatial normalization）,and the volumes collected at the 7^th-10^th time points after each stimulus onset were averaged and then spatially smoothed using Gaussian kernels with four different FWHM of 5mm,8mm,15mm and 20mm.Finally,SVM was used to classify between pain condition and touch condition based on these brain imaging data and different parameter values were used to test how the classification accuracy changes with the change of these parameter values.Results:We obtained the following results:1.SVM kernel type has considerable impact on the classification accuracy;in most cases,linear kernel（i.e.,linear SVM）outperformed other non-linear kernels;2.The penalty factor C in C-SVM or n in n-SVM has relatively small effect on the classification accuracy;3.Similar classification accuracies were obtained by C-SVM and n-SVM;4.In general,the classification accuracy decreases with the increase of the spatial smoothing kernel size;5.The above effects of parameter tuning on classification accuracy were similar for classifications based on structural and functional measurements.Conclusions:1.Linear SVM is the first option when using SVM on brain imaging data;2.Selecting the default values of the penalty factors（C=1 for C-SVM or n=0.5for n-SVM）are good options in most cases;3.C-SVM and n-SVM give similar results and thus either is good to use in practice;4.Large spatial smoothing kernels are not recommended when preprocessing the brain imaging data before MVPA;5.The above conclusions apply to both structural-data-based and functional-data-based classification.