Research On Multiple Kernel Boosting Learning Algorithm

In recent years,Multiple Kernel Learning(MKL)has been widely concerned in the machine learning field,and MKL is a promising method for data mining tasks.It mainly learns kernel-based models by finding the optimal combination of multiple predefined kernels for the challenging classification tasks,such as heterogeneous information or unnormalised data,non-flat distribution of samples,etc.Conventional MKL methods often formulate the problem as an optimization task of learning the optimal combinations both kernels and classifiers,which usually results in some forms of challenging optimization tasks that are difficult to be solved.The critical limitations of conventional MKL methods are large computation and instable.Hao Xia and Steven Hoi proposed the framework of multiple kernel boosting,which adopted boosting to solve variant of MKL problems and avoided solving the complicated optimization tasks.However,it is well known that AdaBoost algorithm is sensitive to noisy data,namely,MKBoost algorithm can't deal with the noisy data sets effectively,and the robustness is poor.To overcome the overfitting problem when regular MKBoost handle noisy data sets,we present two novel MKBoost algorithms,namely noise-detection based MKBoost(ND-MKB)and noise-probability based MKBoost algorithms(NP-MKB).There are mainly two aspects of our methods.Firstly,Noise Recognition.According to the neighborhood information of the instance,the k nearest-neighbor method is exploited to detect the noisy instances.The bigger the number of wrongly classified neighborhood instances is,the instance is more like noise data,vice versa.Binary processing is applied to ND-MKB algorithm,namely {-1,+1},the logistic regression model is used to map the detecting results to the interval [0,1] in NP-MKB algorithm,and it is the final probability.Secondly,Loss Function.The loss function and weight updating of AdaBoost only focus on the classification results.It is necessary to distinguish between noisy instances and normal instances.Based on the noise-detection function and noise-probability function,we propose two new loss functions,which take into account the influence of the noise data,and then forward stepwise algorithm is exploited to algorithm derivation.Experimental results show that both ND-MKB and NP-MKB methods has a better robustness than traditional MKBoost method.