Research On Multiple Birth Support Vector Machines And Optimization Methods

Twin support vector machine is a useful extension of the traditional support vector machine.For the binary classification problem,the basic idea of TWSVM is to seek two nonparallel hyperplanes such that each hyperplane is closer to one and is at least one distance from the other.As an emerging machine learning method,TWSVM has attracted the attention of scholars and becomes a hotspot in machine learning.Twin support vector machine is first proposed to solve the two classification problem.However,in practical problems,most of them are multi-classification.Therefore,researchers applied twin support vector machines to multi-classification,and formed a variety of multi-classification twin support vector machines.Multiple birth support vector machine is a new multi-classification twin support vector machine which has been recently proposed.The decision criterion in MBSVM is the farthest distance of the test pattern to the hyperplanes instead of the closest distance in TWSVM.Compared with other multi-class TWSVMs,it has the characteristics of lower computational complexity and can be expected to be faster than the existing multi-class TWSVMs.However,MBSVM fails to take full account of the sequence-related information of sample features,and its parameters are usually selected by experience,so it can not automatically select parameters.Meanwhile,the loss function of MBSVM can be further optimized to improve the performance of the model.In this paper,we start with multiple birth support vector machine,combined with recurrent neural network and a new loss function,and use optimization algorithm to optimize the parameters of MBSVM,so as to improve the performance of MBSVM.The main contents of this paper are as follows:1.Research on multiple birth support vector machine based on recurrent neural networks.Multiple birth support vector machine does not take into account the correlation sequence information among all dimensions of the samples,which limits the further improvement of the classification accuracy.For the above problem,this paper presents several variants of MBSVM algorithms: multiple birth support vector machine with multilayer perceptron,multiple birth support vector machine with longshort term memory networks,multiple birth support vector machine with multilayer perceptron and long-short term memory networks.After introducing Multilayer Perceptron and Long-Short Term Memory Networks,these algorithms can take full account of the sequence correlation information between different features of samples,and play a role of dimensionality reduction to some extent.Finally,experiments based on UCI benchmark datasets show that the algorithms proposed in this paper are effective,and they can greatly improve the classification accuracy of multiple birth support vector machines.2.Research on multiple birth support vector machine based on triplet loss function.When dealing with the classification problem of unbalanced data,especially when the number of classes is incredible large,multiple birth support vector machine which adopts hinge loss and easily leads to instability for resampling,which affects the accuracy of classification.In order to solve the classification problem of unbalanced data and improve the classification accuracy of the algorithm,this paper applies triplet loss function to MBSVM,and proposes the MBSVM based on triplet loss.Multiple birth support vector machine based on triplet loss controls the distance between positive and negative samples by setting the threshold of triplet loss,which can make full use of the difference information between positive and negative data,and better handle the sampling problem of unbalanced data.Experiments on UCI datasets show that the proposed algorithm can effectively solve the classification problem of unbalanced datasets and improve the classification accuracy.3.Research on triplet loss multiple birth support vector machine based on dynamic multiple particle swarm optimization.Traditional particle swarm optimization searches for the optimal value with a single particle swarm,and can not change the weight in each iteration.In this paper,the particle swarm optimization algorithm is reconstructed by increasing the number of population and dynamically changing the weight coefficients.A new multiple PSO algorithm is proposed.Compared with the original particle swarm optimization algorithm,this method has better optimization effect on the objective function with more parameters,can reduce the impact of local minimum to a certain extent,and can quickly find the optimal value.The dynamic multiple particle swarm optimization algorithm(DMPSO)proposed in this paper prevents the oscillation of the optimal fitness by changing the coefficients in each iteration process,so as to obtain better accuracy.Furthermore,dynamic multiple particle swarm optimization algorithm is applied to triplet loss multiple birth support vector machine,and the optimal hyper-parameters of triplet loss multiple birth support vector machine are searched by dynamic multiple particle swarm optimization algorithm,which further improves the performance of triplet loss multiple support vector machine.Finally,the performance of DMPSO and triplet loss multiple birth support vector machine based on dynamic multiple particle swarm optimization is tested by experiments.Experiments show that the accuracy of DMPSO algorithm is better than that of traditional PSO.Moreover,the experimental results on UCI datasets show that the proposed algorithm can efficiently enhance the classification accuracy.