Margin Distribution Logistic Machine With Its Expanding Researches

In the era of artificial intelligence,as the key part of AI,machine learning is playing an essential role in massive real world applications.Because of its simplicity and applicability,classification is always a hot topic in machine learning research.Linear classifier is the foundation of classification algorithms,it has been successfully applied in applications such as spam classification and document classification especially for its efficiency and flexibility.Logistic regression is a representative linear model,logistic regression is welcomed in large scale classification and researches like medical with its scalability and probabilistic output,so many researches focus on further improving the capability of logistic regression.The main contents of this paper is listed as the followings:In the first part,this paper designs a robust Margin Distribution Logistic Machine with generalization ability by introducing margin distribution optimization to a functional margin fixed generalized logistic loss.Furthermore,aiming at more general multi-class classification tasks,based on Margin Distribution Logistic Machine,this paper gives a multi-class classification framework with a structural sparsity constraint to help the model utilize the shared information across different classes,and this framework can also adapt to do binary classification and feature selection tasks.The corresponding optimization algorithm is also given in the paper.Besides,solutions to non-linear classification and large scale learning are designed for Margin Distribution Logistic Machine to further improving its adaptability.A set of experiments are designed to validate the property of this model.In the second part,because ensemble learning is a good approach for a base learner to improve the capability in many real world problems,it is necessary for a base learner to extend to ensemble model.For Margin Distribution Logistic Machine and Random Subspace Ensemble,because the existence of noise features which may make a redundant random subspace set,the ensemble model's performance could be obviously reduced.So in this paper,an ensemble pruning technique based on clustering is proposed named Diverse Random Subspace Ensemble,the pruning technique is realized with the help of a subspace diversity metric.The experimental results show that Diverse Random Subspace Ensemble is much suitable for data with high feature-sample ratio,and the performance is better when Margin Distribution Logistic Machine is chosen as the base learner.In general,this paper proposed a robust classification model which incorporate generalized logistic loss with margin distribution,and did some more expanding researches under various applications.