Research On Cost-Interval-Sensitive Optimal Margin Distribution Learning

In many machine learning applications,different types of misclassifications usu-ally suffer from different cost.To over come this problem,researchers have proposed many cost-sensitive learning approaches to minimize total costs.These approaches assume that the misclassification cost is a known fixed value.However,the accurate cost is often hard to determined,and usually one can only gets an interval estimate.In the meanwhile,recent theoretical results disclosed that the margin distribution,rather than the minimum margin,was more crucial to the generalization performance.To op-timize the margin distribution in cost-interval-sensitive learning,we propose several approaches for different situations.The main contributions can be list as follows:1.For traditional cost-sensitive setting given accurate costs,we propose the large mar-gin distribution machine(CSLDM)with weighted loss function to minimize the total cost.Experiments on a broad range of datasets and cost settings imply the effectiveness of optimal margin distribution learning in cost-sensitive setting.2.For cost-interval-sensitive setting with abundant unlabeled data,we propose the cost interval semi-supervised large margin distribution machine(cisLDM)to handle cost interval and exploit unlabeled data in a principled way.Specifically,cisLDM minimizes the worst-case total-cost and the mean total-cost simultaneously,and tries to optimize the margin distribution on both labeled and unlabeled data rather than maximizing the minimum margin.Experiments on a broad range of datasets and cost interval settings shows that cisLDM utilizes unlabeled data in cost-interval-sensitive learning effectively.3.To facilitate the difficult objective and grid search based parameter selection in cisLDM,we propose the cost interval sensitive optimal margin distribution ma-chine(CIODM).CIODM combines differential optimization with non-differential optimization instead of the grid search based parameter selection to minimize the worst-case total-cost and the mean total-cost.Experiments on a broad range of datasets and cost interval settings exhibit the impressive performance of CIODM.