Font Size: a A A

Parallel Multi-label Classifier Chains Algorithm Using Apache Spark

Posted on:2019-06-15Degree:MasterType:Thesis
Country:ChinaCandidate:H WangFull Text:PDF
GTID:2428330590965795Subject:Computer technology
Abstract/Summary:
With the development of information technology,multi-label classification plays a more and more important role in the field of data mining.Different from the traditional single-label classification,multi-label samples are associated with several labels simultaneously.Obviously,mining and utilizing the correlations between labels can effectively improve the classification accuracy.Classifier chains is a widely known multi-label classification algorithm.Its structure is so simple that easy to implement,and it is not restricted to single-label classifiers which can adapt to appropriate classifiers according to specific application scenarios.Compared with other classic multi-label methods,it has achieved better performance because of considering the high-order correlation between labels.However,the initial order of chain itself clearly has an effect on accuracy,and it is not stable which chain is randomly generated with taking label correlation into account.The scale of multi-label dataset becomes bigger and bigger in the era of big data.The current serial multi-label classification algorithm can hardly finish task in time and even sacrifice accuracy.Therefore,cc is no exception.In view of the above problems,the main contributions of this thesis are as follows:1.In order to overcome the shortcomings of the CC,a clustering method is proposed in this paper.An improved cosine similarity method was used to cluster label vectors which capture the positive and negative correlation.Moreover,strongly related labels can be casted into same clusters and it requires less computation.And on this basis a novel method called CLCC is proposed in this paper.A complete classifier chain is transformed into a number of classifier chain where the chain covers subset of label space.What's more,the order of chain is determined by the distance between the label and cluster center.The validity of the proposed method is presented by rich experiments on 13 multi-label datasets from different fields and 3 different base classifiers in this paper and the time complexity of CLCC does not increase significantly.Compared with the existing improved algorithm,CLCC also has advantages on classification accuracy,especially efficiency.2.The existing traditional multi-label classification methods are usually unable to get the results on large-scale datasets in time.Combined with the characteristics of CLCC,this paper proposes an algorithm under the parallel computing framework Spark named SLCC to solve above problem.Parallel experiments on 6 large-scale multi-label datasets show that SCLCC has better speedup and scalability,especially for super large-scale dataset.
Keywords/Search Tags:multi-label classification, classifier chains, label correlations, parallel, Apache Spark
Related items