| Research on Imbalanced Big Data Problems is one of the difficulties in the field of data mining. The common data mining algorithms can achieve good results on small data sets. However, The efficiency of common data mining algorithms reduced greatly when deal with imbalanced big data. The traditional algorithms are not work in imbalanced big data with high time complexity and memory-limit. actually, the real-world are always imbalanced data sets which some classes are less than the other classes. The minority class is the positive class and the majority class is the negative class. The big data and imbalance as are as two separate problem to study in the existing research. Because of the scale of real data is big, it will be have the problem of excessively data fitting when deal with large-scale data set. The classification results would be reduce the classification precision when the minority classes was ignored. So the imbalanced characteristics of data sets must be considered.This paper introduced the concept of imbalanced data sets and big data, the research progress in the field of imbalance and big data of all domestic and foreign experts and, learn the classification methods in dealing with imbalanced big data and the parallel processing framework of MapReduce. Introduce some commonly used feature selection methods and decision tree method. We proposed a decision tree classification method based on k-means and artificial colony algorithm. The experimental results show that have a better classification in liver function diagnosis.For the imbalanced big data classification problems, this paper presents a parallel sample selection method based on MapReduce which solve the problem with the secondary parallel pattern. The experimental results show that the proposed approach can effectively improve the classification accuracy. |
