Research On Attribute Reduction Method Based On Map Reduce-based Improved Discrete Glowworm Swarm Algorithm And Multi-fractal Dimension

In the field of machine learning and data mining,attribute reduction is a key step in data preprocessing.Attribute reduction reduces the noise attributes of the original data,which can achieve the goal of dimension reduction while retaining the characteristics of the original data.To address the attribute reduction problems,multifractal dimension is used as the evaluation metric,and glowworm swarm algorithm is taken as the searching strategy in this paper.In view of the massive and complex data set under big data environment,this paper adopts MapReduce parallel programming model to improve the efficiency of the method which can realize the parallelization of the proposed attribute reduction method.The research content is as follows:(1)The Attribute Reduction Method based on MapReduce-based Improved Discrete Glowworm Swarm Algorithm and Multi-fractal Dimension is proposed.First of all,the moving way of glowworm individual’s updated method is processed by discretization,to avoid the algorithm running into the local optimality,a migration strategy and Gaussian mutation strategy are introduced.And then an improved discrete glowworm swarm algorithm(IDGSO)is proposed.Secondly,the improved discrete glowworm algorithm combined with multi-fractal Dimension(MFD)is applied to the field of attribute reduction.Finally,to address the problem of attribute reduction under big data environment,the MapReduce programming model is adopted to realize the parallelization of IDGSO and MFD.Experiments on the UCI datasets and the real meteorological datasets show that the proposed approach can achieve good results with respect to reduction and operation efficiency,and that its effectiveness and feasibility.(2)MR-IDGSO method proposed is applied to the field of haze prediction.MR-IDGSO method is used to reduce the attributes of haze datasets,and the key arributes are filtered to provide effective data for haze prediction.Then SVM is taken to verify the classification accuracy of the key arributes.Experimental results on the haze datasets of Beijing,Shanghai and Guangzhou demonstrate the effectiveness and credibility of the proposed method in the field of haze prediction.