| Under the background of the information explosion,data mining are often faced with a series of challenges whendealing with the large amount of data.Too many samples and high dimension of data are the only two reasons why the data is so big.However,because of the value of the sample resources,they are usually not deleted.Accordingly,reducing the dimension of data will be taken into account if there would not be serious impact on the results,which is also a necessary step for data preprocessing before data mining.Attribute reduction based on rough set is a common dimension reduction method.It can reduce the dimension of data effectively with no need of any additional information or people's prior knowledge.For example,different thresholds may bring about totally different results.The attribute reduction method generally need to traverse each attribute to determine whether it is redundant.As a result,when dealing with high-dimensional data,there existsa high computational complexity problem.To solve this problem,the characteristics of high dimensional data are fully studied.Then,wepropose two approaches from two aspects of hardware acceleration and modifyingexisting reduction process.The parallel positive domain computation method and the new reduction process are proposed.How to improve the efficiency of rough setreduction algorithm to deal with high dimensional data is a core problem in this paper.The main contributions in this paper can be summarized as follows:1.Aiming at the reduction algorithm based on the positive domain,the existing algorithms are optimized from two angles of improving the speed of the positive region and modifyingthe strategy of reduction process.Firstly,a method for calculating the positive domain in parallel is proposed by using multi-process technology.Next,a binary search attribute reduction algorithm based on binary search is given.It can quickly obtain approximate reduction and then get the final reduction by combining with the original method.At last,the granular computing is introduced to improve the attribute reduction method.The definition of multi-granular attribute tree is given,and a preorder traversal attribute reduction algorithm is designed based on it.The experimental results show that using the proposed method can get the reduction results faster than the existing method.2.To verify the practicability of the proposed method in this paper,a news classification system is developed on using preorder traversal attribute reduction algorithm.The news classification system includes four processes: news collection,data preprocessing,news classification and news presentation.Generally,news corpus often gets a high-dimensional decision information table after word segmentation.Among them,because of the high dimension,the process of data preprocessing is slow,and the time of each news classification is a little long.Therefore,using the preorder traversal attribute reduction algorithm to reduce dimensions of a high-dimensional news decision table can decrease the cost of computation and improve the efficiency of the system. |
