Research And Application Of Rough Set In Data Fusion

With the rapid development of network,data fusion has become an important research hotspot.In the process of data fusion,a large amount of real data needs to be processed.There are often noise and missing data in real data.In practical application,the features of data sets can be selected to reduce the amount of data that we need to processing.Rough set theory has two important applications:feature selection and rule induction.Feature selection preprocess the data set for the data fusion,so as to reduce the amount of data,eliminate redundant features,and facilitate processing;Rule induction provides a unified form for data sets of different forms,thus facilitating information fusion of data sets of different types and sources.In this dissertation,the existing rough set feature selection algorithm and rule induction algorithm are investigated,and the following works are completed for their optimization:1.a C-FRFS algorithm based on fuzzy clustering and fuzzy rough feature selection is proposed.Based on fuzzy c-means clustering method to design a new method to generate the membership function,and based on this method to design a new calculation method of fuzzy relations.The algorithm only using the knowledge of the data set itself contains.Membership functions are automatically formed without a user-defined threshold.The experiment shows that this method has good performance compared with other algorithms.2.we design a hybrid mode of fuzzy rough feature selection and rule induction algorithm.This method based on C-FRFS,and we added rules induction extra.We made feature selection and rule induction algorithm into a single integrated method to generate a concise,meaningful and accurate rules.The redundancy calculation is reduced and the complexity of the algorithm is reduced.3.considering that the rough set algorithm performance bad on the big data,Spark framework is to handle huge amounts of data.The feature selection algorithm is adapted to the distributed operation through RDD and Spark operator,which can improve the operation efficiency.The algorithm runs successfully in both local and standalone modes of Spark,providing the foundation for big data fusion.In this dissertation,the application of rough set in data fusion is studied in detail,and the algorithm of feature selection and rule induction of rough set is optimized.Experiments show the effectiveness of the algorithm.