Incremental Learning Algorithm For Large Data

Along with the rapid growth of data size, able to handle large data sets of data mining algorithmhas been widely research and application, and become one of the hot spot of current research. Thispaper mainly studied the big data oriented incremental feature selection and classification ofincremental learning algorithm. Paper main research work has the following two contents:(1) with conditional mutual information as the standard for measuring based on incrementalfeature selection algorithm, through the analysis of large data simulation data flow is divided intodata blocks, incremental information measurement was carried out on the feature subset, improve theoperation efficiency, eventually get the feature subset. In order to verify the effectiveness of theimproved incremental feature selection algorithm, simulation experiments on the UCI data setscompared classification performance. Experiments show that the incremental feature selectionalgorithm in most cases (I-MIFS) are better than other algorithms, I-MIFS algorithm is a kind offeature selection algorithm is suitable for large-scale data set.(2) based on neural network ensemble study incremental learning algorithm: big data is studiedby using improved Boosting technology to complete the formation of the individual neural networkintegration, and the final result will be classified boundary fault samples as the research target, theintegration of neural network can be big data incremental learning, through the design experiment,experiments with UCI data sets, comparison and analysis can get big data incremental learningalgorithm is effective and feasible experimental results. Research based on the improved Learn++algorithm, neural network has the big data incremental learning ability, have solved the problem ofunbalanced category.