The Analysis Technology Of Grain Information Based On Text Mining And System Implementation

Grain is both important strategic resource to keep social harmony, regime stability and financial sustained development, and the people’s basic necessities. With the quick prosperity of web, currently, the grain information grows rapidly. How to search what we want from the mass of grain information and deal with the large amount of it efficiently and accurately in order to obtain relevant grain information have important meaning on ensuring grain security and realizing the management and utilization of grain scientifically. Therefore, the research of grain information analysis technology is an important area of grain information.This paper has systematically studied relevant grain information analysis technology. And on the basis of that, it has focused on the text feature selection, automatic summary technologies and other relevant technologies.(1) Considering the lack in the feature selection, this paper presents improved chi-square statistic algorithm based on CF-EF-RC. Based on the analysis of variance, this algorithm is joined into the characteristic frequency, balancing factor between and within documents, and a restraint mechanism so to improve the rationality of the text feature selection. The experiment result has proved the effectiveness of the improved algorithm.(2) This paper presents an algorithm named SC-RP based on sentence coherence about redundancy processing based on co-occurrence of words. This method regards the sentences as chains. Calculate the sentences’ coherence. Remove redundant sentences with a bigger coherence. In this way, we can improve the rationality of automatic summary in the end. The experiment result has shown that the proposed algorithm can reduce the redundancy of the text summary and improve the precision and simplicity of the summary.(3) This paper builds a basic framework of the grain information analysis system, including text information acquisition, text information pretreatment and automatic summary module and so on. This paper has instructed the work flow and operating mode of each module in particular, and achieved its basic functions.