Research And Implementation Of An Information Pre-process Platform Of Public Opinion

Along with the continuous development of the trend of network Information, the way and speed of public opinion transmission has undergone tremendous changes, the Internet has become a major public opinion gathering place, in recent years, and kept affecting the people's lives.Therefore, it's helpful to understand the public attitude and opinion through the network and it's important to the development of economy, society and politics.However, with the characteristics of large amount, semi-structured and complexity, the data of the network public opinion makes people encountered great difficulties in relevant information collection and researching. Therefore, it is imperative to build a network information pre-processing platform to arrange the web data.Firstly, this paper analyzed and researched the relevant technology of domestic and international,summarized the advantages and disadvantages, and in-depth analysis of the problems in design and implementation of technology and other problems of network public opinion information pre-processing platform, and finally proposed a set of scheme for the information pre-process of public opinions for. This scheme can achieve the goal to process the massive web information, analysis and reuse features.The main contribution and achievement of this paper are as follows:Analyzed the structure of URL in-depth, Used URL comparison to process massive web pages, completed filtering the website which by the users;Proposed Web information extraction Algorithm based Document object model and designed Analysis page template library which based on extraction rules, filtered the irrelevant information and put the key information into the database for future use. Proposed a scheme of segment dictionary based on four-character mechanism completed segmentation of Chinese words and word frequency accurately; Proposed a scheme to make use of thread pool to manage multi-thread technology solutions, and the system efficiency had been improved a lot; The information process scheme had been verified by experiment, the results showed that the design of this scheme efficient and feasible, it had a high accuracy and value in use.Based on the above work, and according to the general principles of platform design, this paper planed overall framework of the Public Opinions Pre-process Platform,completed module division of the platform and devised the database structure and functions of each module. Finally, a stable and high-efficiency pre-process platform was established.I hope it can make a modest contribution through this thesis research to the analysis of theoretical studies of public opinion processing.