Research And Design Of Public Opinion Collection System Based On Meta Search Engine

With the rapid development of the Internet, "user-centric, user participation" open architecture concept has been popular with people, Internet users gradually from passively receiving network information to actively creating Internet information. Portals, forums, microblog and other network media has become an important platform to publish, disseminate information, get comment, express emotions, and express their views. At the same time, The Internet has become an important channel for public information dissemination. Subjective text such as a large number of comments and views in network media that hold the important economic and social values, also guide the direction of public opinion, generate network public opinion information. Network public opinion is a collection of all people’s cognitions, attitudes, emotions and behavior tendency of the event that generated by stimulation of various events, spread by the Internet. Mapping the social public opinion in the Internet space, and is a direct reflection of social public opinion. The gathering and monitoring to network public opinion has an important role. That timely master the emotional tendencies of users, identify and track the network hot events.Therefore, many research institutions, social enterprises and even government agencies are in the network public opinion has been a lot of research and analysis, in order to carry on the monitoring, and make use of it, that hold great practical significance.This paper based on the public opinion network monitoring platform of University of South China, designing and developing the network public opinion information collection system. With the approach of theoretical researching to guide practice to develop the system. First, studied the structure and characteristics of the network public opinion, analyzed the main gathering space and source of public opinion, combined with the current development status of domestic and foreign public opinion researching. Aim to the Widely existed problem of public opinion like efficiency is not high, the limitations of target is strong, determined a personalized public opinion collect strategy based on user’s theme settings. By the way of topic keyword matching, regular expressions filtering, and domain-based crawling. To ensure the system crawling data on topic relevance, filtering redundant data. The sources of public opinion were set the major news portals, blog forums, online communities, and microblog, to reflect the views of the public focus on emerging media attitudes and opinions, tendencies.This paper aims to develop a network public opinion monitoring system that adapted to university and provide source of public opinion information. In real time and efficient digging out the Internet public opinion with respect to sensitive information university, prepared for cleaning and structured the collected data, also for the tendency of public opinion data analysis, finding hot events and tracking event. Main results achieved are:(1)Research specific for structural characteristics of the network public opinion, collecting sources. Combined with existing technologies and models of public opinion gathering research system in domestic and foreign, according to the actual needs of University of South China Network public opinion monitoring platform, analyzed and designed the system.(2) Parsed the public opinion Web pages from different sources, analyzed the importance of different labels of Web pages, extract the relevant elements of public opinion;(3) Implemented the theme for public opinion and public opinion gathering sources can be configured, the user crawling set the sources of public opinion information which based on the theme of keywords and domain information, to archive the personalized public opinion collection;(4) Analyzed the network public opinion crawling strategy, data crawling based on meta search engine,using multi-thread parallelism, to achieve real-time and efficient crawling;(5) Achieved overall system architecture with open source SSH JAVA framework, the application is divided into the presentation, control, business logic and data access four layers, reduce the coupling between the layers. Realized the development and test of the system, ready for the subsequent work.