Design And Implementation Of Network Public Opinion Information Collection System

Public opinion is in a certain social space, with the occurrence of an event, development, and changes in public attitude towards the incident. Public opinion is the opinion of the enlarged body, which the sum of people's views on social phenomena, ideas, attitudes and opinions, but also the part of people's attitudes to social and political that in power to influence and guide decision making. Popularity of the network makes the mechanism of public opinion changing a lot. Public opinion is disseminating and changing in the network may eventually form a network of public opinion and may even affect the political system management, network public opinion has become one of the most important public opinion, which is the social network of public opinion reflected.With the development of the Internet, after newspaper, radio and television media network become to the three most important traditional media. As its special role in the emerging network, public opinion is becoming the birthplace of public opinion and the amplifier. How to correctly guide the network public opinion, to avoid bad posture spread, which gave the ruling party and government unprecedented challenges. Effective monitoring of network public opinion and reasonable guide people on hot social issues, which urgent need to improve the network the ability to monitor public opinion. In order to internet public opinion information for effective control, First will obtain important network media release information, i.e. on the Internet for public opinion information released information collection. However, due to various in form, the network information of various network public opinion sources of information are also different, and involves of information are enormous, traditional collect and analyze the mechanism is difficult to achieve, and therefore must be collection work of public opinion to construct a efficient public opinion to collect information system to have finished the work.It is based on this background which paper will theory research and empirical study unifies, first studying network structure and characteristics and combining the actual situation of domestic Internet application. According to network public opinion information collection source of information is mainly coming from big BBS, online community, public blog and so on which users can focus reflects for a specific issue.Then study and compare the current suitable for network public opinion information collected various theories of directional technology and information collection scheme, Puts forward through the universal search engine and web crawlers combination of network public opinion information collection model, to meet the various levels of network public opinion's acquisition needs. In order to guarantee the real-time and reduce data redundancy, studies the web crawlers search strategies, to access strategies and politeness strategy, proposed by taking regular expressions filter way, eliminated web crawlers crawling process to meet the requirements of the URL, prevent the system from deviate a target site and collecting redundant data.Finally, this article from the network information collection and analysis of early requirement, developed network public opinion information acquisition system, realizes from URL grab, page source grab, title and text extraction, web page to heavy and so on a series of workflow, continuing to network for future public opinion information analysis and processing laid a foundation.