| The wide spread of Internet and the increasing number of internet users make the Internet become the important way of people to express emotions and get informations. Internet public opinion has become the important part of social public opinion, influencing the social events. In recent years, the Internet public opinion events make the government departments, enterprises, universities, and even individuals aware of the importance of the network public opinion informations. Therefore, the public opinion mining system has the vital significance.Small users such as small and medium-sized enterprises, universities and individuals demand for public opinion mining system. But there is no public opinion mining system particular for small users on the market. Therefore, this paper designs and implements a public opinion mining system serving for the small users. First analyze the small users’ requirements such as network bandwidth, hardware equipment and public opinion response rate and the technical difficulties of public opinion mining system below these requirements. Then design the information collection system, the information processing system and the information analysis system more suitable for small users, implement and put to practical use.There are three parts of design and implementation of the public opinion mining system in this paper:The first part introduces the related technology to design and implement the public opinion mining system, including the crawler technology and URLs reduplication removing technology in the information collection system, the text extraction technology in the information processing system and the Chinese word segmentation technology in the information analysis system Then offer the contrast of these different implementation of the related technology to build a theoretical foundation for the latter design and implementation.According to the characteristics of small users, the second part considers the optimization of network bandwidth, hardware performance and public opinion reaction speed, designs the whole architecture of the public opinion mining system. Then designs the detail of three core subsystems. In the information collection system, design the detail of the crawler system, including the collection target selection, the dynamic update technology of collection frequency, the latest URLs reduplication removing method. In the information processing system, this paper proposes a forum home page segmentation method to extract the new article links from the forum home page, the forum text extraction method based on pattern recognition to extract forum web page text and the method to extract the posting time. In the information analysis system, distribute system time properly, choose the accurate and efficient word segmentation method to provide the convenience for correlation calculation and abstract extraction.The third part introduces the choice of the development environment and the open source tools to implement the public opinion system, expounds the implementation of the public opinion mining system and shows the result.In this paper, some technical solutions are original designed, and the public opinion mining system meet the needs of small users, having practical significance. |
