Research And Development Of A Collection And Opinion Mining System For Online Comments

Recently,with the rapid development and extensive applications of Web 2.0,more and more interactive news websites(e.g.news.sina.com.cn,toutiao.com,etc.),interactive e-commerce websites(e.g.dianping.com,autohome.com.cn,etc.)and interactive inquiry system of enterprises’ credit(e.g.www.qichacha.com etc.)are keeping coming to the fore.As a result,these websites not only improve the convenience of on gaining social events,enterprise information and product information for users,but also provide chances for people to express their views freely on the Internet.Usually,the comment information on Internet includes Internet users’ views and preference on social events,enterprises or products and information.It is also an important way to understand social sentiment for related government department and companies,so that they can take active actions to handle them.However,in the era of big data,the amount of the information online is exceedingly huge,while its data quality is quite inferior and the content of the comments is also short and arbitrary.Therefore,this situation not only requires automatic collection,but also propose great challenges to obtain sentiment from these comments.Besides,although there are many comments referring to the same entity in multiple sources,which can solve the problem of data sparseness in single source,due to data missing,these records referring to the same entity are deemed as different entity.This situation increases the difficulty of integrating comments.To solve the above problems,this paper studies the related web crawler technologies,data integration and opinion mining methods.In addition,we implement a collection and opinion mining system for online comments.The related technologies about web crawler can obtain comments from Internet with good performance from various kinds of websites,and then make a fine-grained analysis of public opinion.Specifically,our work includes the following several aspects:(1)In data collection aspect,we not only design and implement a web crawler framework,but also use the framework to collect and save these comments in database to establish a foundation of data analysis.(2)We carry out entity matching between multiple sources and integrating comments,which can avoid the problem of data sparseness.(3)We study and analyze the related technologies and methods about text sentiment analysis,and we also design and implement a public opinion evaluation framework which can be used to do the fine-grained sentiment analysis for the comments on Internet and identify features and evaluation.(4)We show the data in visible way.With the inquiry interfaces,users can not only acquire abstract and the whole sketch of products.