Research On China-involved Public Opinion News Data Cleaning Method

The2013was defined as the first year of big data by academics, through the rapid development in recent years big data has caused profound changes in various fields. Its influence has become increasingly prominent. Big data transformation in the field of news is particularly significant, with the development of automatic writing technology and the popularization of mobile terminal, news media release explosively increased and brings new challenges to the international media personnel. The traditional analysis method and news gathering process have been unable to cope with such a large data source. But the analysis of China involved foreign media release is significant to diplomatic work and can largely influent political sentiment in information age. Thus this is high time to cope with development of the technology. We should change traditional ideas and build a new processing method. According to the specific work process and requirements of China-related public opinion, if we can combine news media rating and the news manuscript rating to form a scientific and effective rules library of data cleansing of China-related public opinion, then the researchers of public opinion would boost the ability of massive news data cleaning. The application of data cleaning method can effectively reduce the amount of data processing and remove noise of independent news. What’s more, it can focus on high level and high influencing news media, and high research value press releases which are on its targeted fields, and control the quality of information and eliminate the noise and invalid information. In this way, it can optimize the process of clean, collection of China-related public opinion, and release the researchers from huge amounts of data, improve the efficiency of the whole production processes of public opinion products.