The Research About The Intelligent Technology Of Information Collection In Mobile Internet ERA

With the development of Mobile Internet,eople have more ways and means to access internet.eople can accesss to internet anytime and anywhere with a terminal,browsing the Web page and publishing content. In recent years,with the development of social networks,there are more and more hot topics emerge,especially after we have Micro-blog. icro-blog accelerate the dissemination of information,expand the dissemination of information.More and more researchers are interested in it because of that.Before the research on hot topics,data collection on the internet needs to be done.The major work of this paper is presents as follows:At first, we analysed the basic procedures of information access. We have known that there are two process in information access,the Web Crawling and the Information Extraction.Then we analysed the the basic procedures of Web Crawling and the type of Web Crawling.We also analysed the strategy how the crawler get the Web page.Second, we analysed the method of Information Extraction and it’s support technology.Then we studied the method of the wrapper and the wrapper algorithm in detail. Third, We proposed a way to use API and Web page together in the information collection of Sina Micro-blog.At last,we studied the relational databse, the large data processing platform Hadoop and the distributed databse Hbase...