Font Size: a A A

The Design And Implementation Of Data Crawling And Processing Moudle Of Trendata Data Analysis Platform

Posted on:2015-11-09Degree:MasterType:Thesis
Country:ChinaCandidate:J W HuanFull Text:PDF
GTID:2308330461956660Subject:Software engineering
Abstract/Summary:PDF Full Text Request
With the great popularity of E-commerce, more and more people choose electronic shopping. After that, these websites have huge E-commerce data. It is self-evident that E-commerce data has a neccessay importance to the E-commerce websites’ construction. The advantage of E-commerce compared to traditional retail is the availability of data, and these data can help E-commerce websites provide more accurate customer service. As a leading global E-commerce website, how to use E-commerce data to improve sellers’ operation decision becomes a very important problem to Amazon.In order to solve the above problem, this paper introduces the TrenData data analysis platform, which can provide Amazon sellers better data operation. TrenData data analysis platform consists of three modules:data crawling module, data processing module and data display module. Data crawling module is the foundation of this platform, and is responsible for obtaining data from the Amazon; data processing module has two functional part:data cleaning and data analysis which includes the statistical analysis and emotion analysis; data display module is responsible for displaying the results of data analysis in a visual form.In the implementation part, data crawling module and data processing module are explained. Data crawling module mainly uses Scrapy framework, and is deployed by Scrapyd; data analysis of data processing module applies statistic and natural language processing technology to analyze E-commerce data.At present, the platform has been opened for registration, and has a number of users. Through this platform, users can conveniently obtain the operation data, and through the analysis of natural language and emotion were improved information on the description of the goods, enhancing the product acceptance. In addition, the platform has high scalability, which means that developers can add new data analysis methods as well as other software development tasks very flexibly.
Keywords/Search Tags:Amazon E-commerce Data Crawling, Scrapy Framework, Natural Language Emotion Analysis, NaiveBayes Classification
PDF Full Text Request
Related items