| Large scale e-commerce platforms have a large number of commodities and corresponding transaction data.The mining and analyses of these data have important research values for optimizing platform construction,increasing product sales,improving consumer shopping experience,etc.Usually,researchers analyze the HTTP requests generated by the product page through packet capture techniques,find the data source,and then use crawler technologies to capture the corresponding data.When we need to collect the commodity data of multiple platforms,because the HTTP requests and interface parameters of the platforms are different,we need to do packet capture analysis on each platform,which will lead to the time and energy invested multiplied with the number of platforms.How to quickly collect commodity data from different platforms is a challenge.At the same time,with the rapid development of artificial intelligence technologies,various knowledge-based intelligent applications emerged one after another.Knowledge Graph technology provides a method to extract structured knowledge from massive texts,which is considered to be the cornerstone of machine understanding semantics and realizing cognitive intelligence.Knowledge Graph expresses the information in the Internet in a form closer to the way human cognitive world,provides a better ability to organize,manage and understand the massive information in the Internet,and has become the infrastructure of various intelligent applications based on knowledge service.At present,many scholars have explored the construction and application of Knowledge Graph in the specific fields.However,most of the existing researches focus on the research of construction methods,and the construction of domain Knowledge Graph often needs to be combined with its own characteristics.Facing different fields,the construction process is not the same.Based on the above problems,the main work of this thesis are as follows:A commodity information collection system for large-scale e-commerce platforms is designed and implemented.This thesis analyzes the commonness of page structure,basic interaction process and commodity loading mode in e-commerce platforms,and puts forward a general data capture strategy for different e-commerce platforms.Through the method of simulating browser operations,combined with Scrapy crawler framework,we can quickly realize the collection of commodity information on different platforms.The construction of prototype system of commodity Knowledge Graph.Based on the collected commodity data,combined with the domain characteristics of e-commerce platforms,this thesis analyzes in detail the advantages and disadvantages in the process of building commodity Knowledge Graph for large-scale e-commerce platforms.On this basis,the construction methods of commodity Knowledge Graph are deeply studied.And according to the construction methods of the prototype system,we can continue to expand the scale.The realization of commodity name extraction algorithm.This thesis proposes a method of "Stop words + Rule",which can extract the name of the commodity from the non-standard string of commodity name. |
