Design And Implementation Of Structured Data Extraction Platform Based On LNMP Framework

The rapid development of Internet technology provides a large amount of information and resources for people, a lot of information comes from database query and then shown in a web page using template, this kind of data is known as structured data or records. Extracting structured data provide value-added services for many fields such as information integration and vertical search, which has a great deal of usage, there have been a lot of researchers studying it. Structured data extraction platform based on LNMP framework is dedicated to creating the process accessing to entire network structured data quickly, creating the process of getting and analyzing web structured data for baidu’s product, providing reliable data support for online product. At the same time, through the platform construction, users cost litter for learning data extraction, improve work efficiency, and bring profits for the enterprise.The structured data extraction platform designed and implemented in the paper is based on LNMP (Linux operating system, Nginx server, MySQL, PHP, database programming language) development. The platform side select the combination of Yii (based on PHP MVC framework), Smarty (based on PHP template engines, used to implement separation), Bootstrap (UI framework based on HTML5 and CSS3). Through the use of all kinds of frame, making the development more efficient. Front building by a template engine with the separation of the before and after and lower coupling to achieve high fault rate. Following the ideas of software engineering, during the process of constructing the structured data extraction platform, first of all, the author involve in the collecting, sorting and analyzing of the demand; Secondly the author participat in the system function module decomposition, the design of the technical architecture and database. During the detailed design phase of the system, the author implement template management module, task management module and data extraction service module. During the testing phase, the author participat in the design the test strategy, complete function test and performance test independently. For the structured data extraction platform, template management module is responsible for guide the user to complete the extraction template visually, and then realize the various maintenance operations; Task management module is responsible for submitting task to the back-end service and realize the maintenance of the task information; The service side is the key link in data extraction, it realizes the function of task scheduling, accessing to web information, performing extraction algorithm, finally getting structured data.At present, the data extraction platform is at the commissioning stage, the modules realized by author work well, and achieve the desired goal.