Design And Implementation Of Cooperative Website Data Resources Access System

With the continuous development of the Internet technology and the continuous innovation of the applications, e-commerce gradually penetrates into the traditional field and all kinds of life service website develop vigorously. The amount of data in the Internet field has explosive growth. Data acquisition is the core factor of data support. The original data acquisition approach has some shortcomings such as working manually, a small range of data acquisition and low efficiency. In order to solve the shortcomings of the original data acquisition approach, the paper realizes the cooperative website resource data accesssystem based on the design characteristics of cooperative website’s data resources. The system allows the user to configure automative resource data access scheme according to characteristics of cooperative website data resources. It expands the approach of data acquisition effectively and improves data support ability.The paper elaborates the background and the significance of the project first, and analyzes the key technologies that have used in the system development process and some similar technologies were compared and analyzed. After that it analyses the functional requirements, system logic structure of the system and non-functional requirements overall. According to the demand analysis and the overall project management strategy of the company, the paper completed the design of the system architecture and completed the system design and realization, solves the key problems in the process of system development reasonably. The author completed the server-side information management module, resource access request processing module, dispatching center module independently, participate in the completion of the server-side template management module, resource file parsing module and operation terminal. In the process of system development, the system uses the B/S framework and Spring MVC framework in order to to improve the scalability and stability of the system and high cohesion and low coupling. In order to improve the data processing ability of the system, it uses multiple servers that work together to deal with data. Considering the problem of synchronous processing data between in multiple servers, the author designs the distributed lock that can ensure synchronization of data processing with Redis technology. In the process of dispatching center module development In order to ensure the rationality and efficiency of task scheduling, the system uses distributed caching and multithreading technology. In the process of dispatching center module development in order to download the resource files that need to execute Ajax to load file content, the system uses HtmlUnit technology. In the process of resource file parsing module development in order to parse HTML and XML format resource file, the system uses Jsoup and SAX technology. Finally, in order to verify whether the function of the system meets the demands, the author uses black box testing technology to test the functional testing of the system.At present the system is at the test run stage. The system at test run state is running well. The modules that the author completed have good performance and are in line with expectations. The author will collect the problems and feedback steadily in order to improve and optimize the system and meet the data support needs of search channel better.