Design And Implementation Of Financial Data Collection And Intelligent Analysis System Based On Web

This project comes from my internship company Thomson Reuters (Beijing). As an information provider, it is crucial to provide the qualified information and qualified information service. In the information age, we are surrounded by the torrent of information. Nowadays, data is needed by every industry and business functional areas and become an important factor of production. Digital information has become the strategic resources to promote social progress and economic growth. The data analysts in the financial trading department of Thomson Reuters have to deal with the real-time data and historical data from more than250stock exchanges or hundreds of over-the-counter markets around the world every day. It is unrealistic to collect and analyze the data only by human work.In order to realize the automation of the process, we develop the system ACE (Accelerated Content Engine) of financial data collection and intelligent analysis based on Web. The background of ACE system can help analysts get the real-time trading data with full automatization, then the collected data for automatic analysis process or distribution process. The foreground of ACE system provides automated data checking and semi-automated data analysis function for data analysis team.Combined with the analysts’ business processes, this system was developed. Details of the work are as follows:(1) Completing the demand analysis of the system. Based on my study of software engineering courses and my survey of demanding with the analysts of ETI (Exchange Trade Instrument) and Corax, I completed the system demand analysis.(2) Completing the overall architecture design and initial implementation of the system. ACE system includes ControllingMaster acquisition subsystem, DataDistribution distribution processing subsystem and the design of the database.(3) The design and implementation of the collecting subsystem. Including App Engines the crawler engine module, scheduler module of Wrapper and the module of crawler engine plant of App Host (Slave Host).(4) The design and implementation of the distributed processing subsystem. Including ACE Web UI (User Interface) semi automatic data processing interface module, ACE Fully Automation automatic data processing module.(5) Testing the system. After the completion of system function debugging, I tested the white box and black box respectively. The system has been completed and used in the data analyst team of Thomson Reuters. It works well and meets the demands of data acquisition and intelligent analysis. It is very advanced and practical.