The Design And Implementation Of Nanjing Technology Market Big Data Analysis Subsystem

Nanjing Technology Market Management Office is responsible for managing the technology market in Nanjing, one of the important tasks is to help the scientific and technological enterprises and organizations for tax refunds. There is an existing business system helps dealing with daily work, which includes functions such as viewing and reviewing the applications submitted by the enterprises or organizations, enterprises or organizations query, contract information query, printing reports, etc., it helps achieving paperless work flow and improving work efficiency. This business system is suitable for small amount of data, it has wide but not deep functions, especially in terms of query and analysis of data.With the rapid development of technology market in Nanjing, the amount of data will continue to increase, the existing business system will be challenged. Meanwhile, the query function which is provided by the existing business system only has the detailed query, and it’s not consummate. For example, its query conditions are not complete, and it lacks several features the Office needs, such as sorting and filtering the query results. In addition, for the contract datas which are most concerned by the Office, the related functions provided by the existing business system are relatively simple and stereotyped, and the statistical datas are not intuitive and rich, and it uses the real-time statistical way, which is relatively time consuming.A new big data analysis subsystem emerged in addressing current and future data amount also offers a variety of query and data analysis functions. The query function not only retains the original business system query function in the design, but also makes improvements to its problems and adds several new features. In addition, the subsystem will provide Overall Query function for contract datas, to show the general characteristics of them at different angles. Statistical analysis will be redesigned, at first, off-line statistical way will designed in architecture, to provide rapid support for the online statistics, and online statistics will be devided into Plane Statistics and Dimension Statistics two kinds, both of them use interactive charts to show statistic datas.Users can use the Detail Query function to get the detailed understanding of technology market datas, and also use Overall Query function to get the general characteristics of contract datas. Also uses can use Plane Statistics function and Dimension Statistics function, which are supported by Offline Statistics function, to obtain the macroscopic understanding of technology market datas from different dimensionals. Offline Statistics function including getting datas needed by the subsystem from the original business system, and doing the offline statistics of contract datas.The subsystem uses Hadoop and HBase and other distributed techniques to handle the big amount of data, as well as to provide the entire subsystem data base and computing environment. In view of the Office staff habits, as well as continuous demand for system updates in the future, it uses B/S architecture. Play, a Restful web framework is used to realize server side logic, operations of database, frameworks of front pages and interaction with front pages. EasyUI and other techniques are used to realize the modern front pages and rich charts, as well as the interaction with server side and end users. In addition, in the development process of the subsystem, UML modeling is used to assist in requirements analysis and design sections.The subsystem has gone live, it can effectively deal with the growing amount of data, and help the Office staff understand the technology market deeply, meanwhile provide data support for making rational decisions. Based on its architecture, more work can be done in the future, there is some room for improvement and development.