The Design And Implementation Of Traffic Data Service System Based On Heterogeneous Data Source

With the development of the Internet,urban transportation is in a stage of rapid development,and the exponential growth of heterogeneous storage data continues to increase the use and maintenance costs of users and developers.If the data cannot be concatenated quickly and efficiently,the value of the data will not be effectively used.At present,the data sources used by Alibaba Cloud City Brain are RDS for My SQL,Tablestore,HBase,etc.,especially those column-based storage databases that store PB-level data.Users often need to struggle with details of resource allocation because of the databases have their own SDKs,which invisibly increases the cost of using data.At the same time,data should support rapid extraction for analysis without being destroyed,which is required by business.How to unify the data acquisition methods with their respective application scenarios,so that the application side can use data in a low-cost and efficient manner has become a strong demand.In order to solve such problems,this thesis implements a data service system based on heterogeneous data sources,which means users can create data interfaces by online IDE,and use RESTful APIs to reveal data.This thesis studies the technologies used in the development of the underlying databases,and data services of various heterogeneous data sources.The technical con-cepts and data service model applications used in the construction of data service sys-tems are also being discussed.In additition,the RESTful style of service API,database persistence framework My Batis,database connection template Jdbc Template and other related technologies are summarized.Based on the research of database technology and data services,this thesis designs and implements a Unified Query SDK and a data service system based on it.The Uni-fied Query SDK focuses on the unified connection method of the database and provides universal database access methods.It uses a separately implemented connection pool and Druid connection pool,to shield the connection methods of heterogeneous data sources.The Unified Query SDK also solves problems of data addition,deletion,mod-ification,and query through database templates.Besides,the ability to quickly query and analyze data through the online configuration interface API are provided to the cus-tomers with the data service module.The data service module has three sub-modules,including the interface configuration module,version management module,and query engine module.Its purpose is to implement online dynamic configuration of the API,API service version management,and some underlying data operations.With the help of this system,users can quickly obtain data from multiple data sources with low-cost data consumption.The online configuration API interface also meets the function of responding to customer needs in a timely manner,this makes the acquisition of heterogeneous data in traffic scenarios simple and reliable.I participated in the design and implementation work of the data service system,and responsible for the deployment of the project in public and private cloud environments.At present,the system can operate stably in multiple scenarios.