| China's expressway is under rapid construction in recent years,road network structure gradually formed and continuously improved.Pay system business is also constantly increasing,The requirements of meticulous management are getting higher and higher the rapid emergence of road network and the formation of multi-dimensional,multi-section,a variety of mass data,,which are huge,complex and diverse in source and so on.Due to the various systems put into operation at different times,different design and construction units,As a result,all the current network charging data have not formed a standardized and unified storage format and data acquisition is relatively complicated,resulting in various forms of data being loosely packed so that they are not fully utilized,resulting in a lot of resources being wasted.And some small road network pay system using centralized architecture,the system requirements on the host is relatively high,if the host fails,may cause the entire network to stop working.This paper starts from the actual needs of users and studies a road network charging data integration scheme based on cloud platform.This article first introduced the data integration scheme and cloud platform related technologies,studied the ETL process and the Hadoop distributed computing platform;secondly designed the MapReduce parallel framework and ETL parallel processing,and transformed a large number of heterogeneous data into the same format,standardized and unified the storage format of the data;The greedy algorithm is then used to design the distributed ETL,which achieves load balancing of the system.In addition,the experimental results are tested and analyzed in this paper,which proves that the cloud platform-based road network toll data integration scheme achieves load balancing,fast processing speed,and can better meet users' efficient management and storage of massive road network toll data.The demand shows that this system has very important application value. |
PDF Full Text Request