Research On Real-time Processing Platform Of Massive Traffic Data Based On Storm

Traffic data real-time processing is a hot spot problem in today's intelligent transportation system.It can provide effective support for various traffic management services and traffic information services and help to solve many traffic management problems such as traffic congestion via accurate and efficient real-time computing of massive traffic data.Traffic data real-time processing has many features,such as data continuity,short calculation delay,calculation of non-stop,multi-type data mixing and so on.Under the background of real-time processing of illegal vehicles based on license plate recognition data flow in laboratory's large-scale urban vehicle inspection and prevention-control system,this paper proposes a real-time processing method of massive traffic data,and designs a real-time processing platform of massive traffic data based on Storm.And focusing on the load tilt in the original scheduler of Storm,this paper studies the scheduling technology for the Topology which deal with traffic problems.The main points of this paper include:1)This paper proposes a real-time processing method of massive traffic data,and defines the contents of key steps of massive traffic data real-time processing such as data access.Data organization,data computing and results output.Combined with the existing streaming computing model,this paper analyzes and describes the key technologies involving in the above steps such as data receiving preprocessing,multi-type data access,index structure of data in nodes,the method of data organization between nodes in the cluster and the data output interface.2)This paper further studies the task scheduling problem in real-time processing of massive traffic data.In order to improve the performance of traffic data real-time processing,and solve the problems in the default scheduling strategy such as simple scheduling mode and load tilt among nodes,this paper proposes a FNS scheduling strategy based on fixed nodes direct allocation and a LBS scheduling strategy based on load balancing,and achieves the corresponding task scheduler.The experiment shows that the FNS scheduler can allocate some simple topology,and the LBS scheduler can allocate tasks from the nodes with the most resources every time.3)This paper designs and achieves a real-time processing platform of massive traffic data based on Storm.Based on this platform,the real-time computing algorithm in the automatic recognition of copy plate vehicles,for example fake plate vehicles recognition,is achieved.And using real traffic data set to simulate real-time traffic flow data,the experiment and evaluation verifies the accuracy and real-time of computing.