Visual Cleaning System For Traffic Positioning Data

In recent years,with the increasing number of vehicles and the city public transportation system gradually improved,a large amount of traffic positioning data were gradually accumulated.In a new era of big data,we can explore the rules of urban road traffic effectively combine with visualization and visual analysis method,however,due to the failure of the acquisition equipment and transmission network delay,a lot of dirty data are scattered in the original traffic positioning data,while the traditional data cleaning tools are mainly used in small-scale structured data,and are difficult to cope with when face of mass storage and high dimensional traffic positioning data.In order to solve the above problems,this paper designs a kind of visual data cleaning system based on Web platform,and different detection and processing methods are given for different kinds of dirty data,through human-computer interaction,this system allows users to control the dirty data detection means and the intensity of data cleaning,The main work and achievements of this paper are as follows:(1)Non-abnormal dirty data cleaning.In this paper,we classify the non-abnormal traffic positioning data into attribute missing data,repeated data and error data,and selected the different detection methods and cleaning methods for each one,to ensure all kinds of non-abnormal dirty data can get a reasonable and effective treatment.(2)Abnormal data detection and processing.For those abnormal data which cannot be recognized by the database,the system supports users to explore them with a variety of visual components firstly,then provide data mining and processing tools,combined with kinds of interactive methods,the system can distinguish data from multiple views.(3)Architecture design of data visual cleaning system.The system is designed based on B/S architecture,and it is not limited to the specified operating system and operating environment,some pre-processing work were completed before the cleaning operation.We introduced React.js to accelerate the rendering work,which can improve the efficiency of visual analysis effectively.In the end we take Hangzhou taxi GPS data as an example,and clean it with our methods and functions provided by the system,system effectiveness is verified by the velocity/flow pixels,and we provide reference for exploring reasons of the dirty data distribution.