Research On Data Cleaning,Mining,Jobs And Residential Locations Based On Mobile Phone Signaling Data

As the political,economic and cultural center of its jurisdiction,the development of city can effectively promote the development of its surrounding areas.Reasonable residential space planning can availably improve the quality of life and work efficiency of its residents,is an indispensable part of urban construction.Thus how to obtain the density of city residents' population and jobs housing spatial distribution is a big problem in city planning and building,the traditional ways of obtaining this kind of information often rely on site visits and questionnaires,these methods not only consume a lot of manpower and resources,but also cannot guarantee the accuracy of the results.With the development of time,some of the more advanced technologies are used in the city to obtain information,such as telephone interviews,video monitoring and recording,but these methods still have the disadvantages of high cost,small coverage.At present,using the mobile phone signaling data to acquire city residents living and working space distribution is a new technology in the era of big data.Based on cleaning,mining,extracting the mobile phone signaling data,this thesis successfully obtained jobs-housing spatial distribution information.The work of this thesis is mainly reflected in the following aspects:In the aspect of data cleaning,the thesis proposes the principle of hierarchical cleaning for signaling data:first,we regard signaling data as conventional data and delete its invalid data;then we delete the drift data which is peculiar to signaling data;finally,based on the need of extracting stay points,we propose LOF outlier detecting algorithm improved by K-means clustering algorithm to clean the track points which are ill-suited for stay points extraction.The experimental results show that the improved algorithm can improve the efficiency of the algorithm without affecting the detection accuracy.In the aspect of data mining,aimed at the defect that DBSCAN clustering algorithm is not suitable for dealing with spatio-temporal data,we start from the basic concept of clustering algorithm and expand its semantics in time dimension to make it suitable for extracting stay point from signaling data;to determine the parameters which are hard to determine for clustering algorithm,we innovatively combine the definition of stay point and the features of data source to determine the parameters;finally,we compare the experimental result with the classical stopping point extraction algorithm to prove the superiority of my algorithm.In the aspect of living and working space extraction,first we analyze the stay points we got from experiment and obtain its basic characteristics;then we set the time threshold for living and working stay point under the relevant information and design an algorithm to extract living stay point and working stay point form stay point data set;finally,we draw resident living and working heat map based on the data we got,and analyze the heat map with the actual situation.