Analysis And Application About Internet Spatial Data Based On Spatial-Spark

Digit City plays a vital role in smart city,and facing explosing amount data fetch,management,analysis and mining challenges.Mobing internet developments cause the cyber spatial data increaing unbelievablely which contains much informations for building the smart cities.Howerver,the heterogeneous,irregular and countless data leads the spatial data's analysis,mining and knowledge discovering becoming more intractable.Traditional spatial analysis method cannot meet the above requires,so this thesis builds the parallel spatial computing framwork,called Spatail-Spark,based on popular open-source parallel computing framwork.Using this framework,it mines the spatial co-location pattern in Weibo's POIs and population graph according to the Weibo users' s spatial locations.The achievements and conclusions of this thesis are listed as follows:(1)It expands the Spark RDD in spatial geometries called Spatial RDD,including point,line and polygon.And those RDDs support the spaital data reading and writing,spatial coordinates converting and spatial index building in separate partition.It also provides three spatial query modules: sptaial topology query,spatial K near neighbour query and spatial join query.At last,it designs experiments to figure out the efficiency of Spatial-Spark by various comparsions.(2)After fetching out Weibo's POIs through Weibo's API,it conducts the spatial co-location pattern mining algorithm in those data.It analysis the vital key of spatial co-location pattern and redesigns the parallel algorithm for efficient performance using Spatial-Spark.It comes out that the there are huge differences among the different cities,taking the Shanghai,Wuhan and Chongqing.It also provides that the patterns shows more commercial characteristics when the pattern's order is bigger on the condition that the spaital distance threshold is 500 meters and the participating index threshold is 0.6.Moreover,the six order pattern is(KTV,Chinese Restaurant,Coffe Bar,Dessert Stall,Barber Shop,Bar Pub).(3)It constructs the cities floating population graph using Spatial-Spark from the national Weibo's users location in 2016 Chinese New Year holiday.At first,it calculates the popluation amount of flow-in and flow-out,and flow ratio,which displays the diversity of floating population among the national cities.Then it figures out the national cities' weights in floating population network under PageRank algorithm.It finds out the relationship between city's weight and city's development and divides the natianal cities into four levels.Lastly,it applys the parallel community mining algorithm in this network and uncovers the phenomenon that the cities' s popluaiton relationships are almost limited by the provinces and there are also some exceptional cases.