Research On Storage And Processing Of MongoDB For Laser Point Cloud Data Under Distributed Environment

In recent years, the application of laser point cloud data has increased dramatically. How to efficiently store and fast process the data becomes an important research direction at present. Point cloud data contain a wealth of geographic information, belonging to the category of spatial data. Traditional relational databases are relatively weak in massive spatial data storage and processing, while the application of non-relational databases In a distributed environment provides a new perspective of study for this fact. At Present, the study on parallel processing of massive amount of spatial data is completed mostly under the assistance of MapReduce parallel computing framework of the Hadoop cluster. Therefore, in the highly available Sharding cluster of the non-relational MongoDB database, the distributed storage and parallel processing of the laser-point cloud data are of great research significance.Firstly, this dissertation makes comparisons among several typical non-relational databases, pointing out the advantages of MongoDB on the storage of spatial data and it gives a thorough analysis on the basic structure of MongoDB database, the operating principle of Sharding cluster and the voting mechanism in the replica set achieved by Bully algorism. Then, based on the theoretical basis and technical support, it creates a general structure of the highly available Sharding cluster with combination of the replica set, and gives a detailed description about the establishing process of the cluster from two phases, document configuration and fragment configuration. Finally, it implements Hash and rage fragment storage, geoNear spatial query, sorted and unsorted contrast experiment of MapReduce calculation of the laser-point cloud data in the cluster, as well as the test on disaster recovery and load balancing of the cluster.This dissertation's research characteristics lie in the following aspects. First, set up highly available Sharding cluster of the MongoDB database with combination of the replica set; second, contrast experiment on the distributed storage of laser-point cloud data and geoNear spatial query implemented with selection of Hash fragment and range fragment; third, the improvement of GeoHash algorithm spatial index used in MongoDB database based on the Hilbert space filling curve; fourth, the parallel processing experiment of laser-point cloud data using MapReduce framework of MongoDB database and the optimization of parallel processing of Map Reduce by sorting.The experiment conclusions from this dissertation are as follows. First, Hash fragments make sure that laser-point cloud data is evenly distributed in various nodes of the cluster, while range fragments cause uneven distribution of the data; second, efficiency of spatial query and the distribution of laser-point cloud data are closely related, and in the situation where there is large amount of data, the efficiency of range fragments is higher than that of Hash fragments; third, sorting the laser-point cloud data, to a certain extent, can improve the computational efficiency of MapReduce.The highly available Sharding cluster of Mongo DB database established in this dissertation has the following functions, easy extendibility, automatic data backup, failover and recovery etc. Under such a distributed environment, the study on distributed storage and parallel processing of laser-point cloud data, to a certain extent, can provide the support for the construction of spatial data platform and have a promoting effect for the development of digital city.