Research Of Real-time Online Organization And Management For Massive Airborne Laser Point Clouds Based On MongoDB

With the rapid development of Unmanned Aerial Vehicle(UAV)technology and Light Detection and Ranging(Li DAR)technology in recent years,airborne Li DAR technology is becoming more and more mature,which has been widely used in the fields of terrain survey,coastal zone monitoring,natural disaster assessment,road construction survey and so on.The cost of collecting airborne laser point cloud data is becoming lower and lower,and the amount of point cloud data is increasing rapidly.However,the research on real-time online organization and management for massive airborne laser point cloud data is still in its infancy.So far the research on laser point cloud data mainly focuses on either static point cloud management or point cloud modeling,and there is insufficient support for massive point cloud data with dynamic range changes in real data collection phase,which cannot meet the requirements of real-time online point cloud data retrieval from scanning region online monitoring,aircraft path planning and other similar applications.To solve the problems mentioned above,this paper focuses on exploring real-time access technology,designs and develops a real-time online management system for massive airborne laser point cloud data,which is based on data index design,storage system selection and data access optimization.The main research and innovation are as follows:(1)A double-layer index structure of internal and external storage is proposed,which is adaptive to the real-time collection of massive point cloud data.In memory layer,the space is partitioned by extended 3D grids and the 3D grid indexes obtained are converted to 1D Morton codes by Morton encoding.In external storage layer,a B tree or hash index is built on the 1D Morton code field generated from memory layer,which greatly improves the efficiency of dynamic data scheduling between internal and external storage.(2)Mongo DB,a Not Only SQL(No SQL)database is selected as the storage system of massive airborne laser point cloud data,and the access efficiency of massive airborne laser point clouds can be optimized by unordered and unverified batch insertion,double-layer index of internal and external storage,variable-length and object-oriented BSON document structure,data partial updating and pre-reading optimizations in region of interest(ROI).(3)A highly available and easily scalable Mongo DB distributed cluster is built.According to the characteristics of massive airborne laser point cloud data,the 1D Morton code field generated by memory layer is selected as the shard key,and the collection of the cluster is partitioned by using hashed sharding strategy.Besides,tests of disaster tolerance and load balance for Mongo DB distributed cluster are performed.(4)A real-time online management system framework for massive airborne laser point cloud data is proposed.The data storage link and the data retrieval link are analyzed as well as the system software architecture is designed.The system is constructed through the analysis,the implementation and the optimization of the system modules such as data storage,data retrieval,point cloud coordinate computation,UDP transmission,index generation and realtime monitoring.Besides,the collection of laser point cloud data in ROI can be monitored in real time through the real-time monitoring module developed by Qt and Open GL.Based on the research and innovation mentioned above,some comparative experiments are carried out to verify the effectiveness of index structure,storage systems and data access optimizations proposed by this paper.And the performance of the real-time online management system for massive airborne laser point clouds is also tested in the experiments.The results show that the system designed in this paper has the capability to organize and manage massive airborne laser point cloud data and monitor the scanning region in real time.