Application Research For The Calculation Method Of Massive Complex Structure Data

In recent years, affected and promoted by the impact of new Internet technologies, more and more traditional industries use Internet technology to process data services. Massive date with complex structure makes traditional relational database no longer applicable. With high flexibility and high concurrency, non-relational databases provide efficient support to store and calculate complex structure data. More than just relational and unstructured data, some non-relational databases also offer certain capabilities or deal with spatial or geographic data. Collaborative applications in multiple database systems have become a tendency due to weakness compare to dedicated GIS databases.Base on MongoDB shard technology and basic ways to choose shad key, this thesis gives an automatic decision mechanism of shard schema and shad key by analyzing database operational scene and calculating database operational vectors, and proposes new deal for distributed database storage planning. Then this thesis focuses on the pros and cons of difference between relational and non-relational databases under various circumstances. Performance and applicability analysis of PostgreSQL and MongoDB are made by comparing performance of data loading, disk footprint, query on primary key and geometry coordinates. Different types of indexes are also taken into consideration. An application method of the collaborative architecture of MongoDB and PostgreSQL is established with the application in telecommunications pipeline resource management system. It shows good performance in practices.