Multi-level Cache Management Of Three-dimensional Spatial Data Engine

It is the research frontier in geo-information science that how to effectively realize integrative storage, management, analysis, processing and application of above-ground, under-ground geographical or geological spatial data, which is with the features of complex structure, various types, large amount of data, the uneven spatial distribution, discrepancies between objects and expression of multiple levels of details. Key technologies are pressing to be addressed in three-dimensional data management that integrative organization of different above-ground, under-ground, out-door and in-door 3D spatial data, parallel management of large-scale three-dimensional data, three-dimensional spatial index, multi-level caching and dynamic scheduling.Based on analyzing existing cache management algorithm and cache software module, making the comprehensive consideration of various information technologies such as three-dimensional spatial data technology and spatial database engine technology, the paper presents the cache management module for three-dimensional spatial data and development approach of massive three-dimensional multi-level data cache scheduling engine management module for file system, relational database management system and its parallel cluster management system etc, combining the characteristics of geological spatial data(multi-source. multi-class, large quantities, multi-dimensional, multi-scale, long state, multi-subject) and taking into account the full three-dimensional spatial data sharing and efficient utilization.In order to integrative organization of various three-dimensional spatial data describing ground and underground, outdoor and indoor, to meet the demand of efficient data scheduling for three-dimensional real-time visualization applications, caching is an important way to improve system performance. The paper focuses on multi-level cache management of data scheduling in massive three-dimensional spatial data engine. For the need of three-dimensional real-time visualization applications.the cache management system can make a effective integrative organization of various three-dimensional spatial data describing ground and underground, outdoor and indoor, and be applied in three-dimensional spatial data engine of three-dimensional spatial data management system. The main research results are summarized as follows:1) Cache replacement strategy:Traditional substitution algorithms for cache management are more unitary, and generality each algorithm only has correspondingly good effect on certain type of access pattern. In order to achieve dynamic management of cache, we should take the running conditions of the database and the actual implementation queries into consideration. So it needed to devise a cache substitution algorithm which makes comprehensive consideration of frequency, time and space.2) Object caching pool and its management algorithm:the paper defines multiple buffer pools, start with the object-oriented ideology to classify the objects using the object types. Dividing and initializing buffer pools according to data types of the object, then schedule different data type of objects in different caching pool. Object association query algorithm works as follows:query the relative ones when dispatching objects and generate a complete spatial object model as a return value including detailed attribute information.3) Multi-level caching:establish efficient data caching and application caching in multiple levels, like server/client and hard disk/memory models, store the application objects and their associated data. For three different applications systems, which are file system, relational database management system (such as Oracle 11g) and parallel cluster management system, has simultaneously implemented multi-level cache adaptive setting.4) Research on adaptive caching:run caching data management and adjustment in tow threads. Adaptively adjust cache and replacement strategy according to system resources usage and querying statistics on cache, making cache performance effectively and reducing the competition for system resources, while ensuring a high hit rate.5) Research on pre-schedule algorithm:combining the features of cache management and spatial index, considering the memory capability and CPU's computing power, design adaptive scheduling algorithms for three-dimensional spatial data.In accordance with the needs of three-dimensional spatial data engine for adaptive visualization of multi-level detail real three-dimensional spatial data and I/O bottlenecks and resource competition in the large scale real-time three-dimensional visualization of physical data, the paper presents a self-adaptive multi-level cache management system, combining the real-time scheduling and pre-scheduling of three-dimensional spatial data, integrating the usage of system resources, adopting multi-thread mechanism to reasonably schedule cache management. It improved hit rate in searching three-dimensional spatial data, reduced competition to system resources, and satisfied the efficient request for the seamless integrate visualization data. In the paper the main innovations are:1) Oriented-object caching pool management:the caching management technology scheduled object in different caching pool, classifying in object types. Object caching pool solves the problem of the cumbersome in object searching because unitary caching queue stores so many types of data items. Oriented-object caching pool management technology manage all caching pools by caching management class, and each object caching pool adopts independent access management mode. Object association query algorithm query the relative ones when dispatching objects and generate a complete spatial object model as a return value including detailed attribute information. It effectively reduces the time of object scheduling, avoiding the difficulty of dynamic organization of object module and centralized scheduling.2) Adaptive adjustment of cache:It respectively manages caching pools according object types, adaptively adjusts the settings of cache and cache pool according to cache items and cache pool queried record. The replacement algorithm based on the hit times and the visit time interval can be reset according the running information of database and system resources. It achieved the dynamic management and adaptive adjustment.3) Cache structure design based on spatial relations constraints:Building spatial index tree to caching object adds spatial issues to cache, improving the traditional cache replacement algorithm which principally used hit frequency or access time as standard to determine which data block should be replaced in replacement operation. It can be used to forcedly replace caching object specified spatial position, and support pre-scheduling algorithm based on cluster cache entries.The achievement in the paper has applied successfully in National High Technology Research and Development Program (863 Program)-related topics of key project in earth observation and navigation technology. So does the QuantyView geosciences information software system of CUG, the Geoscope system of Wuhan University, three-dimensional visualization system of Zhejiang University and planning and management information system of Wuhan Urban Planning Bureau. The research result can be applied in Indoor and outdoor routing, roaming, reserves estimates, oil and gas simulation, geological analysis which need quick scheduling of massive three-dimensional spatial data.