A Multi-level Hybrid Spatiotemporal Index Method For Multi-modal Scene Data

With the rapid development and application of internet,sensor network,Internet of things and social network,smart city will produce massive multi-modal spatiotemporal data every moment,which provides data support for multi-level visualization tasks such as view-only,analytical and explorative.The multi-modal data required by the task is called multi-modal scene data,which has the characteristics of heterogeneity,dynamic change and complex association.It needs to be processed in a timely manner with high performance and low latency,and puts forward higher requirements for data storage,index and query.How to organize and manage multi-modal scene data efficiently to meet the different needs of data-intensive,computing-intensive and interactive-intensive applications in smart city is a serious challenge for the research of scene data organization and management.Existing scene data organization and management is mainly oriented to a single low-level view-only visualization task.Data storage is mainly disk-based,I/O latency is high and mode is single,and index method is mainly tree-based.It is difficult to organize the time,spatial,semantics and relation of multi-modal scene data efficiently to meet the needs of diversified queries.In order to realize adaptive aggregation of scene data for multi-level visualization tasks and meet the needs of real-time scene construction and interaction,a multi-level hybrid spatiotemporal index method for multi-modal scene data is proposed.Firstly,the characteristics of multi-modal scene data and the connotation of multi-level visualization tasks are systematically studied,the different scheduling requirements of multi-level visualization tasks for scene data are analyzed and a global-local collaboration of spatiotemporal index mechanism is established.Secondly,a multi-level hybrid spatiotemporal indexing method based on spatiotemporal relation graph is proposed,and on this basis,the update and optimization methods of internal and external index are studied.Finally,a scene data organization and management engine based on micro-service architecture is further studied,and a memory-centric multi-modal storage micro-service for scene data is designed to achieve efficient storage,indexing and query of multi-modal scene data for multi-level visualization tasks.The main research contents include:(1)A spatiotemporal indexing mechanism for multi-modal scene data.By studying the multi-source,multi-dimensional and multi-scale characteristics of multi-modal scene data in cyber-physical-social systems,as well as the connotation and driving force of three levels of visualization tasks,this paper summarizes the different requirements of multi-level visualization tasks for multi-modal scene data scheduling.Aiming at the problems of I/Ointensive,computation-intensive and interaction-intensive in the process of scene data organization for multi-level visualization tasks,a global-local collaboration of spatiotemporal index mechanism is designed,which provides a theoretical basis for subsequent research.(2)A multi-level hybrid spatiotemporal index method based on spatiotemporal relation graph.On the basis of the above-mentioned spatiotemporal indexing mechanism,the memory implementation of sparse matrix-based spatiotemporal relation graph is studied.Then,using the spatiotemporal relation graph as global index,a multi-level hybrid spatiotemporal indexing structure of scene data is designed,and the association mapping from global index to local index is established,which breaks through the complex scene data management and scheduling technology of efficient collaboration between internal and external memory to meet the diverse needs of scene data scheduling.(3)A scene data organization and management engine based on micro-service architecture.In order to realize the efficient organization and management of scene data ready for view-only,analytical and explorative,the structure of scene data organization and management engine and its operation mechanism are designed based on micro-service architecture and the efficient integration of scene data storage,indexing and query is realized.Furthermore,a memory-based multi-modal storage micro-service for scene data is proposed,and a view-only oriented,analytical oriented and explorative oriented view of multi-modal scene data query is studied,which improves the visibility and realizes efficient organization and management of scene data.(4)Based on the above research results,a prototype system of scene data management is developed.The mainstream relational database Postgre SQL and No SQL database Mongo DB are used as comparison methods.The efficiency experiments of scene data organization and management are carried out for typical basic framework data,intelligent sensor data and associated data respectively.The experimental results show that the proposed method is superior to the contrast methods,and can organize and manage multi-modal scene data more efficiently,meet the different requirements of multi-level visualization tasks for scene data scheduling,and provide data support for real-time scene construction and interaction.