Design And Implementation Of An Adaptive Storage System Oriented At Heterogeneous Big Data Of Power Grid

After several decades of development,the traditional relationaldatabase has become successfully used in data storage,service provision and problem solving field.However,with the advent of the era of big data,the relational database is showing many shortcomings and facing new challenges,due to the increasing diversity,abundance,variability and sharp increasing rate of the big data.Meanwhile,the non-relational database has been rapidly developed in big data field due to its flexible data model and the advantage of dealing with large-scale data.What is more particularly significant is that with the development of the smart grid,the power industry's big data has been dramatically grown in size,with much more abundant data sources and changeable structures,leading to the requirement of the use of multiple databases to store and manage its data.However,there is no uniformed database standard for the operation of multiple databases,resulting in a lack of a unified technical standard which can adapt multiple various database operations.Therefore,it is very urgent to devise an adaptive data storage technology in order to unify different database operations,so that high level applications can operate data on a unified base and will be benefit from the improvement of its business development efficiency.This thesis mainly targets to devise and develop an adaptive storage system for power grid heterogeneous big data.The purpose is to solve the problem of adapting multi-database operations for power grid heterogeneous big data,and also provide a unified set of data operation interface for other big data applications.By providing common data operation interfaces,its goal is to hide the details of multiple underlying databases and simplify the data operation for developers.This is no doubt of great significance to those applications based on a large amount of data which has rich data types and need to use a variety of databases for data support.This set of adapted data operation interface has a relatively high scalability and can be modified with a low cost.In addition,there exists two innovations.Firstly,it will automatically select an underlying database which best meets the requirements for database performance.This is achieved by accepting the input of data performance parameters from the clients and then calling the model selection algorithm.Secondly,the system itself has an index management function to manage database index which is also transparent to clients.The main content of this thesis is to design and implement an adaptive storage system for power grid heterogeneous big data.Firstly,this paper outlines the research background,and introduces the related theories and technologies.Secondly,it makes a comprehensive analysis of the requirements of the system,states the suitable use conditions and majorly elaborates the functional requirements.Thirdly,in accordance with the functional requirements,it illustrates the system's architecture with a more detailed discussion of the system's internal modules and also covers the realization of relevant implementation details.Next,the testing part shows the process,results and analysis of module test and integration test for the system and also gives out a brief analysis and summary of the non-functional requirements for the system.Finally,the paper summarizes the whole design and implementation of the adaptive storage system and points out the research direction of this project in the future.