Design And Implementation Of Independent Metadata In Cloud Storage

Nowadays, the continuous progress of technology, research and innovation in the field of the Internet has changed people’s life. Data also showed the explosive growth, the model for data processing of the Internet brings inspiration and promote to the aerospace field. Traditionally, data storage and analysis rely on a single computer, it already cannot satisfy the variety business requirements for data storage and analysis that produced by aerospace field. Now the rapid development of cloud storage technology, has brought new vitality to solve this problem. Cloud storage can have many implementation and operation mode, each of which strive for perfection to the demand of the target. Today, the field of aerospace, the requirements of safe data storage and reliability data analysis are constantly improving. Large scale data continue to produce not only lists a puzzle to the data storage system, at the same time, the access to the data and how to show it is a great challenge.This study involves the cloud storage, metadata management is the core of it, and the important thing to construct the system. This paper mainly research on the construction of independent metadata based on distributed file system HDFS, and relying on the metadata architecture for access control to solve the large data file storing and reading problems in the fields of aerospace. According to the characteristics and overall demand in the aerospace field test data, use HDFS for distributed data storage and in-depth study of the aerospace industry permission relation model and distributed cache technology.This article first carries on the division to the possible data types in the aerospace field, and then analysis the specific storage and reading method, set the classify method according to the system requirements. According to the various file types and large amount of data in the aerospace field, this paper uses the extraction of the metadata information from the data file then build a tree structure to storage and indexing different types of data file. We also design unified storage and read interface, improve the efficiency of the user to upload large data files, and according to the specific requirements of the user groups and system set corresponding permission manage, make the different group and people have different operating privileges to the project, and will control the granularity resource permissions to the file level, so that the stored data documents more secure. At the same time, this paper implements a metadata cache and a distributed data file cache between the service and data layer, reduced read latency of file data, the system response speed is improved greatly, so as to ensure the user to read the data in real-time, improve the user experience degrees.This paper finally design a number of experimental cases to prove the method that implemented in this paper. Separately on independent metadata, user permissions and fast storage and access of large files are multiple condition testing, and compared the influence to the system on whether there is distributed cache, the results achieve the expected.