Research And Application Of Unstructured Data Storage For Cloud Storage

Continuous development of computer applications led to a sharp rise in the amount of data, Since the structure of the data is limited to manual handling, resulting in the growth rate of unstructured data is far greater than structured data. How to store unstructured data in applications, including text, images, audio, video, PDF, spreadsheet, etc., is a question often troubled system designers. Typically, there are two ways to store unstructured data. The first way is using the file system to store unstructured data as files, at the same time, the file path or link is stored in the table of relational database. The advantage of this approach is that the speed of data reading and writing operation is faster, the disadvantage is inconvenient to manage the data, and also needs to consider security and transaction conformance data. The other way is storing unstructured data in the Blob field in table of traditional relational database. This way full use of transaction, management and security of the database. But conversely, it could lead to performance problems on data querying and writing. Thus, it is of important significance to research and analysis of how to store large amounts of unstructured data.Cloud storage is the development and extension of a number of technologies, such as grid, parallel and distributed computing. The key factor restricting the storage of unstructured data is that the degree of data sharing between data center and heterogeneous terminal is not high enough, and storage systems lack the ability of high concurrent reading and writing and linear scalability is relatively poor. In this paper, based on cloud storage and NoSQL technology as research background, proposed a storage solution for unstructured data for cloud storage, the data is distributed in cloud to be processing. In this case, not only reduces costs, but also improves the storage capacity of the data.This paper firstly discusses the status of unstructured data storage and development prospects of cloud storage. Studying the ways of unstructured data storage, and discussing the characteristics and shortcomings of the current storage ways. Then studying three storage technologies in cloud environment, and discussing the new database technology NoSQL as well as non-relational database MongoDB under the cloud environment. Combine with the characteristics of unstructured data, advantages of cloud storage and storage characteristics of MongoDB, Proposed the necessity of storing unstructured data in cloud. Based on the popular cloud platform Hadoop, designed unstructured data cloud storage platform. On this platform, the unstructured data and their metadata information stored separately, unstructured data stored as files in HDFS distributed file system, metadata information is no longer stored in the NameNode but switch non-relational database MongoDB for storage. Then design experiments to verify the feasibility and effectiveness of the platform, and the experimental results show that store metadata information with MongoDB, can speed up data accessing, and the memory consumption of NameNode have significantly reduced. Finally, For the problems in massive image storage system, analyze the feasibility of unstructured data cloud storage platform, design cloud storage system for massive photo and implement.