Based On No Database Of Mass Distribution Of Astronomical Image Storage Research

With the rapid development of computer and network technology, as well as the continuous upgrading of hardware and software, the data will be into exponential growth trend. We call such a large data a massive data, or even big data. This marks the arrival of the big-data age. Unlike the previous data, more and more data belongs to unstructured data, such as sound, pictures and video etc. In the astronomical field, with the advance astronomical observation equipment and terminal equipment, the more and more large-scale observatory, and the constant expansion of astronomical observation capacity, the ancient optical observation of astronomical research turns into full-band astronomy. Astronomical data increases at an alarming rate per hour or even every second. Astronomical field are faced with the challenge of mass data storage.Facing mass data storage requirements, traditional relational database is not an ideal scheme for our problem. It even has become the limit of mass data storage because of its inherent characteristics. And the entirely new storage pattern of Cloud Storage brings a new revolution for Storage areas. This paper is based on this storage trends, discussing the application prospect of cloud storage technology and NoSQL database in astronomical mass storage.This paper researches on the cloud storage technology and its application in the astronomical field by using NoSQL database-MongoDB.First, we have basic subject research.Second, it is the study on construction and key technology realization of the mass data storage system based on MongoDB.Once again, we have done lots of experiment analyses on the mass astronomical data storage system. This part starts with four groups’research experiment. Start to store reams of astronomical data (FITS) to collect experimental data. As a result of analyzing the experiment datum, we have come to the following conclusions. Firstly, in such distributed storage as NoSQL database, sharding can largely improve data storage and retrieval performance. Secondly, different chunksize can also affect the storage and retrieval performance. Finding the best chunksize for distributed storage is very important, and in the selection of subdivision512K is the optimum chunksize for4M FITS of files. In this case we can achieve the highest storage efficiency. Thirdly, memory mapping storage database like MongoDB always appear certain block in storage and retrieval of data. And the block has no significant relationship with chunksize. Fourthly, storing FITS files with different size has different optimum chunksize. In the selection of seven components slices if the file size less than16M, the best chunksize size and file size have the proportion with1:8. And if we have FITS files greater than16M, the best chunksize is1M, and it won’t increase with the file size. In addition, this study under conditions with only two ordinary server data nodes can access the data with speeds of up to80M/s through experiment data analysis. If we improve cluster conditions (such as high memory, high bandwidth, Multiple Cards, more data nodes, etc), storage capacity and speed can have a big degree of ascension. So it can realize the efficient storage of mass astronomical data. And cloud storage is such a platform with integration of the resources and data nodes, thus we can deduce that cloud storage is the necessary trend for mass astronomical data storage.At last, the thesis is concluded and future work is proposed.