Sub-Image Storage Method For Astronomical Image Data In The Cloud

With the construction of astronomical observation equipment and the implementation of large-scale sky survey projects,the image data of astronomical observations has showed fast growth.What followed was the urge requirements of storage and retrieval of massive astronomical image data.The popularity of emerging technologies such as big data,cloud computing,and virtualization has led to widespread attention in cloud-based services,the huge astronomical data began to migrate to the cloud environment.However,the “pay-as-you-go” fee model makes astronomical workers pay high fees.The users usually use sub-images to do research,and the storage and transfer cost of the whole raw image are high.Therefore,a sub-image storage service that let users use the retrieval service efficiently while paying less is in demand.The data shows that the frequently used data only accounts for about 20% of the total data,and the rarely used data is about 80%.Therefore,using different performance storage media for tiered storage is the mainstream research direction of current data storage problems.However,most of the storage strategies in the current study are applied to storage media with limited erasure and write life,so the goal of these storage strategies is to maximize the hit rate or minimize the retrieval response time,and the cost is not taken into account.Aiming at the actual needs of cloud environment storage,this paper proposed a method for storing and retrieving astronomical sub-image,archive storage index and request data index are built to accelerate retrieval process.MinCT storage method is proposed,which uses the hybrid storage mode of archival storage and standard storage in cloud storage,and establishes the storage model of multi-constraint target based on the cost and retrieval response time At the same time,this paper proposed a MinCT?GA algorithm based on genetic algorithm to solve the established model.Finally,the sub-image data that may be requested again by the user is maximized in the standard storage,so that the cost and request response time are relatively balancedIn order to fully verify the feasibility of the model and algorithm,this paper carries out experiments under different request loads.At the same time,the performance of MinCT?GA algorithm and the traditional storage strategy according to the access frequency,storage cost and retrieval cost are tested in many aspects.The experimental results show that compared with the storage strategy of storing the first 20% data according to access frequency,MinCT?GA is 20.61% more expensive,but the request response time is reduced by 33.75%;the request response time is 0.47% higher than the storage strategy of storing the first 60% data according to access frequency,but the cost is reduced by 60.04%;the response time and cost are better than the rest of the storage strategy according to the storage access frequency,storage cost,and retrieval cost.