Key Technologies Of Data Placement For Meteorological Proprietary Cloud

In recent years,as the process of meteorological informationization speeding up continuously,the meteorological data in the meteorological industry keeps increasing.The problems of storing the meteorological data have been explosive appeared.How to process and store meteorological data reasonably and effectively has become a key to promote the informatization and intelligence of meteorological data.Cloud computing,which is an important technical method of providing resources,put a large number of tasks and data in the cloud for computing and storage to save the local resources.Besides,the information service in the cloud computing center can be obtained at any time.Therefore,building a meteorological proprietary cloud platform release the problem of the meteorological data storage effectively and promote the development of the meteorological industry at the same time.Currently,the researches on the data layout of meteorological proprietary cloud focus more on the cloud platform construction.During the process of data layout,the impact between the cloud platform construction and layout optimization technology in the meteorological proprietary cloud data center is often ignored.In order to satisfy this demand,the main challenges of the data layout of the meteorological proprietary cloud are following: 1)During the construction of the meteorological proprietary cloud data center,the internal constraint relationship between data layout and energy consumption is not analyzed.2)In the meteorological cloud platform,the internal correlation between meteorological big data and tasks is ignored.3)In the mixed cloud computing environment,based on the protection of privacy,a more comprehensive implementation method is needed to combine the resource scheduling technology and data layout of the cloud data center.In view of the challenges in the above-mentioned meteorological cloud data layout,this paper mainly focuses on the data layout method and key technologies for the meteorological proprietary cloud.The main work includes the following four aspects:(1)In order to support the meteorological service system and various data applications,a meteorological cloud service framework is designed for the national meteorological industry in this paper.The framework has four layers,including hardware layer,data layout method layer,collaborative layout technology layer,and mixed cloud platform management layer.Specifically,the hardware layer serves as the physical resource guarantee for data layout.The data layout method layer is designed to achieve the data layout of energy consumption awareness.The collaborative layout technology layer is designed to achieve the collaborative layout between energy-efficient meteorological services and data.The hybrid cloud platform management layout is designed to the efficient data layout in the hybrid cloud environment.(2)In order to take both the data access time of meteorological tasks in the meteorological cloud platform and the energy consumption of the meteorological cloud platform into consideration comprehensively,an energy-efficient data placement method based on the meteorological proprietary cloud platform is proposed in this paper.To be specific,a resource utilization model based on the platform of meteorological proprietary cloud is proposed firstly,and the resource utilization rate of physical machine in the process of meteorological task execution is analyzed and calculated.Secondly,a data access model based on the meteorological proprietary cloud platform is proposed,and the data access time of the task set which accesses the meteorological data set is analyzed and calculated.Finally,an energy consumption model of meteorological proprietary cloud platform is proposed,and the energy consumption of the physical machines,virtual machines and switches in the platform is analyzed and calculated.Finally,NSGA-III(Non-dominated Sorting Genetic Algorithm III)is leveraged to achieve the global optimal data placement strategy.Then,in order to obtain the optimized data placement strategy,SAW(Simple Additive Weighting)and MCDM(Multiple Criteria Decision Making)are used to evaluate the solutions.The goal of reducing data access time and energy consumption of the meteorological cloud platform is achieved.(3)In the meteorological big data environment,meteorological data resources are characterized by large volume,dynamic opening and multi-source,etc.Based on the cloud environment,this paper makes a reasonable optimization between the meteorological tasks and data collaborative placement.Specifically,the average data access time model,the total energy consumption model of the network equipment and the load balance model of each node is proposed in this paper.NSDE(Non-Dominated Sorting Differential Evolution)is selected to optimize the layout strategy.Finally,based on SAW and MCDM,the utility value of the optimization result is calculated,and the individual with the highest utility value is selected as the final optimization result to realize the collaborative placement of meteorological tasks and data in the cloud environment.(4)From the perspective of big data privacy protection in practical applications,the rental cost and execution time of the public cloud services rented by meteorological operations and meteorological data layout in the public cloud environment,as well as the task execution time in the private cloud environment and the performance indexes of the cloud data center,are studied and analyzed in this paper,To be specific,at first,the cost,access time and energy of the big data in the meteorological mixed cloud are analyzed.Furthermore,for the placed privacy data in the meteorological cloud,the greedy data layout strategy is designed by analyzing the Fat-Tree network topology to reduce the overall energy consumption of the meteorological proprietary cloud and optimize the timeliness of task access data.