Research And Implementation Of Data Placement Strategy In Cloud Computing

The emergence of information technology has changed the scale and scope of information systems that generate large amounts of data every day,opening the field of big data processing and analysis.The public cloud environment based on virtualization can not only meet the growing data storage needs,but also meet the heavy and large-scale computing needs of big data applications.However,Moving user data from a private storage system to a public environment for storage and processing presents new challenges.The problem of cloud resource access efficiency is a very important issue,which directly affects the user experience of the quality of service provided by cloud providers.In addition,security issues,especially data privacy,are also of great concern to users.However,due to the complexity of the cloud infrastructure,the heterogeneity of cloud resources,and the multi-tenant characteristics of the cloud environment,it is very difficult to solve the above problems.Therefore,it is necessary to design a new cloud storage model to accommodate the diversity of cloud infrastructure and usage.This thesis firstly designs a new cloud storage architecture based on the characteristics of multi-tenancy in the public cloud environment.The architecture is divided into user layer,data management layer and storage layer.This thesis introduces the concept of local indexing at the user level to speed up the indexing of user data.In order to maximize the value of the local index,this thesis optimizes the cluster metadata organization form,and groups the metadata by users,which greatly improves the efficiency of concurrent indexing.Considering the availability of data and balancing load of storage node and minimizing the data indexing time,this thesis firstly establishes a mathematical model,and then designs an effective placement algorithm.It guarantees the storage node's balanced load,data availability and minimizes the indexing time of the data.In order to adapt to the dynamic demand of users for data,a dynamic copy-based mechanism is implemented to provide users with the best data retrieval path based on real-time server load.Finally,in order to improve the security of user data,this thesis adds a security module,which is mainly used to encrypt and decrypt the data in the storage cluster and manage the secret.This thesis implements a high-performance cloud storage cluster based on the open source distributed storage system MooseFS,and introduces the local index security module into the system.At the same time,the thesis replaces the original mechanism in MooseFS with our data placement algorithm,dynamic copy strategy and data location strategy based on server real-time performance.Finally,this thesis builds a corresponding test environment,and tests the availability,stability,and read/write performance of the cloud storage system in different scenarios,and also compares the testing result with the native MooseFS.It turns out that the performance of our cloud storage clusters is higher than MooseFS in many scenarios.