Design And Implementation Of Data Storage Optimization Method In Cloud Environment

With the development of science and technology,diverse medical methods have led to an exponential increase in the size of medical data,which has brought huge challenges to the storage and access performance of medical systems.In the era of cloud computing and big data,many medical industry information systems cannot store massive data,and cannot carry highly concurrent access traffic.In order to achieve efficient data storage in the era of big data,this paper designs and implements a cloud platform-based radiotherapy data storage system based on the Hadoop cloud platform and distributed cache technology.A small file storage model of HDFS in the cloud environment is proposed to store unstructured documents,images and other data generated in medical informatization,a distributed cache model of Redis in the cloud environment is used to cache structured data generated by medical services.By analyzing the research status of related technologies at home and abroad,this paper applies HDFS distributed file system and Redis distributed cache technology to data storage,and designs a cloud platform-based radiotherapy data storage system.The system uses the HDFS distributed file system in the Hadoop platform to complete the storage of unstructured data.The My SQL database is used to provide structured data storage and persistence.The No SQL database Redis is used to provide data caching.The JAVA language-based Spring technology stack completes the programming of system business functions.Aiming at the storage performance problem of small files in native HDFS,compared with the existing small file processing schemes,a HDFS small file storage model with better storage performance and access efficiency was designed in combination with No SQL database.This model adds a small file processing layer on top of the HDFS system to improve the storage efficiency of small files from file preprocessing,file buffering,file merging,and file indexing methods.Based on this model,a distributed file storage service is provided for the system.Aiming at the problems of uneven data distribution and poor cluster availability in a distributed cache cluster,the existing load balancing and cluster availability schemes were compared,and a distributed and highly available Redis distributed cache model was designed.The cache model introduces a dynamic sharding strategy based on the consistent hash algorithm to prevent service interruption caused by load tilt.Based on the highly available solution of the Redis cluster,Zookeeper is introduced to centrally manage all cache nodes to achieve nodes automatic failover.Based on this model,a distributed cache storage service is provided for the system.Finally,the system designed in this subject is developed and tested.The test results show that the cloud platform-based radiotherapy data storage system designed in this paper can meet the actual needs of medical work,and the system has been deployed in an actual production environment for use,get tested in practice.