Research And Design Of Massive Image Cloud Storage System Based On Hadoop

In recent years, with the deepening of the continuous development andapplication of Internet technology, some large Internet portals and e-commerce sitesdeveloped rapidly, such as “Taobao”,“Jingdong”,“Amazon”,“Sina”，etc. Theseresource site occupied by a large picture, and the explosion of the number, nature andhas a high concurrent access. The face of a flood of pictures resources, build efficientlow-cost storage systems become software architect needs to be urgently addressedthe issue of how efficient storage and under the premise of how to meet the highconcurrent access.The emergence of cloud computing provides us with an idea, we can use throughthe analysis of distributed storage systems to solve the above problems. By analyzingthe massive image storage needs, as well as research on existing distributed systems,proposed massive image storage model based on cloud computing. This model isdeployed on linux machine clusters to the Hadoop HDFS-based and optimized toachieve high fault tolerance to provide reliable high concurrent access. Using the newdata structure, and the file name mapped to physical addresses, thereby providing agood read of the. While using HA architecture to ensure system availability.The content and innovation of this paper are as follows:First，by the analysis of the need for massive image storage and research onexisting distributed systems, storage model is proposed based on Hadoop; through theuse of Master/Slave architecture to achieve a low-cost computer clusters high faulttolerance and scalability on the deployment of the system;Second，through caching system designed to ensure the stability of the storagesystem; through the design load balancing to achieve the optimization of each storagenode.Thrid，Storing picture metadata is used in the Hadoop Hbase. By redesigning thepicture file name, so that the physical address of the same type of picture stored asclose as possible, thereby improving the efficiency of query.On the basis of laboratory equipment, we set up the system, through a series oftest data obtained from the analysis of the feasibility of the system, and the text of theproposed method is verified the effectiveness and practicality. In order to meet the massive image storage requirements, this paper designed anew type of mass image storage model. This storage model based on Hadoop relatedtechnologies, and has high fault tolerance, high reliability, high scalabilitycharacteristics. And by means of laboratory equipment, we deploy the system andconducted a feasibility experiment.