Research On Cloud Storage File System With PSO Scheduler And Buffer Model For Interaction Intensive Application

With the significant development of network and hardware technology nowadays, on-line service providers intend to establish a concentrate computer cluster to gain the massive capacity of data manipulating and storing. Then, they lend this capacity to the outside society clients via matured networktechnology in order to satisfy the huge and increasingsocial requirement of information processing and thus make profit. This new pattern of providing and gaining computing resource is called Cloud Computing, which is basically built on the infrastructure of large-scale computer cluster.In another aspect, the innovative breakthroughs of the Cloud Computing model also bring some new challenges. For instance, its large scale and loose tight architecture can cause problems which cannot be detected in the traditional unique node'computing structure. The research described in this paper intends to tackle one of them that to improve the performance of dealing with interaction intensive task on a large scale Cloud Storage system.Based on the platform of Hadoop Distribute File System (HDFS), the research in this paper denotes a hierarchicalnamenodes approach and buffer architecture with Particle Swarm Optimization (PSO) scheduler in order to relieve the dramatically shrinkage of throughput when system encountersI/O interaction intensive applications. First of all, the unique namenode structure in HDFS is changed into two-level one which contains a master namenode as well as several secondary slave namenodes. Then, a tmpfs-based buffer infrastructure is deployed on each slave namenode. Meanwhile, due to the solution of original PSO has a high ratio to be trappedinto local optima, the algorithm denoted in this paper is modified to have the abilities of evolution grade evaluation and parameter self-adaption. The architecture presented in this paper has been deployed in a loose tight distributed cluster which composed by130commercial computing nodes as well as high performance routers. Various experiments, including the case of a real I/O interaction intensive application, the original and modified Project Montages, have been down to probe the performance increment brought by this model. The final results show that the hierarchical management and buffer architecture with SAPSO can significantly resist the throughputshrinkage when processing the interaction intensive applications.