The Research And Design Of A Storage And Distribution Architecture For The Innovation Knowledge Based On HDFS

The application of innovative knowledge has received increasing attention. A highlighted issue is the conflict that the innovative knowledge needs sharing in the whole society while the sensitive knowledge need privacy. Only by sharing the innovative knowledge community-wide, the reusing value of innovative knowledge can be maximized. But knowledge owned by some communities or organizations may have distinct characteristics and some private knowledge is inappropriate to be shared. Also by the constraints of resources and performance, these organizations also face the problem of unable to contact to all of the shared knowledge. Therefore, it is necessary to design a distributed innovative knowledge sharing management framework, through which the gathering, processing, storage and distribution task could be done for innovative knowledge to promote innovative knowledge sharing. When dealing with the massive innovative knowledge, traditional storage structures aren’t able to meet performance requirements. HDFS(Hadoop distributed file system) with its high performance and fault-tolerance and other advantages has become the industry’s standard model in the field of cloud computing and storage. But it shows its insufficient in mass small files storage, and the primary node’s single point of failure issue, which means it is difficult to provide an efficient and reliable storage service.In this paper, a flexible distributed cloud computing architecture model is designed and implemented on the platforms of innovation knowledge cloud. This model builds the relationship between the innovation knowledge cloud and the innovation bases, designs the collection and dissemination process of the innovative knowledge, and a formal specification is given to filter out the required knowledge from the complete set of shared knowledge in the knowledge distribution phase. By introducing the package file and distributed indexing service, the efficiency of small files storage is improved. A high availability and high resource utilization architecture is achieved after the optimization of the name-space backup and failure recovery services in HDFS.The innovation knowledge cloud provides a flexible communication channel for innovative knowledge. Through the theoretical derivation and experimental results, a conclusion can be seen that this model is capable to solve the contradiction between knowledge sharing and privacy, has the character of feasible and flexible, and, to some extent, provides certain ability of prediction and recommendation for the updating process for innovative bases. The improved HDFS provides an efficient, reliable storage support for the innovative knowledge, and can also be applied to other similar storage areas.