Design And Implementation Of Knowledge Management System Based On Hadoop

Research institutes accumulated a rich set of valuable data,information andknowledge in the long-term scientific research.It’s a powerful measure to accelerate thepervasion and integration of the mutual knowledge between reasearchers and enhancethe original innovation and integrated innovation by collecting,managing,processingand using these knowledge effictivelly.It is badly in need of developing a informationservice platform of knowledge management to bring better development opportunitiesfor Research institutes.Hadoop distributed computing platform appeared with the emergence of cloudcomputing technology.A feasible way is provided to solve the problems above forresearch institutes. The storage capacity and computing power of research institutes canbe promoted by applying Hadoop to knowledge document storage.China Academy of Engineering Physics are facing huge information datacontent,too many file types,and it costs too much to search and obtain information andknowledge.A knowledge management system based on Hadoop has be designed anddeveloped in this thesis to slove mass storage and processing problem for researchinstitutes.This thesis chooses HDFS to be the underlying file system in knowledgemanagement system，and uses MapReduce as a data-processing tool after in-depth studyand research on HDFS and MapReduce distributed computing frame.This thesisanalysised and researched the architecture and storage principle of HFDS,thearchitecture and implement of MapReduce programming model,the architecture ofLucene full-text retrieval framework,and SSH layered architecture model.Based on theanalysis and research of the feature of knowledge document data processing,theprinciple of full-text retrieval,the work mechanism of log analysis and the arithmetic ofpersonalized document recommendation,this thesis presented some measures tooptimize and improvement the system.It also designed the knowledge managementsystem based on Hadoop,including business,logic,data and deployment structure of thesystem.This thesis designed and implementation full-text retrieval,log analysis andpersonalized document recommendation as the big three function modules byprogramming and seting up Hadoop cluster,hardware and software environment.Inaddition,paging bean and data persistence class was designed to page and accessdatabase.At last the system is proved to be feasible and reliable by testing function modules,keeping data good fault tolerance,security and stability.