Research On Reliability Evaluation Method Of Big Data System Based On Fault Tree

In recent years,the explosive growth of data has raised the upsurge of big data research,making it the focus of attention all over the world.More and more industries and areas involve the processing and storage of big data,and all kinds of big data systems emerge as the times require.Reliability is one of the important indicators that affect the performance of big data systems.Therefore,it is very important to use the correct assessment method to analyze the reliability of the system.This thesis proposes a reliability modeling method based on fault tree for big data system,considering the lack of reliability models in big data system.And a reliability evaluation method for big data system based on Fault Tree Analysis(FTA)is proposed.Finally,the validity of the model and evaluation method is verified by experiments,which can provide reference for the fault treatment and quality assurance of big data system.The specific work is as follows:(1)This thesis summarizes the research status of reliability of big data system at home and abroad,and describes the definition of big data and reliability problem.This thesis introduces the architecture of big data system from hierarchical angle.(2)This thesis introduces the process and main steps of fault tree analysis,including qualitative analysis and quantitative analysis.The classification method of big data system failure is proposed,and the failure of big data system is classified and summarized.This thesis proposes a reliability modeling method of big data system based on fault tree,and constructs a fault tree model with big data system failure as the top event.(3)This thesis proposes a reliability evaluation method based on Fault Tree Analysis.The fault tree model is qualitatively and quantitatively analyzed by means of CAFTA(Computer Aided Fault Tree Analysis)reliability prediction analysis tool.(4)Build the Hadoop+Spark basic framework,and deploy upper Scan,K-means,and Page Rank applications to collect fault data.The validity of the model and evaluation method is verified by experiments,which can provide reference for the research and improvement of the reliability of the big data system.