Research Of Ecological Risk Prediction Based On Spark Framework

In recent years,with the increasing emphasis on the protection of ecological environmental,the change of ecological environment and the negative impact of human activities on the ecological environment have attracted more and more attention.At the same time,the protection of resources and environment and ecological civilization construction have become an important part of our country’s development and construction in the new era,and ecological risk assessment and prediction is an important part of it.Through the establishment of a quantitative evaluation index system,the scientific and objective assessment of the progress and effectiveness of ecological environment and ecological protection has been carried out,it provides decision-making and data support for environmental protection and ecological restoration,mitigation and adaptation for risk,ecological risk prevention and control,and research of ecological risk management mechanism.This paper takes the ecological risk prediction under the Spark big data cluster as the core research goal.Based on carding and analyzing the research results in the field of ecological risk assessment,an ecological risk assessment index system based on the Importance of Ecosystem Service Functions(IESF)and the Sensitivity of Ecological Environment(SEE)is constructed which is named IESF-SEE index system,and a PressureStatus-Response(PSR)ecological risk assessment model based on Entropy Method(EM)is established,the algorithm of ecological risk prediction based on deep learning neural network is realized,the parallel processing of ecological risk prediction algorithm based on Spark framework is explored,and the experiment and analysis of the model algorithm are designed and operated.This paper provides a feasible solution to the problem of long training time in the deep learning task of big data set.The main research work and results of this paper are as follows:(1)By learning and sorting out the theory and model of ecological risk assessment,and combining with the actual situation of the ecological environment in the research area,the evaluation index system based on IESF-SEE and the ecological risk assessment model based on EM and PSR are established.The model not only considers the effect of multi-risk sources on ecological environment,but also makes up the deficiency of the traditional evaluation model in the interaction of multi-risk sources,it reduces the workload and subjectivity of determining the weight artificially,thus expanding the scope of application of the model and improving the efficiency of the evaluation work.(2)Through the research on the application of deep learning in ecological risk prediction,the characteristics of Recurrent Neural Network(RNN),Long Short-Term Memory(LSTM)and Attention Mechanism(AM)are summarized,and the AM is brought in based on the LSTM model.Att LSTM shows great potential in time series prediction.This paper designed and implemented the ecological risk prediction algorithm of Att LSTM based on Python programming language,and compared the single LSTM with the Att LSTM,the Att LSTM model is proved to be feasible and effective.(3)The effectiveness of the model and the execution efficiency of the parallel processing mode in Spark cluster environment are verified by experiments,and the experimental results are analyzed and discussed.The experimental results show that the deep learning platform based on Spark cluster has a good speedup to the parallel processing of prediction model,and can effectively improve the parallel processing ability of big datasets.(4)On the basis of the research results of this paper,the Decision-Support System for Ecological Protection of Qinghai Urban Cloud Bigdata is designed and developed,which integrates and encapsulates the algorithm model of ecological risk assessment and prediction,it has certain applicability,expansibility and transplantability for ecological risk assessment and prediction,and provides distributed deep learning tasks in other fields with certain reference value.