Analysis And Simulation Prediction Of MBR Membrane Fouling Data Based On Hadoop

Membrane bioreactor (MBR) is a wastewater treatment technology with the combination of membrane separation technology and biological reactor technology, which has great advantage and huge development space in sewage treatment. Experimental research shows that membrane fouling severely hampered the development of MBR, membrane fouling directly causes the decrease of the flux. In order to get better monitoring of membrane fouling, we can determine the main factors influencing the MBR membrane flux through the analysis and study of MBR membrane fouling mechanism from the perspective of the membrane flux. The real data of the factors which affected the MBR membrane flux was obtained by and using experimential and actual MBR process.In order to solve the storage and analysis of the data of the MBR membrane fouling, the mainstream of today’s big data Hadoop loud platform was used. Hadoop is a reliable, high fault-tolerant and scalable distributed computing platform, its core including distributed file systems (HDFS) and computing framework (MapReduce). Hive, based on the advantage of Hadoop, is a kind of data warehouse storing data on HDFS. We could use it to build distributed mass MBR membrane fouling storage and analysis platform which realize the management of the data, finally. Hive has very high delay because it is based on Hadoop’s MapReduce computing framework. We use the Impala to do real-time query and analysis. But Impala does not support all Hive support function, and rely on the Hive. Finally the way of combination both was adopted, using the Impala to do real-time analysis, using Hive to do batch operations. Given the high latency of the Hive, in order to obtain a better user experience, the TCP/IP communication and Java multi-thread technology for Linux standard output and error output was used to realize the Hive query and analyze the progress of the monitor.MBR data management was realized by using Hadoop platform. In addition, in order to realize the simulation of the membrane flux prediction, we introduce least squares support vector machine (LSSVM). The particle swarm algorithm(PSO)was used to optimize the parameters of LSSVM. To achieve a better prediction of membrane flux, the first principal component analysis was carried out on the MBR membrane pollution data, extracting important factor, and then as input layer data of LSSVM, build a model, train model and simulation and predict. Finally, in order to compare the effectiveness of the model, which was compared with the BP neural network prediction model, results show that LSSVM has higher precision.