Research And Implementation Of Urban Sewage Supervision System Based On Big Data

With the rapid development of China’s economy,people’s living and industrial water consumption increased significantly,resulting in more and more discharge of sewage,especially industrial waste-water.For this phenomenon,establishing urban sewage supervision system can effectively find problems in advance and solve the problem from the source,which is also the inevitable trend of solving this problem in the future.At present,the source of sewage is widely distributed,the discharge time is not fixed,and the monitoring and judgment indexes’ number of sewage is large.The traditional solution has the problems of long acquisition period,heavily high dimensional evaluation indexes,and low processing capacity of system.Regarding the issue above,this paper studies and implements a urban sewage supervision system,which is based on the deep analysis of the characteristics of sewage data and big data processing technology.The main work of this paper includes the following:1)Propose a real-time monitoring technology of sewage big data.Firstly,based on the idea of integrated learning Adaboost algorithm,use FNN_Adaboost model to predict the unknown complex waste-water index,which improves the prediction accuracy of unknown index.Then,on the basis of reducing the dimension of sewage data,by improving the k-means clustering algorithm to enhance the clustering effect of large sewage data.Finally,achieve real-time excessive monitoring of sewage big based on data stream processing framework called Spark Streaming.2)Design a storage system architecture for sewage big data.According to the large quantity and diversity of sewage data,use the master-slave architecture to build Hadoop cluster,and design sewage data storage model that is oriented column storage in HBase database.It is not only can store the massive sewage data,but also can improve the efficiency of data processing.3)Propose a forecasting analysis technology of sewage big data.Considering the difference between the discharge of sewage emitted by different factories,and the dynamic character of the sewage data,compare the results of the data prediction by the linear and nonlinear time series forecasting model after the serialization and analysis of sewage data,and then,select the optimal model to achieve the forecastinganalysis of sewage big data.Finally,verify the feasibility and validity of the technology by practical application.4)Based on the research of the above technology,this paper designs and implements a urban sewage supervision system based on big data.This system learn form MVC thinking,implement many modules including the data parse,data analysis and processing,data storage,data query,prediction alarm and so on.Not only can it supervise the urban sewage comprehensively and intelligently,but also can make big data correlation analysis,so as to promote the construction and development of the big data in urban environmental protection actively.