Research And Application Of Data Pre-processing In Chemical Mining

In process industry, the technics processes are very complex. As the production statuses that associate with real-time data and relational data are in the constantly changing, so the chemical engineers need tools based on MES for real-time database and relational database data mining, parameters computing and conditions analyzing in order to guide the adjustment of operating conditions. While, running real-time data is the basis to run data mining tools. The data mining results are based on the accurate measurement of parameters required, the correction of data on-line collection directly affect the results of data mining system.The chemical process instruments prone to malfunction as they often need to work in high temperatures, vibration, corrosion and other harsh environments, that wll result in data mining system collecting wrong data; In addition, the data may also be subject to interference, drift and measurement of the environmental impacts, so that the measured real-time data are not accurate ones. Incorrect measurement data often result in serious consequences. In this context, this thesis combined with the actual situation of a large state-owned chemical plant explore various types of data pre-processing methods, and obtain a pre-processing algorithm based on SVM regression.With this algorithm, we can reduce nosie of real-time data effectively, realize and correction the error data, and we can also fill the missing data with corrected ones. It is effective algorithm in improving the quality of real-time data.The real-time data pre-processing system was built with the obtained algorithm. It used .NET development tools, applied ADO technology, XML technology, Matlab COM component technology base on VS2005 development platform, and the PI real-time database of OSI and SQL Server relational database of Microsoft as the underlying data sources. The system can pick and store the real-time data, at the same time, it can pre-processing these data and identify the bad ones with the algorithms. Finally, the real-time data pre-processing system integrated in data mining systems and other information systems associated with the real-time data.The real-time pre-processing system that based on process industry MES has effectively improved the quality of real-time data, and provides a support for the chemical process data mining and other real-time data analysis systems. It increases the manufaction monitoring level up-to-date decision-making level of process industry, and makes the ERP and MES more usefully.