Design And Implementation Of Data Quality Governance System Of Industrial Internet Of Things

With the rapid development of the industrial informatization,the data volume accumulated by the factory has increased every day.However,the data quality problems of the industrial Internet of Things can be seen everywhere such as link problem,hardware malfunction and human factors,etc.The poor data may lead to industrial accidents or wrong decisions.The high-quality data is the precondition for giving full play to the efficiency of the industrial big data.Therefore,the thesis has designed and implemented the industrial data quality governance system as well as studied the data cleaning and the quality assessment algorithm by taking the main body of the industrial Internet of Things data——machine and equipment data as the breakthrough point.Firstly,in order to improve the industrial data quality effectively,the thesis has designed a set of complete data cleaning procedures,such as correction of invalid values,completion of missing values,merge of duplicate values and detection of outliers in proper order.The completion of missing values has been calculated with data of related dimensions comprehensively through LSTM algorithm for time series prediction.As for the outliers,both outliers and pattern exceptions are detected through different algorithm in accordance with characteristics of industrial time series data.The SNM algorithm is adopted to deal with the merge of duplicate values to ensure the accuracy of the detection while reducing the time complexity.Invalid values are corrected with Python functions.Secondly,the thesis has established the complete evaluation index system of the data quality in accordance with the characteristics of the industrial Internet of Things big data.After the evaluation index and rules are determined,the entropy method is proposed to calculate the weight of each evaluation rule of the index.Then the fuzzy analytical hierarchy process is used to calculate the weight of each index.Finally,the grey comprehensive clustering algorithm is adopted to finish the data quality comprehensive evaluation of the industrial data.Finally,the simulation experimental result is used to verify the effectiveness of the industrial data cleaning algorithm and the quality evaluation algorithm.Then the complete data quality governance system for industrial Internet of Things is established.The use case is applied to test and prove that the function of the system meets the requirement.The data quality governance system of industrial Internet of Things proposed in the thesis can provide support and guidance for improving the industrial data quality,and at the same time lay the foundation for further data analysis and mining.