Research On Anomaly Detection And Data Quality Assessment Of Bridge Health Monitoring Data

Bridge health monitoring system is running in order to collect sensor data from those sensors installed in the main part of bridge and monitoring the bridge status.It provides scientific decision-making for management and maintenance of bridge through the analysis of monitoring data,so as to avoid bridge accidents.In the long-term operation of the health monitoring system,there will be some abnormal data with diversity different abnormal degrees which is caused by sensor faults and transmission line interference during data collecting,transporting and storage.These abnormal data will seriously affect the results of data analysis,so the data processing technology based on data anomaly detection and data quality assessment become the basis and the key to the utilizing of monitoring data scientifically and effectively.This dissertation does some research about abnormal data detection and data quality assessment processing method by analyzing the characteristics of abnormal data.The main contents are as follows:1.For abnormal data detection,a pattern anomaly detection method of single variable based on KNN distance and a multivariate time series anomaly detection method based on covariance matrix and singular value decomposition are proposed.The pattern anomaly detection method of single variable mainly used to detect temperature,wind,humidity and other environmental monitoring data,the change of this kind of data cannot be reflected by the changes of structural data such as strain and deflection.In this method,the original data sequence is firstly compressed and segmented based on important data points,which generates multiple time subsequences.Then,the method calculates the similarity distance between subsequences according to the similarity measure of time series.Finally,the KNN method is used to select the abnormal data series.The multivariate time series anomaly detection method mainly used to detect structural data such as strain and deflection,the changes of those data is affected by environmental factors and the correlation between the various types of data is relatively stable in the long term.This method firstly uses covariance matrix to calculate the correlation between multivariate data,then uses singular value decomposition to obtain feature vectors and the contributions of variables to the correlation of covariance matrix,finally the extended Frobenius Norm Distance is used to calculate the similarity distance of multivariate time series and the KNN method is used to select the anomaly pattern.2.For data quality assessment,a method based on K-means classification is proposed.This method combines the concept of data change rate with cluster method K-means to bridge data clustering.Through dividing it into several clusters,a part of abnormal data may be clustered as a part of independent data which we can possibly recognize out and the proportion of abnormal data use to judge the data accuracy.Finally,the clustering results of multiple sensors are combined to total statistical assessment of data quality.In this paper,the algorithm is verified by the real bridge data of a bridge in Chongqing,which shows that these algorithms are effective in detecting various abnormal data and data quality assessment.