Research On Data Outlier Detection And Modal Selection Methods In Geotechnical Engineering

Field and indoor test data in geotechnical engineering are the basis of engineering design,construction and evaluation.The existence of abnormal data often misleads the determination of parameters such as design and construction.Data anomaly detection is the most basic but extremely important work to ensure the safety and reliability of engineering.In many practical engineering projects related to geotechnical,some data can be obtained directly through experiments,but the acquisition of some data often requires a lot of time and cost,so it needs to be estimated indirectly through existing experimental data.In addition,simple models and the selection of complex models is a difficult task and often depends on subjective judgments.Complex models usually fit the data well,but the robustness of the estimated model in the presence of modeling errors and measurement noise may not meet the requirements.Therefore,the research on anomaly detection and modal selection for geotechnical engineering data has important research significance and application value.Anomaly detection and modal selection of data in geotechnical engineering has become popular in recent years.Existing anomaly detection algorithms need to continuously adjust the training model,which is likely to cause problems such as low detection accuracy and waste of resources.The method is easy to overfit,resulting in poor prediction performance,so further exploration of more suitable anomaly detection algorithms and modal selection methods has become the focus of research.The main work of this paper is as follows:(1)Analyze the existing typical anomaly detection and modal selection methods in geotechnical engineering,introduce the relevant theories and techniques of anomaly detection and modal selection in detail,and point out their advantages and disadvantages respectively.On the problem of anomaly detection,the idea of screening by two steps is proposed to improve the accuracy and robustness.In terms of mode selection,a mode selection method with a penalty term is selected to ensure that the final model can tolerate modeling errors and measurement noise within an acceptable range.(2)Because the characteristics that geotechnical engineering data is easily affected by external factors,such as environment and weather,this paper proposes an algorithm using the double screening mechanism of "artificial rules + inspection model".This method adopts the idea of two-step screening.First,through human intervention,the initial model is selected from five aspects: probability-based,global distance-based,local density-based,linear-based and integration-based,and the voting ratio is determined to perform preliminary segmentation of the data set.Five different models are integrated,and then the dataset is classified twice to improve the accuracy and robustness of detection.To evaluate the effectiveness of the model,the machine learning test dataset(shuttle dataset)in the UCI database is used for test analysis.The results show that when using this model for anomaly detection,the precision rate reaches99.99%,and the recall rate reaches 96.99%,which is much higher than other detection algorithms and has great potential for outlier detection.Finally,this method was applied to the data analysis of the Junchang Tunnel of the Cenxi-Shuiwen Expressway,and the abnormal detection for the monitored surface deformation,foundation settlement,cracks and water level data was carried out.(3)In model selection,a complex model class is often more suitable for the data than a simple model class with fewer variables,but it may not provide better predictability.Based on this,this paper applies the Bayesian information criterion model selection algorithm,the complex parameterized model class is penalized by adding a penalty term to prevent overfitting of the model and improve the prediction accuracy.The model with the largest posterior probability,that is,the smallest BIC value,is selected as the final model class.The uniaxial compressive strength is used for model selection and the selected model class is compared with other regression models,and the results show that the algorithm outperforms other models in prediction performance.