Machine Vision Based Real-time Monitoring For Bridge Traffic Flow Load And Its Applications To Digital Twins

Bridge health monitoring is an important tool for the management,safety maintenance and condition assessment of existing bridge.With the increasing traffic load and the continuous aging of built bridges,the demand for bridge health monitoring systems in practical projects has become more and more urgent.However,there are still many problems in the existing bridge health monitoring theory,which greatly limits its application to practice.One of the most concerned problems is the incompleteness of monitoring data.Structural response monitoring is the common choice of current bridge health monitoring.However,as the number of measurement points that can be deployed on the actual bridge is too limited to meet the monitoring requirements for structures with a huge number of DOF(degree of freedom),the monitoring information of structural responses is seriously incomplete in practice.Although limited information of structural responses can be used to identify modal parameters such as structural frequency and mode shape or other structural indicators,when it is used to solve problems such as model modification(updating)and damage detection,it is inevitable to solve ill-conditioned equations,making the solution of the problem very difficult.On the other hand,in addition to the incomplete structural response monitoring,there is also a lack of applicable monitoring methods for bridge input loads.The monitoring of traffic flow loads,temperature loads and wind loads still remains at the realization of individual measuring points,and the changes of loads over time and space cannot be obtained.Therefore,the value of the obtained load monitoring data has not been fully used.Considering that traffic flow loads are the main live loads the bridge subjects in operational condition and abnormal traffic flow loads are also the main cause of bridge accidents,accurate observation of traffic flow loads is expected to make up for the lack of structural response information when other types of load effects are weak.The existing traffic flow load identification methods are generally based on the direct measurement of pacement weigh-in-motion(WIM)systems or the inversion from structural responses using mechanical equations.For these two methods,the former can only observe the moving loads at a few bridge sections,thilw the latter can hardly identify the load distribution in complex forms.Therefore,the first goal of this paper is to develop traffic flow load monitoring system that can monitor the whole bridge and adapt to various complex bridges and load forms.For the first goal,the main research contents and method innovations of this paper are as follows:1.A basic method for full-bridge traffic flow load(FBTL)monitoring based on the information fusion of WIM systems and multiple cameras is proposed.In order to overcome the shortcomings of the existing moving load identification methods,this paper applys the WIM system and multiple cameras to obtain the values and spatial positions of traffic flow loads,and fuses the load value and position information through time clues.A multi-view information fusion method based on the field of view lines is proposed to realize the real-time monitoring of the traffic flow load distribution on the whole bridge deck.2.A real-time,accurate and illumination robust identification algorithm for traffic flow load monitoring is proposed.The vehicle detection algorithm applied in the basic FBTL monitoring framework is difficult to adapt to the scene with drastic changes in lighting conditions.Also,due to the camera imaging error,the traditional vehicle detection algorithm targeting the whole vehicle will generate a large transverse error when the vehicle is closer to the camera.Therefore,this paper proposes a "vehicle-rear" dual target detection algorithm based on YOLO-v3,and develops a vehicle transverse position correction algorithm based on the detection results,which improves the illumination robustness and the identification accuracy of vehicle positions.3.A long-term FBTL monitoring algorithm with the ability of small target detection,occluded target tracking and fast vehicle position correction is proposed.Aiming at the problems of small target detection and continuous tracking of occluded targets in the long-term FBTL monitoring for long-span bridges,this paper improves the target detection method in the original system based on the YOLO-v4 network,and proposes a new tracking strategy that integrates vehicle trajectories and apparent features,and deduces a regression-implemented method for fast correction of vehicle position.Verified by real bridges,the improved system has basically possessed the long-term monitoring capability of the FBTL of long-span bridges.Based on the above three innovations,this paper develops a relatively perfect FBTL monitoring system,which is applicable to long-term or short-term monitoring of traffic flow loads for various types of bridges.Based on the proposed traffic flow load monitoring technology,this paper proposes the concept of bridge digital twins connected by traffic flow loads.The core of digital twin systems is the information interaction between the physical and digital models,that is,continuously updating the digital model with the measured data and feeding back the state changes of the real structure based on the digital model.However,in the field of civil engineering,the research on digital twin system has just started,and the research on the information interaction between digital and physical models is not specific and deep enough.In view of this,starting from the two basic tasks of operational condition monitoring and service performance monitoring for bridges,this paper takes load effect analysis and damage detection as examples to gradually discuss the basic method of applying the traffic flow load connected digital twin system to bridge health monitoring.First of all,for the monitoring of bridge operational conditions,this paper focuses on the short-term(real-time)and long-term(statistical)effects analysis of bridge structures,and proposes monitoring or early warning methods for abnormal bridge operational states that combines measured traffic flow loads and digital twin models.The main contributions and achievements are as follows:1.Based on the measured traffic flow load distribution,a digital end overturning risk monitoring and early warning system for continuous box girder bridges with single-column piers is proposed.Overturning accidents of single-column pier box girder bridges have occurred frequently in recent years,and the main cause of these accidents is abnormal traffic flow load queues.The research on this kind of overturning accident is still focused on the post-event analysis,and there is a lack of monitoring and active early warning methods for overturning risks.In view of this,this paper proposes a digital model of single-column pier box girder bridge in analytical form,and takes the measured load as the model input,and develops a digital-model-aided real-time monitoring and early warning system for the overturning risk of single-column pier bridges.2.An intelligent simulation method for traffic flow loads and a long-term effect analysis method for the cable-stayed bridge that is based on the lightweight digital model are proposed.Using the measured traffic flow load of a cable-stayed bridge,an intelligent traffic flow load simulation method based on visual measurement and traffic flow theory is developed;then a lightweight finite element model is established based on OpenSees;finally,the actual load distribution generated by the simulation is used as the information medium while the finite element model as the digital model to achieve long-term analysis of the load effect of the cable-stayed bridge.The experimental results show that the proposed load simulation method can reproduce the real load distribution more accurately than the traditional method,and the obtained long-term effect index has better robustness to noise.Then,for the monitoring of bridge service performance,this paper proposes two bridge damage detection methods and systematically demonstrates their feasibility and accuracy,considering the digital-physical information interaction and based on the framework of load-connected digital twin system.The main innovations are as follows:1.From the static point of view,a bridge digital twin model and damage detection method based on the measured displacement influence line are proposed.This paper firstly proposes a set of visual measurement methods for bridge displacement influence lines,and deduces the damage location and quantitative analytical expression for a simply supported beam based on the difference of displacement influence lines.Then,this paper uses the analytical expression as the digital model,the displacement influence line measured from the physical model is used as information medium,and the difference between the displacement influence line as the main indicator of information transmission,to realize the damage identification of the bridge structure.Experiments show that the proposed method does not need a benchmark model,can accurately identify multiple damages within a span of about 5/12 under the condition of a single measurement point,and can realize the quantification of multiple damages.2.From the dynamic point of view,a bridge digital twin model and damage identification method based on high-dimensional feature extraction are proposed.This paper proposes a digital twin model integrating deep learning technology and introduces its application in bridge damage detection.In this model,the traffic flow loads measured from the physical model act on the digital model as well.The digital model can generate a simulated response database of any scale through parameter adjustment.Based on the simulated response database,deep learning technology can be further applied to train a CNN model for extracting high-dimensional features of structural responses.The CNN model can directly predict the damage of the physical model according to the measured bridge responses,and realize the interaction between the physical model and the digital model.The experimental results show that the proposed framework can accurately locate the damage by using the acceleration signal when the structural parameters are unknown,and has the potential for damage quantification.In a summary,this paper proposes an FBTL monitoring system that can be applied to real bridges,which preliminarily solves the problem of incomplete monitoring information of bridge responses.Taking the traffic flow load as the link,this paper establishes better bridge digital twin systems,and systematically proposes new methods for load effect analysis and structural damage detection which are separately for the monitoring of bridge operational conditions and service performance.The feasibility and applicability of the proposed digital twin systems and analysis methods are fully verified through a number of field and laboratory experiments.