Research And Development Of Intelligent Inspection Methods And Equipment For Bridges Based On Computer Vision

Bridges play a crucial role in China’s transportation network,supporting the country’s economic development and facilitating people’s daily commuting.China has achieved remarkable accomplishments in bridge construction in the past few decades.However,occasional bridge accidents and collapses highlight the need to ensure the safety of this extensive bridge infrastructure,making it a prominent research focus in the field of civil engineering.Bridges undergo gradual deterioration and experience unexpected events during their long-term operation,resulting in a decrease in durability and load-bearing capacity that can jeopardize their safe operation.Regular inspection of bridge health status,along with appropriate repair and strengthening measures,is essential to maintain their safe operation.Traditional manual bridge inspection methods,currently widely employed,are becoming inadequate due to their low efficiency,high cost,and associated risks.Based on the aforementioned considerations,this paper presents a comprehensive approach to bridge inspection,encompassing three levels: bridge damage detection,deformation inspection,and 3D geometry inspection.The study focuses on enhancing bridge inspection through two main aspects: "optimal measurement" and "optimal analysis." The "Optimal Measurement" and "Optimal Analysis" levels aim to improve the automation,intelligence,efficiency,and precision of bridge inspection."Measurement optimization" involves the development of intelligent inspection equipment tailored to the specific data collection requirements of bridges.On the other hand,"analysis optimization" focuses on the design of stable,high-precision,and automated analysis algorithms to handle the vast amount of multi-source data acquired during the inspection.The main contents of this paper and its contributions are as follows:(1)Crack is the most common and most concerned type of bridge damage in damage inspection,for which a real-time crack inspection method based on wall-climbing UAV and smartphones is proposed.The method firstly addresses the contradiction between the small width of cracks and the large size of bridges by designing a UAV with both flight and wall-climbing modes of operation,thus enabling a quick rough inspection of the structure first in the form of flight inspection,and then executing a refined inspection close to the adsorption for the location where the damage is found,so that even small cracks can be captured.Secondly,a crack image analysis method using a smartphone as the processing terminal is designed for the automated analysis of the video acquired by the UAV.The method uses a wireless video transmitter to transmit the UAV video back to the smartphone in real time,and performs real-time crack detection and fast width calculation by means of a lightweight crack detection network and width calculation algorithm ported to the smartphone.In a test for crack detection on the facade of a building,the proposed crack detection method achieves crack detection with 0.1 mm accuracy and real-time detection efficiency of 6 frames/second on the smartphone side.(2)Further,for the inspection of multiple types of bridge damages(cracks,spalling,rust and loose bolts),a multi-type damage identification and localization method based on a vision positioning UAV and edge computing is proposed.For the acquisition and pre-processing of damage images,the method firstly addresses the problem that traditional UAVs cannot be positioned due to weak GPS signals around the bridge,especially under the bridge,by designing an UAV system that replaces GPS positioning with stereo visual positioning.Then,for the problem of motion blurring and insufficient resolution of images caused by the moving of the UAV during inspection,a motion blur removal method based on optical flow estimation and an image clarification method based on super resolution are proposed.For damage analysis,a lightweight network with fast inference speed and low computational performance requirements is obtained by replacing the lightweight backbone network with the YOLO v3 network,and the network can be implemented on the UAV-mounted micro onboard computer for real-time damage identification after the training is completed in the established multi-type damage database.Finally,a damage localization method based on visual real-time map building and image matching is proposed so that the detected damage can be directly reflected on the structure.In a concrete cable-stayed bridge underside and tower damage inspection,the proposed method and system achieve inspection in non-GPS environment such as bridge underside,real-time identification of multiple types of damage at 17.5 frames/sec and automatic location of the damage.(3)There is also a need for internal information measurement in the damage inspection,for which a damage inspection method based on a collision-tolerant UAV combined with both visionbased and contact-based inspection is proposed.This study takes coating inspection of bridges as an entry point,its inspection includes two parts: apparent deterioration and coating thickness measurement.Firstly,for the problem of large magnetic interference and a large number of obstacles around steel bridges and the demand of contact inspection,a UAV system with 3D printed collision-resistant frame,UWB pseudo-GPS positioning and elastic force feedback robotic arm is designed to perform inspection,which can maintain a stable contact state by controlling the UAV through pressure feedback when contacting the bridge,thus enabling the magnetic eddy current probe on the robotic arm to measure the coating thickness.For coating inspection,a vision-based and contact-based combined method is proposed.Initially,a camera-based rapid visual inspection of the bridge coating is conducted,utilizing an anchor-free object detection network to identify the presence of spalling,hollowing,and fading of the coating in real-time.Then for the detected areas of coating deterioration,a contact non-destructive inspection method is applied to its perimeter to measure the thickness distribution.The proposed method was applied to the bottom coating inspection of a steel pedestrian bridge,and the error of the proposed method was less than 5%compared to a manual handheld measurement device.(4)For the bridge deformation measurement,to overcome the problem that the accuracy of traditional vision-based measurement method decreases significantly when measuring large-span bridges at a long distance,a bridge deformation measurement method based on UAV airborne dual cameras is proposed.The core of this method is to propose the use of coaxial dual cameras with telephoto and wide-angle cameras to simultaneously capture the deformation parts on the bridge girders and the stable points of the bridge piers.The base point motion of the UAV is removed from the theoretical derivation perspective using the bridge pier as the reference object.For the deformation measurement method,to overcome the issue that traditional methods based on image feature matching are susceptible to light changes and local occlusion to lose the target,a deformation measurement method that uses deep learning object detection to locate the measurement target and Kalman filter prediction to enhance the measurement robustness is proposed.The proposed method is applied to the deformation measurement of a suspension bridge under traffic load,and the results demonstrate the advantages of the proposed method in terms of high accuracy and stability.(5)To further extend deformation measurement to full-field deformation measurement,a deformation measurement method is proposed.It utilizes a panoramic camera for static structure deformation measurement and a simultaneous tracking method based on multiple UAVs for measuring the deformation of moving members.The key for deformation measurement method based on a panoramic camera is to propose a projection segmentation and independent calibration method of the panoramic image to transform the severely distorted panoramic image into a normalview image,and then a method based on a small target detection network-assisted positioning and sub-pixel linear detection is proposed to address the problem that traditional deformation measurement methods are very easy to lose the target due to the lack of small texture of nodes in the scaled model.The node center coordinates are accurately extracted based on small target detection network-assisted positioning and sub-pixel linear detection,which can solve the node displacement stably and accurately.For the deformation measurement problem of large-size member lifting in bridge construction,multiple UAVs are used to track the structure in simultaneous flight around the structure,and obtain the deformation of the sides of the structure at the same time,and use a high real-time deformation calculation method based on Ar Uco coded targets to obtain the deformation of each side,and finally recover the two-dimensional deformation into three-dimensional deformation by the field-of-view matching relationship of multiple UAVs and the three-dimensional geometry theory.The above method was applied to the deformation detection of a large-size structure scaled-down model and the deformation measurement of a largevolume steel component lifting of a bridge under construction,and the comparison with the results of total station and other equipment verified the practicality of the proposed method.(6)The 3D geometry inspection of bridges is crucial during both the construction and service phases.One of the challenges lies in efficiently building accurate 3D models of large-scale bridge structures.To address this issue,a bridge 3D reconstruction method utilizing unmanned airborne lidar-camera fusion and a fast segmentation method based on image-point cloud matching is proposed.The method first builds a set of low-cost 3D scanning equipment fusing non-repetitive scanning lidar and camera,and integrates them on the UAV for dynamic scanning,in which the inertial integration method is used to remove the motion distortion of the point cloud,the targetbased calibration method is used to calibrate the camera and lidar,and the vision-lidar tight coupling method is applied to establish the visual odometry to reconstruct the 3D point cloud,so that the 3D point cloud model of the bridge can be established in real time while flying.In addition,for the automatic segmentation of large-scale point clouds,a two-level image segmentation network from coarse to fine is adopted to quickly segment bridge components from images,and the segmentation structure of 2D images is projected into 3D point clouds according to the calibration relationship between cameras and LIDAR,so as to quickly segment bridge components and extract bridge 3D line shapes.The proposed method is applied to the shape measurement of an in-service concrete arch bridge,and the accuracy of the proposed method is verified by comparison with a high-precision 3D laser scanning device.The proposed method is applied to the alignment measurement of an in-service concrete arch bridge,and its accuracy is validated through comparison with a high-precision 3D laser scanning device.