BIM-assisted UAV Safety Inspection And Automated Visual Recognition Of Hazards For Water Diversion Channel

Due to the complexity of geological,hydrological,meteorological and cultural condition along the route,the operation of long-distance water channel is threatened by many different types of risks and hazards,e.g.,ice jam,contamination and slope failure.An important prerequisite to mitigating the loss brought by such hazards is to regularly perform safety inspection and accurately recognize the occurred hazards in a timely manner.Current practice relies on onsite personnel to manually perform daily inspection,and structural health monitoring(SHM)systems to recognize potential or occurred hazards.However,manual inspection is a time-consuming process that has limited coverage and lacks an effective mechanism for information sharing to support immediate condition assessment.SHM systems can only be deployed in several typical cross sections along the route,and thus might fail to identify hazards and failures occurred between the cross sections.In light of the drawbacks of current practice,there is a critical need to provide new means and methods to improve the efficiency and coverage of safety inspection and hazard recognition for long-distance water channels.The development of emerging technologies such as unmanned aerial vehicle(UAV),building information modeling(BIM)and image recognition provides opportunities to renovate the practice of safety inspection and hazard recognition.UAV is endowed with the merits of high mobility and wide-angle view,and thus has a potential to improve the efficiency of inspection;BIM integrates multiple-source information of a project in a visualized way,and thus can be applied to support structural condition assessment;Image recognition techniques automatically process visual assets to identify target objects,and thus have a potential to provide an automated approach to identify hazards from aerial images of the whole route of the water channel.This research systematically explores the specific ways to apply the above technologies in the safety inspection and hazard recognition of water channels.The research efforts have been made according to the workflow of “Aerial image collection – BIM integration – Image preprocessing – Image recognition”,and have achieved several innovative outcomes with application values.The primary contributions and findings are listed as follows:(1)An augmented-reality inspection method integrating UAV and BIM was proposed.To meet the demands of long-distance water channel inspection,many UAV products have been investigated;The applicability of UAV has been validated in the inspection of longdistance water channels.A BIM model of a water channel is integrated with the dynamically collected SHM data to form a dynamic BIM.The dynamic BIM is then connectively animated along with the aerial video captured by the UAV.A case study implemented in northwest China indicates that the proposed method can effectively improve the efficiency of safety inspection and provides a visualized and integrated platform to support decision-making.(2)A BIM-driven region of interest(ROI)extraction method was proposed to process aerial images collected by UAV.This paper proposes to perform 3D registration into BIM by using the recorded locations and postures of the UAV;With the registered BIM,the regions of interest in the aerial images are then automatically extracted for further application without any prior knowledge.An improved algorithm based on the spatialtemporal continuity of aerial videos is also proposed to mitigate the extraction errors induced by GPS drifting and inaccurate registration.The proposed ROI extraction method paves way for subsequent image recognition for ice jam,foreign objects and slope failure.(3)The image recognition of ice condition in water channel was investigated.According to prior knowledge of river ice,this study proposes several image feature descriptors to discriminate different ice condition,which include a color feature descriptor St V,a texture density descriptor EP,and two texture orientation descriptors ?-EHD and ?-HOG.The degree of correlation between the proposed feature descriptors and the ice condition has been analyzed based on effect size.Using the correlated features as input,support vector machines are trained for ice condition classification.A case study was carried out for validation purpose,which demonstrated that the proposed method can recognize different ice stages and ice flow strength in images with a relatively high accuracy.(4)Research was carried out to detect,recognize and locate foreign objects in water channels from aerial images.An image detection algorithm for foreign objects was proposed by integrating SLIC segmentation and local binary pattern(LBP).A bottom-up“hierarchical voting” mechanism was developed to identify the specific categories of the detected foreign objects.In order to track the detected foreign object and determine its scale,methods of spatial localization and geometric feature calculation were proposed based on photogrammetry and georeferenced information provided by the aerial images.The proposed methods have shown robust performance in a case study.(5)The image recognition of slope failure was investigated for water channel.A workflow to recognize slope failure was developed,which comprises four steps,i.e.,SLIC segmentation,superpixel feature hand crafting,SVM model training and slope condition assessment.Multiple feature descriptors have been proposed for slope failure recognition,i.e.,LBP,EHD and HSV histogram.Among the proposed feature descriptors,“LBP+HSV”has shown the most robust performance in discriminating slope failure in a case study.The case study has also demonstrated the efficacy of the proposed method.