Lightweight Improved YOLOv5 And Real-Time Detection Of Road Damage

Road damage detection is the most important work in road maintenance.Chinese road coverage is wide and the situation is complex,and the operation and maintenance cost of professional road damage detection equipment is high,which poses a great challenge to road maintenance work.At present,a light,fast and low-cost road damage detection method is urgently needed.As the performance of smart phones and other mobile devices has made remarkable progress,smart phones and other mobile devices have been gradually applied in the field of road damage detection due to their advantages of low cost and portability.Therefore,this paper focuses on the real-time detection of road damages based on smart phones and other mobile devices.In order to complete the real-time detection task of road damages,this paper firstly uses YOLOv5 network,combined with appropriate initial Anchor,CIoU loss function and Diou-NMS to improve the accuracy of the network and speed up the convergence.Due to the complexity of the YOLOv5s network,the YOLOv5s directly deployed on mobile device such as smart phone cannot meet the requirements for real-time detection of road damages.This paper attempts to use MobileNetV2 module,ShuffleNetV2 module and PP-LCNET module to lightweight the YOLOv5s model,so as to improve the model detection speed and ensure high accuracy of the model,so that it can achieve real-time detection speed.Pp-lcnet-yolov5 obtained by lightweight of YOLOv5s model based on PP-LCNET module reaches 0.519 and 0.576 in RDD-2020 data set and local data set mAP respectively,achieving the best balance of speed and accuracy among the three lightweight methods in this paper.In the process of road data collection by mobile devices such as vehicle-mounted smart phones,the same road damage object may appear in consecutive multi-frame images at the same time,resulting in many redundant information in the detection results.In this paper,an object-based method of continuous multi-frame road damage reduction is proposed.Firstly,the parameters of the mobile device are calibrated.Then,the geometric relationship between the calibrated mobile device and the road surface is used to screen the detection results.Then the road damage coordinates in continuous multi-frame images are unified.Finally,the same road damage object in the continuous multi-frame image is screened out and one object is reserved to complete the task of removing the redundant detection results by using the characteristics of the same road damage object in the continuous multi-frame image.In order to collect road images every same distance in the process of vehicle driving.The average accuracy and recall rate of road images collected from 10 road sections reached 0.781 and 0.765,respectively.Due to the limited field Angle of the camera,mobile devices such as vehiclemounted smartphones cannot photograph complete longitudinal cracks.Therefore,cracks in continuous multi-frame images need to be matched and splicted into complete cracks.In this paper,inverse perspective transformation is used to transform the orthographic image to remove the perspective deformation of longitudinal cracks and extract the fracture curve.Then,a two-stage matching method from coarse to fine based on curvature similarity is proposed.After the fracture curve was divided into a series of overlapping sub-curves,the curvature pair was used to describe the sub-curves as feature vectors,and the kD-tree nearest neighbor matching algorithm was used to match the feature vectors quickly.According to the constraint of spatial continuity of longitudinal cracks in two consecutive road images,the interval of coarse matching and segmentation is gradually reduced,and the number of normalized cross-relations between sub-curve descriptors is iteratively increased to achieve fine adjustment of the matching results of longitudinal cracks.Finally,according to the matching results,the cracks in the continuous multi-frame image are spliced into intact cracks.The experiment was carried out with different types of continuous longitudinal cracks in the campus of Wuhan University as the object.The minimum error was 0.688 pixels,and the error of fine adjustment was 24.19%lower than that of rough matching on average.When the standard deviation of gaussian noise increases to 2 pixels in the simulation experiment,the error increases by 1.083 pixels only.By comparing the proposed method with SIFT algorithm,the proposed method can match successfully in 10 groups of experiments,while THE SIFT algorithm has completely wrong matching results in two groups of experiments.