Glass Detection And Reflection Removal Methods Based On Deep Learning

Glass is ubiquitous in daily life,bringing convenience but also causing some negative effects.For example,photos taken through glass can be contaminated by reflections,which affects the visual quality of the images and reduces the performance of subsequent computer vision applications.Additionally,transparent glass can also interfere with the visual judgment of some intelligent robots,resulting in collision between robots and glass.Therefore,reflection removal and glass detection have important research value.Current reflection removal and glass detection tasks face many challenges.The existing reflection removal algorithms generally do not explore the reflection features,which may lead to the inability to accurately detect and remove reflections.In addition,the lack of a large amount of real-world data limits the realworld generalization of reflection removal.The existing glass detection algorithms generally rely on a single image,which limits the accuracy of detection.The ground truth of the glass also has the problem of inaccurate annotation.To address these issues,this paper proposes a method for generating virtual world glass scene data based on physical rendering,a single image reflection removal algorithm based on multi-scale reflection detection and a glass detection algorithm based on visible and near-infrared image pairs.The main contents include:(1)A virtual world glass scene data generation method based on physical rendering is proposed to address the issues of insufficient real-world reflection-removal data and inaccurate annotation of the ground-truth of glass detection.Compared with existing reflection removal data synthesis methods,the proposed method considers factors such as lighting,glass thickness,and shooting angle,generating a large amount of reflection removal data that is closer to the real world.Additionally,the glass detection data generated by proposed method can obtain the accurate ground-truth of glass without manual annotation,which greatly improves the efficiency of data collection.(2)Based on the observation that reflections on glass usually have different sizes,a single image reflection removal algorithm based on multi-scale reflection detection is proposed.The algorithm has three main modules.First,two parallel feature extraction modules that extract reflection and transmission features respectively from the input reflection-contaminated image.Then,this paper proposes a multi-scale reflection-aware transmission recovery module,which detects multi-scale reflections through different dilated convolutions and uses reverse attention mechanism to restore the transmission layer covered by reflections.Experimental results show that the proposed reflection removal algorithm can effectively remove reflections in images and outperforms state-of-the-art methods.(3)Based on the finding that reflections of the glass in near-infrared(NIR)images can be suppressed,this paper proposes a glass detection algorithm based on visible and near-infrared image pairs.The proposed algorithm takes RGB and NIR images as inputs.First,two independent Transformer encoders are used to extract RGB features and NIR features respectively.A multi-modal contextual comparison module is then proposed,which detects glass by comparing the reflection difference between RGB and NIR images.Finally,the glass detection results are recovered by progressive decoder.Additionally,a new RGB-NIR glass detection dataset is constructed.Extensive experimental results demonstrate that the proposed algorithm can accurately detect glass in different scenes,while also proving that near-infrared image is a powerful clue for glass detection.