Indoor Semantic Segmentation System Based On Fusion Of Image And Depth Information

Semantic segmentation task,as one of the fundamental vision tasks for machine understanding of the world,has attracted the attention of many scholars,engineers and technicians.There has been a lot of work on semantic segmentation based on RGB images.However,RGB information,as an inherent property of objects,is greatly affected by light and is difficult to distinguish in dark light or similar color situations,which poses a great challenge to semantic segmentation tasks.In recent years,with the development of depth sensor techniques such as camera arrays,TOF(Time of Flight),and structured light,more and more work uses depth information to compensate for the lack of RGB information,i.e.,using RGB-D data as input to accomplish the semantic segmentation task.In this paper,we analyze various works related to fusing RGB and depth information and find that they have problems such as large computational and parametric quantities.To address the shortcomings of the extant methods,this paper explores the fusion approaches and proposes two simple and effective fusion strategies.1)The GWConv(Geometry-weighted Convolution)module is designed to fuse RGB and Depth asymmetrically.Compared with other symmetric fusion approaches,this method can effectively improve the performance of the network under poor lighting conditions.And the method does not generate too much computation,which ensures the running speed.2)A Shape-Conv(Shape-aware Convolution)module is designed to extract shape features efficiently.This module splits the input features into position(mean)and shape(difference)component,and makes the convolution operation adaptively process both components,which has a great improvement on the final results.