Solid Waste Object Segmentation Based On Multimodal Information

With the continuous deepening of China's socialist construction modernization,more and more construction waste is generated from the renovation of old cities and infrastructure construction.It is an excellent economic development mode to protect the environment by recycling waste generated during construction and converting it into resources and energy.However,manual sorting has problems such as slow efficiency,serious pollution,and great harm to people.At present,the industry is exploring an effective automatic solid waste sorting system for building solid waste based on robotic arm grabbing.Image segmentation technology is a necessary component of such systems.However,the environmental factors of the industrial site cause the color of the solid waste object to be seriously degraded,which will affect the final solid waste object segmentation?This topic mainly studies and optimizes the segmentation of solid waste objects from two aspects: deep convolutional neural network segmentation algorithm based on multimodal information and random field adaptive algorithm based on convolutional conditions.The specific work completed is as follows:Aiming at the serious deterioration of the color of building solid waste images,the serious stacking of solid waste objects,and the large number of solid waste objects,a deep neural network method based on cross-channel fusion of RGB information features and depth information features into multi-modal features is proposed to solve the problem of solid waste object segmentation.On the basis of the proposed algorithm,the energy dependence of the conditional random field is used to calculate the pairwise correlation of feature adaptation,and the accuracy of the algorithm is further improved.The main flow of the algorithm is as follows: Firstly,in the scene with severe color degradation,the color image and the depth map are used together as the input of the deep convolutional neural network,and the deep convolutional neural network is used for high-dimensional feature learning,and the label allocation probability of each pixel is obtained by the softmax classifier;Then,based on the new energy function,a fully connected conditional random field is established,and the global optimal solution is obtained by minimizing the energy function to segment the image,thereby generating an independent segmentation block for each type of solid waste object;then using the local contour information to calculate the depth gradient,to achieve accurate segmentation of solid waste objects of different instances of the same category.On this basis,for the current state of slow algorithm,the improved full connection condition random field,using the convolution condition random field,limits the pairwise correlation between the binomial calculation pixels and other pixels in the conditional random field to the convolution kernel size,which greatly reduces the size.The complexity of calculating potential pairs is greatly reduced.Then,based on the graphics processor to accelerate the calculation of the new energy function,The algorithm takes less than one order of magnitude,and the segmentation accuracy is improved.On the construction waste image test set,our method achieves 92.15% Mean Pixel Accuracy and 91.00% Mean Intersection Over Union.In addition,compared with some excellent semantic segmentation algorithms,the experimental results show that the proposed method obtains better performance and improve segmentation accuracy.The algorithm proposed in this paper can segment and classify most construction waste object effectively at the same time,and provide accurate contour and classification information of the construction waste object to a construction waste automatic sorting system,so as to facilitate the automatic grasping construction waste by the robotic arm.