Research On Depth Estimation, Instance Segmentation And Reconstruction Based On Visual Semantics

Robot vision system is a complicated system that simulates human vision.The depth estimation and instance segmentation of it are challenging visual tasks.Based on the results of advanced computer vision and deep learning in recent years,this paper discusses tasks such as depth estimation,depth image inpainting,instance segmentation and reconstruction,and proposes some new methods.This paper first proposes a Siamese branch network for binocular depth estimation.The connection between the identical Siamese branches and the 3D convolutional hidden layer ensures that the convolutional network can learn binocular parallax information,the binocular parallax information is the key to improve the robustness of the depth estimation method.Then,a kind of depth image super resolution network is proposed to reconstruct the results of the Siamese branch network with small resolution,and the high-resolution and high-precision depth estimation results are obtained.Experiments are designed to compare with other methods to prove the effectiveness of this method.Then,in view of the popularity of depth cameras in indoor robot vision systems in recent years,this paper proposes a method for depth image inpainting based on Full Convolutional Networks to inpaint the depth images which taken by a depth camera with a large area missing,and designed experiments to compare with traditional methods.Then this paper proposes a method of workpiece instance segmentation for non-mobile robots,and design experiments to show the accuracy of workpiece classification when several machine learning algorithms are used as the core algorithm.Then,a multitask learning network combining general instance segmentation and depth estimation is proposed for general instance segmentation and depth estimation tasks,which shows that multitask learning makes the information learned more robust.Finally,combining with the methods of depth estimation,depth image restoration and instance segmentation,this paper completed the instance semantic reconstruction of the workspace from the perspective of the robot.