Research On Shelf Product Semantic Segmentation And Pose Estimation Based Hybrid Fully Convolutional Autoencoder

Robots are increasingly deployed in unstructured and complex fields,including homes and various mall and supermarket scenarios.In order to perform complex tasks autonomously,reliable environmental awareness is essential.Different tasks may require different perception capabilities.For more complex interactions,such as finding a specific product on a supermarket shelf with hundreds or thousands of products,high-precision semantic segmentation is a key factor necessary for a robot to find a specific product on a complex shelf.In this paper,aiming at the problems of robot information statistics in the process of identifying goods in unmanned supermarkets,such as the obscured two-dimensional code and the bar code of the recorded product information,it is difficult to statistic the goods.Then,the position and pose estimation method of point cloud registration is used to obtain the product information and its spatial pose,thereby ensuring the integrity of the information statistics and improving the robot's ability to sense the environment.First,this paper researches the semantic information of shelf goods.By constructing a high-speed,low error rate,and high recognition rate supermarket shelf environment semantic recognition framework(hybrid full convolutional autoencoder),the three major categories of supermarket shelves Commodities(bottled,canned,and bagged)are semantically segmented.The framework introduces a full convolutional neural network and a stack sparse autoencoder structure.The FCN network is used to generate a thermal high-dimensional feature map of the goods on the shelf.At the same time,the segmented features The graph is up-sampled.Finally,during the up-sampling operation,the convolutional features are refined with the Stacked Sparse Coding Algorithm(SAE)and the image boundary details are preserved,so that the classification result is more accurate.Second,using the proposed hybrid full-convolution self-encoder semantic segmentation framework,the semantic segmentation map obtained after the three major categories of shelves are semantically segmented is combined with the depth information under the Kinect camera to determine the point cloud information of the products on the supermarket shelves and estimate the products.The real-time attitude of the point cloud is lower than that of the traditional ICP point cloud registration algorithm in the point cloud registration process.This article downsamps the Gaussian filtered point cloud to make the point cloud registration process faster and more accurate.Finally,In order to verify that the method proposed in this article can achieve high-precision semantic segmentation and an effective algorithm for estimating the pose of the product,this paper performs a grab simulation experiment on the mobile robotic arm in the ROS-kinetic environment,and verifies this article through a large number of experiments The proposed hybrid full-convolution self-encoder semantic segmentation framework compared to other semantic segmentation frameworks for the accuracy of product grabbing and enables the AGV car to perform positioning and navigation on a two-dimensional map established by itself.Due to positioning errors,this paper also verifies it through experiments.The experimental comparison of the proposed semantic segmentation framework with other semantic segmentation frameworks when the mobile robot arm reaches the target point is verified,thereby verifying the effectiveness and robustness of the method.