Design And Implementation Of Deformable Bag Identification And Positioning System Based On Monocular Vision

In the traditional steel-making pipeline,the ambient temperature is very high and the working environment is poor.The workers need to wear professional and heavy high-temperatureprotective cloth when they feed the cover agent.It is extremely inconvenient and efficiency of the feeding procedure is low.The feeding task is relatively simple but it needed to be repeated usually.A automatic system of coverage agent bag grasping and feeding need to be built to confirm to keep the procedure time and volume controllable.Focusing on the task of capturing and delivering deformable bags,this paper mainly solves the problem of the identification and positioning of the deformable covering agent bags in bad conditions of steel-making process.Considering the accuracy and time efficiency,this paper designs a matching method and search strategy based on the local affine template of the target single edge point.The bags can be recognized and located quickly and accurately.A multilayer perceptron network and a multi-scale fully convoluted residual network are designed.Local features of single point and global image features are extracted respectively.Estimation of the grab depth of single deformable bag is achieved by training the networks.The main contents of this paper are as follows:1.Considering the change of lighting in the factory,the leakage of the covering agent powder caused by the improper packing of the bags,the scattering and deforming of bags caused by the stacking and transporting process,we design an identifying and horizontal positioning algorithm for deformable bags based on template of the edge features.We improve the similarity measure rule of single-edge point matching on the basis of traditional translation matching process which is based on templates.We make the set of the nearest k edge points of single-edge point as the rigid representation of it.In the matching process,the affine transformation templates are established to compute the single-point similarity measuring scores.At the same time,the image pyramid search strategy corresponding to this method is designed.The deformable covering agent bags can be matched exactly,while the matching time is reduced.2.In this paper,considering the requirement of the depth estimation of covering agent bags,we propose a multi-layer perceptron based on single point features and a multiscale full-convolution residual network based on the whole color image features.By constructing the multi-layer perceptron,we establish a non-linear model of the key-point local features and its own depth.Using SIFT in combination with RANSAC,several key points on the surface of the bag are extracted.The local area features of the key points are extracted by Law’s mask and krisch direction operator to build a feature vector.The surface of the deformed bags are reconstructed by the depth estimation process based on multi-layer perceptron.It provides reliable depth data for the feeding task.Adopt the convolution layer and the pool layer,combined with the residual module,a full convolution residual network from coarse to fine are constructed.By extracting the global characteristics of the input color stack image,the corresponding depth image is got.An improved projection module is designed by joining the quick connection,which reduces the network parameters and achieves a depth image with the same resolution as the input image.Combined with the identification and horizontal positioning results of bags and the region growing algorithm,the single bag area is extracted based on the estimated depth image which realize the process of single bag depth estimation.3.In this paper,a large number covering agent stacking pictures of steel-making pipeline were collected.The experiences of deformable bags recognition and horizontal positioning based on template matching and depth estimation based on monocular vision were conducted respectively.A set of depth data sets of deformable bags in real environment was established by collecting stacking data with Kinect.The depth of individual deformable bag was estimated through network training,with a root-mean-square error of about 3 mm.To sum up,in view of the task of automatic grasping and delivering of the deformable covering agent bags,a method of identifying and positioning the deformable bags based on monocular vision is proposed.It can deal with issues such as light change,bag obscuration,characteristic defacement and bag deformation in the actual steel-making environment without human intervention,which provides reliable data for the task of bag grasping and delivering.