| In the era of economic and trade globalization,railway container trade is becoming more and more frequent.However,in the railway container terminal,the positioning of the spreader and the container is still assisted by the baffle,which is not only inefficient,but also easily damages the surface of the container.Therefore,it is urgent to realize automatic spreader and container alignment.The key to automatic alignment is to determine the center of the lock hole from the container image and obtain the corresponding three-dimensional coordinates of the center.In this paper,based on deep learning,the spreader and container visual alignment technology is studied,and the main work and results are as follows:(1)In view of the fact that the keyhole only occupies a small part of the container image,and there are many interferences between the spreader and the container,a two-step method for determining the keyhole pixel center is proposed.The first step is to detect the bounding box of the keyhole from the container image,and the second step is to segment the keyhole from the keyhole bounding box,and determine the keyhole center by the the keyhole centroid.Then,the container images under the alignment condition are collected to make two datasets of container and keyhole.For the keyhole dataset annotation,a third-order Bezier curve is introduced to label the keyhole contours.(2)Taking the container image as input,the YOLOv4-tiny network is used to detect the keyhole bounding box.Considering that only the keyhole is detected during the alignment of the container and the spreader,and the size of the keyhole bounding box only changes within a small span,the YOLO head is improved and the anchor is redesigned.Since the keyhole bounding box detection emphasizes speed rather than accuracy,the channel pruning method Network Slimming is used to lighten the network.The keyhole frame detection accuracy of the lightweight network YOLOv4＿prune＿0.4-tiny meets the alignment requirements,and its real-time performance is comparable to that of the anchor-free Center Net network,better than the two-stage Faster-RCNN network,and the network complexity is much less than both.(3)Using the keyhole bounding box as the input,use the EGNet network to segment the keyhole.For accurate keyhole segmentation in real time,a suitable backbone network is selected from Vgg16＿improved,Res Net50 and Mobilenet to provide keyhole features for EGNet.In view of the simplicity of the keyhole feature,the depth of the EGNet network is reduced.Considering that the effective receptive field of the deep reduction network is small,the segmentation of large-scale keyhole images is not ideal,and the input image reduction strategy is adopted.To further improve the real-time performance of keyhole segmentation,the network width is cropped.To compensate for the impact of width cropping on keyhole segmentation accuracy,a batch normalization layer is added after each convolutional layer.Finally,the improved network Light-EGNet＿bn is obtained,and its keyhole center positioning accuracy is much better than Canny operator + ellipse fitting and keyhole center positioning method based on the bounding box of the keyhole.(4)Automatic alignment of spreader and container based on binocular vision.After completing the selection and deployment of the binocular camera,the ideal model of binocular camera ranging is obtained through Zhang Zhengyou’s calibration method and Bouguet stereo correction algorithm.The three-dimensional coordinates of the center of the keyhole are determined by using the disparity map generated by the SGBM algorithm and the principle of binocular ranging.Finally,according to the camera deployment,the matrix relationship between the camera coordinate system and the spreader coordinate system is established,so as to determine the orientation of the spreader and the container,and complete the automatic alignment of the spreader and the container.Through experimental verification,the two-step method proposed in this paper has an accuracy rate of 100% in detecting keyhole bounding boxes,and the pixel error of locating the keyhole center is only 1.27 pixels.The total time on Intel-I7 CPU is 208 ms,and the total number of parameters is only 5.36 M,which can accurately locate the container in real-time.When the vertical distance between the binocular camera and the keyhole is within 0.5m-1m,the positioning error of the keyhole center in the direction of the camera coordinate axis is no more than 3mm. |
