Scattered Parts Bin-Picking System Based On Binocular Vision

In recent years, with the escalation of the aging process and the decrease of labor population in China, Industrial robot has been widely used in industry of aerospace, automotive manufacturing. However, industrial robot is not enough Intelligent, so we have to equip industrial robot with sensors to improve its intelligence. One efficient method is to equip robot with visual sensors. Visual sensors could get the information of environment. It highly improve the intelligence of robotic system.In the field of robot pickup, monocular vision can solve the problem of planar pickup. Binocular vision can get 3D information from 2D image, it can solve the problem of 3D location. This paper develop a scattered parts bin-picking system based on binocular vision and industrial robot. We use binocular vision to reconstruct point cloud of parts, and use template matching method to recognize and locate the scattered parts, finally drive the robot to the right position and gesture to pick up the parts. The main job as follow:First step is to design the hardware and software of pickup system.Second step is to 3D reconstruct the parts based binocular vision. Firstly, I chose the put means of binocular cameras based on the requirement. Secondly, I Use Zhang's method to calibrate two cameras based on binocular cameras math model, and calculate the intrinsic parameter, extrinsic parameter, and the position relation of two cameras. Thirdly, I calibrate the relation of robot coordinate system and camera coordinate system. Fourthly, I use structure light method to code the parts. In this paper, I mention a improved method to segment the structure light image due to uneven illumination. According to the code image, we can match the same code area of two image. Fifthly, we can 3D reconstruct the point cloud of parts according to principle of triangulation. Lastly, I measure the maximum error of 3D reconstruction is 2.3665mm by chess board.Third step is to use template match to recognize and locate the parts. Firstly, I calculate the normal of point cloud by PCA algorithm and use the region growing method based on normal information to segment the point cloud. Secondly, I chose the ISS algorithm to extract the key point and chose the SHOT algorithm to describe the feature of key point through the contrast experiment. If the two feature descriptor is closed enough, I consider two key points to be the matching key points. Thirdly, I use Hough method to estimate the rotation and translation of two point cloud based on the matching key points. Lastly, I use ICP algorithm to fine match the rotation and translation of two point cloud.It takes 3.8s to match 247 points.Last step is to design the robot pickup plan. The plan divide the process of pickup into two parts, one is template library building in offline, another one is parts recognition and pickup. In offline stage, system obtain the point cloud from different view, and calculate the key points and feature descriptors of key points, and drive the robot to the pickup position and gesture. Lastly, write down the key points, descriptors, pickup position and gesture.On online stage, system segment point cloud into several pieces, get the highest one to match the template, and estimate the rotation and translation of two point cloud. Use this rotation and translation system can calculate the robot pickup position and gesture. Through experimental verification, the success rate of first pickup is 58.3% and success rate of second pickup is 86.8%,16 parts take 676s.