Binocular Vision Positioning Technique Used For Cotton Picking Robot

Compared with other robots,the working conditions of agricultural robots are more variable,non-structural environment mostly,which is very challenging.As a result,the intelligence level needed in agricultural robots are very high,and more researches about machine vision and artificial intelligence have come along to make agricultural robots more intelligenced.In order to provide the mechanical arm of the cotton picking robot with the needed movement locus parameters,a cotton distance measuring device based on binocular vision with a full implementation of SIFT algorithm was introduced,which realized the positioning of all cotton on the plant,whose accuracy was high enough to provide the mechanical arm with the needed parameters to determine its movement locus.Under indoor environnent,capture cotton 3 pair of cotton images with the control of projector flashlight.Take the first pair of cotton images for an examole,segment the unneeded backgrounds.Turn the RGB images into gray scale and enhance the gray value to make the cotton more obvious,then sharpen the edges and pretreatments of cotton images were finished.Down-sample the image 4 times and blur the image under 6 different Gaussian kernels to form a scale space.Calculate the Difference of Gaussian(DoG)of Gaussian images and acquire the extrema of 3×3×3 neighboring pixels within neighboring scales,thus 1634 and 1581 SIFT key points were detected in left and right images respectively,after sub-pixel implantation and edge points discard,1529 and 1493 SIFT key points in left and right cotton images were finally detected.All these key points were invariant to rotation,translation,zoom and affine,which were suitable for the match of cotton images.Calculate the gray gradient modulus value of the 4X4 seed points in 8 directions within the key point neighborhood,and the 128-dimensional SIFT descriptor of each key point was acquired.As to all the SIFT key points in the right image,select the dimension with the maximum variance,and calculate the median value of this dimension,find its corresponding key point and split the other key points according to its median value,repeat this step and the KD tree was built,As to every SIFT key point in the left image,search its potential matches(probably more than one)in the binary tree of the right image until its leaf node was found,and save the brother nodes found along the path,establish priority sequence with BBF(Best Bin First)and expand from the brother nodes to their leaves,find the nearest and second nearest neighbors according to the similarity degree of the 128-dimensional key points between the potential matches until the sequence was empty or the algorithm exceeded its 200 times constraint,and if the ratio of the nearest and second nearest neighbors is below 0.49,then consider the nearest neighbor as a matched point to this left image point,Repeat this step and 172 pairs of rough cotton matches of key points in two images were acquired.There was a possibility that there might be wrong matches among rough matches.In order to eliminate the wrong matches,estimate fundamental matrix F with RANSAC algorithm and recover epipolar geometry constraint,during each sampling,use 8-point algorithm to compute an initial F,calculate the distance from every point to its corresponding epipolar line and count the ones within the threshold 1.5 as inliers.Repeat this step and choose the F with the most inliers or the least error in case there were more than one F with the same inlier number as the final output F,and the corresponding inliers were called refined cotton matches.After implementation of RANSAC algorithm we got 151 pairs of refined cotton matches,with the error being 0.8018,and there were no wrong matches in the refined matches,which helped make the results of cotton 3D reconstruction more accurate.Set the optical center of left camera as the origin of the world coordinate system and calibrate the cameras with Zhang Zhengyou calibration method.Get essential matrix E according to F and epipolar constraint.Process singular value decomposition to E and the camera’s external rotation matrix and translation vector were recovered.To this point,2-dimensional cotton image coordinates could be transformed into 3-dimensional coordinates based on Structure From Motion algorithm,and the 3D positioning of cotton point cloud on the plant was realized.At last get the 3-dimensional coordinate values of every cotton and calculate their centroid coordinate values.Result shows that all cotton are all successfully 3D positioned.After 3 experiments,the average error are 0.0393m,0.0378m and 0.0382m respectively compared with manual measurement,which proves the calculated data are valid and this binocular vision system is reliable enough for practical application.