Study On 3D Reconstruction For Plants Based On Autonomous Navigation Robot In Facility Agriculture

Plant measurement system based on autonomous mobile robot has become one of the hotspots in plant phenotype research due to its advantages of flexible movement,low cost of arrangement,high degree of automation as well as intelligence.This paper developed a practical3 D reconstruction robot system of plant in facility agriculture based on SLAM and deep learning.This system can reconstruct the point cloud model of plant automatically.Growth parameters such as plant height and leaf size can be measured from the model accurately.The main contents and conclusions are as follows:(1)This paper combined RIA-R100 mobile platform and TIM561 Lidar to realize the robot autonomous navigation based on Cartographer after compareing two SLAM algorithm.The linear error and turning error of navigation were 3.10% and 1.45%.(2)According to the pinhole camera model,the 3D point cloud corresponding to the single frame image was generated from the color image and depth image,and the Kinect V2 camera was calibrated to improve the accuracy of point cloud generating.The camera pose of each frame was calculated by ORB-SLAM2,and an adaptive reconstruction frame selection algorithm was proposed to fuse muti-frame.Finally,the 3D point cloud of the scene was reconstructed.(3)By comparing two deep learning approach of image segmentation,a Mask R-CNN +CRF method was built to plant image segmentation,and the IOU of the method is 0.9253.Then deblurred the motion blurred image by Deblur GAN and the proposed blurred image selection alrigothm,which improved the PSNR of the test dataset 8.66%,the feature points of the real blurred image 50.00%,the feature matching number 17.46%,and improved the IOU of manmade blurred image 9.93%,and improved the accuracy of pose estimation of ORB-SLAM2 on the blurred KITTI dataset.Thus,the accuracy of pose estimation and image segmentation can be imporved by this method.(4)Combined with debluring,camera pose estimation,plant image segmentation,single frame point cloud generating,multi-frames point cloud fusion,point cloud denoising and point cloud smoothing,a 3D reconstruction method of plant is proposed.Combined with autonomous mobile platform,built a 3D reconstruction robot system of plant,and tested in the laboratory environment.Firstly,a Kinect V2 depth camera was mounted on the robot’s arm and captured the image of the plants,so the plant images were captured by the robot automatically in 53 seconds,with the 3D reconstruction method,the 3D point cloud model of ten Phalaenopsis was reconstructed in 82 seconds.The plant height,leaf width and leaf length were measured from the model,and the errors were less than 8.2%,the RMSEs were less than 6.7 mm.The 3D reconstruction robot system proposed in this paper has the function of automatic information collection,and the construction of plant shows favourable accuracy and time saving,thereby demonstrating good practicality and prospects for application.