High Precision 3D Point Cloud Reconstruction And Segmentation For Plants’ Fruit Leaf Recognition

Plant phenotypic parameters are important basis for judging the merits of varieties in the selection and breeding of crop varieties.However,most existing methods for measuring plant phenotypes are completed manually,which is time-consuming,labor-intensive,and has large errors.At the same time,image-based phenotypic measurements often cause information loss.The three-dimensional plant model is an important method for highprecision automatic measurement of plant phenotype and an important basis for simulating the growth process of the crop.At the same time,the threedimensional reconstruction of the plant hardly loses the three-dimensional morphological information of the plant.The morphology of the plant is complex and changeable,and the characteristics of flexibility and deformation make the 3D reconstruction and splicing of the plant more difficult.The interleaving of the leaves,fruits,stems and other organs of the plants makes the registration and phenotypic measurement of point cloud data difficult.Plant 3D point cloud model segmentation is an important method for phenotypic measurement using plant 3D models.Complex,non-standardized plant 3D model segmentation lacks efficient algorithms.Therefore,in order to shorten the crop breeding time and improve the accuracy of crop phenotypic measurement,it is necessary to study the reconstruction method of plant three-dimensional point cloud model and develop high-precision plant point cloud segmentation algorithm.Based on the projector and camera,a gray-coded line structured light3 D scanning device is constructed in this paper.Based on the equipment mentioned above,a 3D reconstruction platform based on a turntable for small plant and a 3D reconstruction platform based on multiple perspectives large-scale plant are developed.We realized the fast and high-precision 3D point cloud acquisition of small plants.The problem of lack of effective 3D reconstruction methods for large-scale plants is solved.A series of algorithms for the stitching of plant point cloud sequences obtained from the above platforms is proposed in this paper.Aiming at the point cloud stitching of the 3D reconstruction platform based on the turntable,a simple and efficient rotation axis calibration algorithm for the turntable was proposed.The analytical solution of the pose of the rotation axis was given.Aiming at the point cloud stitching of the 3D reconstruction platform based on multi-viewpoints,2D tool and 3D tool are used to achieve the calibration of different structured light scanning equipment poses and mobile platform movement direction.The above-mentioned calibration results are used to realize the rough stitching of the point cloud sequence.After that,the ICP algorithm is used to achieve the high-precision registration of the plant point clouds.Finally,a software of plant point cloud registration was developed based on GPU for the relevant calibration of the above platform and the plant point clouds registration.A method based on active vision is used to solve the occlusion issue while collecting the plant information.An occlusion information completion scheme based on active vision was designed.At the same time,a six-axis robotic arm is equipped with a camera and hand-eye calibration is used as an active vision device.Based on the SHOT descriptor of Harris corners,the fruit is identified and the end position of the robotic arm is calculated.Completion of occlusion information through the camera at the end of the robot arm.The 3D point cloud segmentation of plants is studied in this paper.Leaf segmentation was performed on the plant’s 3D point cloud using the DBSCAN method and the 3D region growing algorithm.For the ear of rice point cloud segmentation,a rice ear point cloud dataset was produced.Based on 3D point cloud convolution/deconvolution,a convolutional neural network for plant point cloud segmentation is designed.The comparison experiments were implemented on other neural network models.The result has proved the accuracy of the designed model for segmenting point cloud of rice ear.The 3D point cloud reconstruction method and plant organ segmentation method proposed in this paper are of great significance for the analysis of the physiological and ecological structure of the plant,the measurement of the phenotype and the selection of the plant cultivar.