Research On Leaf Segmentation And Shading Completion Algorithm For 3D Phenotype Measurement Of Plant Seedlings

With the rapid development of modern information technology and the requirement for precise regulation of plant seedling breeding,it has become increasingly important to measure the phenotype of plant seedlings throughout their growth period.However,the current plant phenotype detection is relatively lagging behind compared with genomics research.A high-throughput,high-precision and nondestructive plant phenotype detection method is urgently needed to realize the monitoring of plant seedlings’ phenotypes during the whole growth period.Therefore,the research in this paper focuses on solving the problems in the process of 3D plant phenotype measurement,including the slow speed and poor quality of 3D reconstruction,the existence of missing point clouds collected by sensors,and the inability of seedling leaf area to be measured nondestructively.Specifically,the research work focuses on the following:Firstly,RGBD 3D reconstruction method,SFM multi-sequence image 3D reconstruction method and instant-ngp multi-sequence image deep learning based 3D reconstruction technique proposed in this paper were used for 3D reconstruction of single plants and whole tray seedlings of fruiting vegetables.The experimental results show that RGBD is a simple method to obtain 3D point clouds and can fully meet the high-throughput demand,but there are many limitations in the process of measuring the full phenotype of plants due to the lack of 3D point clouds obtained.And the SFM multi-sequence image 3D reconstruction method can obtain relatively clean and complete3 D point clouds,but the reconstruction time is too long,which is not conducive to the demand of high-throughput data acquisition.Finally,for the instant-ngp multisequence image deep learning 3D reconstruction technique proposed in this paper,the experimental results found that its quality is better than the SFM multisequence image 3D reconstruction method,and the reconstruction time is shortened nearly 30 times,which can fully meet the demand of high-throughput data acquisition.Second,this paper proposes an end-to-end point cloud completion method called PCA-Net,which differs from existing methods by directly learning the mapping between missing and complete point clouds,instead of requiring a "simple" network to generate rough point clouds and then a "complex" network to enhance local details.The point cloud is encoded into point cloud blocks by farthest point sampling and k-nearest neighbor algorithm,and then the depth interaction features between the missing point cloud blocks are extracted by an attention mechanism.In the decoder,a new learnable trilinear interpolation method is introduced in this paper to recover the missing point cloud details by combining the coordinate and feature information of low-resolution point clouds.Experimental results show that PCA-Net exhibits effectiveness and superiority in several challenging point cloud completion tasks,and also shows great versatility and robustness in real-world missing point cloud completion.Finally,this paper proposes a new framework for point cloud organ segmentation and leaf complementation aimed at achieving high-quality leaf area measurements of melon seedlings.The input of the algorithm is the point cloud data in the top view of seedlings captured by the Azure Kinect camera.The method in this paper improves the measurement accuracy of the acquired data in two ways.On the one hand,this paper proposes a method of neighborhood space constraint to effectively filter suspended points and outlier noise in the point cloud,which greatly improves the quality of the point cloud data.On the other hand,a new network named MIX-Net is developed in this paper to realize organ segmentation and leaf complementation of point clouds simultaneously by using a pure linear feature mixing mechanism.Experimental results demonstrate that for the seedling organ segmentation task,the method obtains a performance improvement of 3.1% and 1.7%compared with PointNet++ and DGCNN,respectively.Meanwhile,the R2 of leaf area measurement improved from 0.87 to 0.93,and the MSE decreased from 2.64 to 2.26.Therefore,the point cloud organ segmentation and leaf complementation framework proposed in this paper can be used to accurately measure the leaf area of melon seedlings and has potential applications in other point cloud complementation tasks.