Research On Plant Point Cloud Segmentation Based On Deep Learning

As an important food source and an important part of the ecological environment,plants are closely related to human survival and development.Improving the efficiency of plant phenotype acquisition is an urgent problem for modern smart agriculture.High-throughput plant phenotype data can help researchers to quantitatively analyze the growth status of plants.Traditional methods of plant phenotyping are inefficient,tedious and destructive to plants,affecting their growth.Although image processing-based plant phenotypes are more efficient and less destructive,there are limitations in obtaining plant phenotypes from two-dimensional images due to the complex structure of plants and their growing environment,as well as the existence of certain occlusion between plants.Therefore,this thesis conducts a research on plant point cloud segmentation from the perspective of 3D point clouds,and the main work is as follows:(1)To address the problems of low efficiency of point cloud acquisition methods or expensive point cloud scanning equipment.In this thesis,we propose a semi-automatic and inexpensive method for acquiring plant point clouds.The method automatically acquires multi-view image sequences of plants by the plant image acquisition platform built in this thesis,uses the incremental SFM(Structure From Motion,SFM)algorithm to reconstruct the multi-view image sequences of plants in three dimensions,and uses point cloud filtering to denoise the reconstructed plant point clouds,and obtains the plant point cloud dataset after manual annotation.(2)A graph convolutional neural network based plant point cloud segmentation model(GCNN)is proposed.We propose a graph convolution method that acts directly on the point cloud,firstly,we use FPS(Farthest Point Sampling,FPS)to sample some of the centroids,and then find the neighboring points by KNN(k-Nearest Neighbor,KNN),each centroid and its neighboring points form the local graph structure of the point cloud,and the points within the local graph structure are aggregated into the features of the local graph of the point cloud by MLP(Multi-layer Perception,MLP)feature upscaling.Multi-layer Perception(MLP)features are up-dimensioned and aggregated into features of the point cloud local graph by the maximum pooling operation.The graph convolution method fully captures the point-to-point information between point clouds.Experimental results show that the GCNN model achieves 81.9% IOU on the plant point cloud dataset,which is 6.3%,1.2%,and 0.6%better than Point Net,Point Net++,and Point CNN,respectively,verifying the effectiveness of the model.(3)The multi-scale dynamic graph convolutional network model(AS+MSDGCNN)based on adaptive sampling is proposed based on the optimization of the plant point cloud segmentation model based on graph convolutional neural network.For the problem of noise in point clouds in different scenes,an adaptive sampling method is proposed,which is an attention mechanism-based method to obtain different weights through feature learning and weighted summation of the points sampled by FPS to realize the adjustment of the original sampling results and reduce the interference caused by point cloud noise;for the problem that traditional graph networks build graphs statically through geometric space,a multiscale In order to further improve the feature extraction ability of the model,we propose a multi-scale dynamic graph convolution module,which constructs local graph structures according to the feature space at different scales,fully captures different perceptual field information of the point cloud,and dynamically updates the construction of local graphs in each subsequent layer of graph convolution;for the MLP feature learning method in point cloud deep learning networks,we propose a joint perception model,which fully integrates low-dimensional feature information and high-dimensional To further improve the model performance,a joint perception model is proposed to fully integrate low-dimensional feature information and high-dimensional feature information.3.6% and 3.0%,respectively.The ablation experiment analysis of the three modules of AS+MSDGCNN model shows that all three modules can improve the segmentation performance of the model,among which the adaptive sampling module and the multi-scale dynamic graph convolution module improve the effect more obviously;in the public dataset Shape Net,the average IOU of AS+MSDGCNN reaches 82.8%,and 2/ The best performance was achieved in the point cloud category of 16.Finally,the AS+MSDGCNN model is used to segment the plant point clouds and measure the segmented leaf area,and the error is maintained at 12.9% compared with the traditional manual measurement method.