Apple Tree Phenotypic Parameter Measurement Method Based On LiDAR And RGB Camera

Apple trees do not produce fruit for the first three years after planting.And farmers cannot judge the growth of apple trees by their fruit.In contrast to biochemical information,fruit tree phenotyping information is a readily visible exterior characteristic and offers a visual depiction of harvest quality and development data,making it simple to implement in production.A rapid,effective,and precise collection of large-scale fruit tree phenotyping information can support a management of smart agriculture,as well as effective control and precise application of water,fertilizer,and medicine.However,methods for gathering phenotyping information to assess quality fruit trees’phenotypic traits such as tree height and crown length and breadth still primarily depend on human sampling and measurement,which is time-consuming,labor-intensive,imprecise,and ineffective and restricts the advancement of smart agriculture.Large automated phenotyping information acquisition systems are being introduced into the market,but they are very expensive and frequently depend on ground pass circumstances,which makes it challenging to push them as a typical method of collecting high throughput phenotyping information.This paper uses apple trees as the research object and employs an unmanned aerial vehicle(UAV)platform to integrate Li DAR and RGB camera data for generating color point clouds and segmenting fruit trees,which can complete phenotyping information acquisition task.Main findings and results of this study are as follows:(1)Construct the UAV remote sensing system and adjust the camera and LIDAR calibration.To collect phenotyping information,the study’s requirements were taken into consideration when choosing the UAV drone rack,Li DAR,RGB camera,and on-board computer.Inner reference matrixes of the camera were acquired at various resolutions while the camera self-calibration was done.An experiment was finished with a black rectangular calibration plate,and the LIDAR and camera’s exterior reference matrix was solved using Bundle Adjustment.Six different scenes were chosen to show the coloring effect and RGB images with a resolution of 2576×2048 pixels were chosen for point cloud coloring.Results show that the coordinate conversion error in the depth direction is 0.06 mm,which can satisfy calibration requirements of the Li DAR and camera.Projection errors of the u and v point clouds of the inner and outer reference matrices obtained at this resolution are 1.72 pixels and1.50 pixels,respectively.(2)Research on generation and construction method of color point cloud dataset.SIFT+SFM and R~3live networks were developed to generate color point clouds of the same target,respectively.The accuracy of the generated point clouds was compared based on the size of the same target to choose an appropriate model for creating point clouds in an orchard environment.The findings indicate that while R3live obtained a correlation coefficient of0.9839,SIFT+SFM got a correlation coefficient of 0.8577.Additionally,R~3live produces17,311 more-point clouds for the target than SIFT,and it does so with a greater point cloud density,making it a superior option for producing color point clouds.Then,in accordance with the needs of the research,colored point clouds of an apple orchard were created using R~3live,and the process for creating and enhancing the dataset was chosen based on the segmentation network that was subsequently selected and the actual situation of the orchard.Finally,file format was modified in accordance with the Rand LA-Net network’s requirements to train the deep learning network.(3)Semantic segmentation of colored point clouds in orchard.This chapter proposes an algorithm of segmenting single rows of trees and removing support systems of colored point clouds generated in orchard in response to the general performance of Rand LA-Net on low tree segmentation,which was based on shape differences between fruit trees and support systems.Based on a difference in the number of point clouds above and below the target,the automatic segmentation point of the tree rows is selected based on the point cloud projection and the support system is eliminated.The Rand LA-Net target segmentation model is subsequently taught.This chapter demonstrates that the model gets accuracy of 0.837,0.876,and 0.817,respectively,without the use of the pre-processing method.The Io U of 0.817demonstrates the effectiveness of the pre-processing algorithm suggested in this study in enhancing point cloud semantic segmentation of orchards.(4)Phenotyping information gathering and experiments on fruit trees in orchard.The issue of extracting apple branches individually was resolved after the instance segmentation network of the ITE-net network had finished its training,the phenotypic characteristics of a single apple tree were determined based on the rectangular box dimensions and the function based on Open3D was also used to determine the minimal exterior wrapped rectangle box of a single apple tree.When the phenotypic characteristics of 100 apple trees measured using this paper’s algorithm were compared to those obtained by manual measurement,the linear correlation coefficient of determination R~2 were 0.9366,0.9449 and 0.9134,with good correlation,demonstrating the method’s effectiveness.