Research On Soybean Canopy Phenotyping Method Based On Multi-camera Machine Visio

The precise reconstruction of the soybean canopy’s three-dimensional(3D)morphological structure and the accurate acquisition of phenotypic parameters are of great theoretical significance and practical value for the selection and scientific cultivation of soybean varieties.On the basic of the reconstruction of the 3D morphological structure of the soybean canopy,the calculation of morphological phenotypic characteristics parameters and the control of soybean growth trends is one of the important ways to scientifically fertilise,manage and monitor the growth of soybean.However,at this stage,the point cloud data of the soybean canopy is mainly acquired by monocular or binocular acquisition,which makes it difficult to reconstruct the 3D morphological structure of the soybean canopy in a comprehensive and high-precision manner,thus affecting the calculation accuracy of the canopy phenotype parameters.Therefore,in this project,we have conducted a study on a multi-view machine vision-based method for calculating soybean canopy phenotypes,using laser sensing technology and image processing techniques combined with crop phenomics.In this paper,a multi-view acquisition platform based on the Kinect 2.0 sensor was built to reconstruct the 3D morphological structure of soybean canopy from an all-rounded perspective,calculate phenotypic traits such as plant height and canopy width,and develop a 3D reconstruction and phenotypic trait calculation system for soybean canopy.The main contents are as follows:(1)A multi-view vision acquisition platform based on the Kinect 2.0 sensor was built.Three Kinect 2.0 cameras were set at an orientation of 120°to cover the full range of views of the soybean canopy to build a multi-view machine vision acquisition platform.54 pots of soybean samples were planted with Suinong 26 cold land soybean experiment.The acquisition platform was used to obtain a full range of depth information of the soybean canopy from three viewpoints,as well as to measure the phenotypic traits of the soybean canopy,providing a valid data base for accurate 3D reconstruction and phenotype calculation of the soybean canopy.(2)A soybean canopy reconstruction method based on multi-view machine vision was established.Based on the acquired point cloud data,the conditional filtering algorithm and the K-nearest neighbour filtering algorithm are used to remove the redundant information in the original point cloud.Then the point cloud data from the three views are matched and fused using RANSAC+ICP.The average error of the reconstruction is 1.09cm,which is 2.14cm smaller than the error of the ICP algorithm.As a result,an accurate reconstruction of the 3D morphological structure of the complete soybean canopy was achieved,providing realistic and reliable data for the rapid and accurate calculation of canopy phenotypic traits.(3)A method for calculating soybean canopy phenotypes based on 3D morphological structure was established.Based on the reconstruction of the morphological structure of the living soybean canopy,the Alpha-Shapes algorithm was applied to extract the canopy point cloud outline,obtain the distance between two points in space,and calculate the plant height and canopy width of the soybean.The leaf area index of the soybean canopy was calculated using the stratified calculation method by combining algorithms such as the convex package and the gradient ascent method,which removes the stalk portion of the soybean canopy.The projection method,image binarization method,and vector angle calculation method were used to detect branch points and vertices,and the angle between the petioles of soybean canopies was calculated.The correlation coefficient R~2 between the calculated and measured values of soybean canopy phenotypic parameters was above 0.9.The calculated results were more accurate,achieving non-destructive acquisition of soybean canopy phenotypic traits.(4)A 3D reconstruction of the soybean canopy and a phenotype calculation system were developed.This system was programmed through a hybrid of Python and Matlab.The user interface was based on Pyqt5 and the App Design development framework.Applying technologies such as SSL encryption protocols and Application Programming Interfaces(APIs),the point cloud data transfer and interface engine interaction are implemented.The system is capable of analysing and processing soybean canopy point cloud data and mainly consists of a basic data manipulation module,a point cloud filtering module,a 3D reconstruction module and a phenotype calculation module.It completes the three-dimensional reconstruction and phenotype calculation system for soybean canopy,providing a visual platform for acquiring soybean canopy growth information.