Research On 3D Reconstruction Method Of Plants Based On Deep Learning

With the rapid development of computer vision,plant 3D reconstruction technology is used in many scientific research fields such as agricultural robot navigation and ecological environment monitoring.For traditional 3D reconstruction algorithms,due to the complex morphology of plants themselves,it is difficult to meet the requirements of high-quality plant 3D reconstruction when under natural environmental conditions,such as different lighting changes.With the development of deep learning,3D reconstruction combining with deep learning can achieve high-quality3 D reconstruction.However,there are few relevant studies on deep learning plant 3D reconstruction at home and abroad.Therefore,in this dissertation,3D reconstruction network model based on deep learning is constructed to obtain high-quality 3D reconstruction by using multi-view plants taken under different lighting conditions.The main research work is as follows:First,to remove the influence of backgrounds on plant 3D reconstruction,a uniform threshold image classification method is firstly proposed,in which the traditional algorithms such as principal component analysis(PCA)and grayscale statistical histogram parameters are used over the multi-view plant images under different lighting conditions.Then,different segmentation methods are used to segment the plants from the backgrounds to eliminate the influence of backgrounds on 3D reconstruction.That is,the color factor combining with the threshold method is used for segmentation of plant images taken under normal lighting conditions;the contour coefficient optimized SLIC superpixel blocks combining with the decision tree algorithm is used for segmentation of plant images taken under abnormal lighting conditions.The experimental results show that the effectiveness of the methods in this dissertation are good,with the average error of the image classification method of 0.004%,which can achieve the accurate classification of plant images affected by lighting changes;the accuracy of plant image segmentation can reach more than 95% under normal lighting conditions,the accuracy of plant image segmentation is 83.75% under abnormal lighting conditions,and the accuracy of improving plant image segmentation by using contour coefficient is 84.37%.Second,based on plant image segmentation above,the plant 3D reconstruction model is constructed based on the deep learning network Patchmatch Net.To address the problem that the original network multiscale feature extraction module weight sharing cannot effectively separate the features of foreground and background,leading to the bad 3D reconstruction,firstly,the attention mechanism models SENet and CBAM are introduced into the multi-scale feature extraction network module respectively,where the weights are assigned to the channel features as well as spatial features to improve the attention of the network for the target reconstruction.Then,to further optimize the depth map quality to ensure the final 3D reconstruction quality,a multi-scale residual network is used to the original network depth map module,where the reference map is fused to generate a multi-scale optimized depth map about residuals to achieve high-quality plant 3D reconstruction.Finally,the whole algorithm model is implemented based on the deep learning framework Py Torch.The qualitative and quantitative analyses and discussions about different attention mechanisms and multi-scale residual networks are provided,and the proposed network model in this dissertation is compared with some traditional 3D reconstruction algorithms.Experimental results show that compared with the traditional 3D reconstruction algorithms Camp,Gipuma,and COLMAP,the error of the proposed network model in this dissertation is reduced by 44.3%,45.9%,and 30.8%,respectively;compared with the deep learning networks MVSNet and Patchmatch Net,the error of the proposed network model in this dissertation is reduced by 24.1% and 4.6%,respectively;compared with the depth map quality and 3D reconstruction,the improved network can generate a wider range of depth maps with higher smoothness.Finally,the depth maps are fused to build a high-quality3 D point cloud model,which has better robustness and generalization ability for 3D reconstruction of weak texture plants in images taken under different lighting conditions.