Research On Oscilating The Branch Of Lycium Ruthenicum Images Deblurring And Identification And Localization Based On Deep Learning

Lycium ruthenicum is one of the important economic forest fruits in Ningxia Hui Autonomous Region,and the current method of picking Lycium ruthenicum mainly depend on manual picking,but there are many problems such as strong workforce,manpower scarcity and poor harvesting efficiency in manual picking.With the continuous development of picking robots in agriculture,the combination of picking robots and vision recognition systems are employed.Manual picking methods cannot meet the requirements of modern agriculture.The vibration type picking has the problem of damaging wolfberry fruit and fruit trees,and the operation process of comb picker relies heavily on manual determination.Therefore,the picking robot adopts the design of human-like picking mode,and the vision system and double-arm structure imitate the operation process of manual picking of wolfberry to ensure the successful harvesting of wolfberry.The intelligent Lycium ruthenicum picking robot employs a double-arm collaborative picking robot designed in a human-like pattern,but during the operation process,the Lycium ruthenicum branches swing because of factors such as wind or mechanical arm collision with the branches,which makes the images acquired by the imaging device have motion blurred,and the blurred images have an impact on the identification and positioning of the Lycium ruthenicum branches.In this paper,we introduce a multi-scale network-based image deblurring algorithm,taking into the consideration the key problems faced by Lycium ruthenicum harvesting in natural environment.Secondly,to meet the problem of real-time precise identification and positioning of Lycium ruthenicum branches in the swing state,we propose an improved SparseInst instance segmentation network and binocular stereo vision system to identify and localize Lycium ruthenicum branches.Finally,we verify the reliability of the proposed algorithm by building a tested in an experimental environment.The main researches of this paper are as follows:(1)Research on the deblurring problem of swinging Lycium ruthenicum branches images.Firstly,the swinging Lycium ruthenicum branches fuzzy-clear defuzzification dataset is constructed and the dataset is expanded by employing data augmentation,with the purposes of improving the replicability of the model and preventing the overfitting problem of the network.We propose a multi-scale feature extraction network to address the image blurred problem for swinging Lycium ruthenicum branch images.The specific network structure includes a multiscale depth-separable convolution module,a multiscale perceptual field structure module and an attention refinement residual module.The experimental validation is performed on a self-made Lycium ruthenicum branch dataset and a GoPro dataset after elaboration of the experimental equipment and evaluation metrics.Comparison experiments show that the proposed deblurring algorithm is evaluated compared with three other models to verify the effectiveness of the proposed model.(2)Research on the identification and positioning of Lycium ruthenicum branches.Firstly,the instance segmentation dataset is constructed,and the dataset is expanded with data augmentation.We propose an improved SparseInst instance segmentation network,and the improvement part is to replace the instance activation mapping module with a decoupled dynamic filter module,which reduces the computational cost while achieving aggregated instance features.With this algorithm,the segmented regions of branches in the image can be detected.The detected segmentation regions are localized according to the binocular camera.On the basis of the proposed evaluation metrics,the comparison tests showed that the improved SparseInst model yields an average accuracy of 36.72%and a detection speed of 3 1.86 frames/s on the self-made instances segmentation dataset,which is better than the detection speed of Mask R-CNN and SparseInst models.(3)Oscillating Lycium ruthenicum branches identification and positioning experiments.The experimental platform is built in the experimental environment,the research contents of chapter 2 and chapter 3 are technically combined,the Eye-to-hand approach is used to establish a stereo visual localization system to realize the calculation of pixel coordinate system and world coordinate system transformation,and the proposed algorithm is verified through experiments,the improved SparseInst model and ZED 2i sensor positioning error within 8%,which it can meet the requirements of 3D spatial localization of the grasping point of the wolfberry branch.The experimental results show that the proposed image deblurring algorithm can clarify the blurred images generated by the motion of swinging branches of Lycium ruthenicum.The integration of the improved SparseInst instance segmentation algorithm and the binocular stereo vision localization system can identify and localize the oscillating branches of Lycium ruthenicum,providing theoretical and technical support for solving the problem of oscillating branches in intelligent harvesting of Lycium ruthenicum.