Study On Binocular Vision Location Of Multiple Fruit Picking Robot Based On Illumination Normalization

Fruit picking robot has become a research hotspot in the field of agricultural engineering because of its advantages of improving fruit picking efficiency,reducing labor intensity and saving fruit harvest cost.And the development of a robot for multiple fruit picking has been paid more attention by researchers.However,in the process of fruit picking,the vision system of robot is often affected by fruit growth environment such as changes in the intensity of illumination and random occlusion of fruit surface,which makes the fruit recognition and localization missing and false due to the lack and difficult recognition of surface information,and eventually lead to the fruit picking robot failure.Therefore,this paper studied the recognition and localization of multiple friuts under the unstructured environment based on binocular stereo vision,illumination normalization principle and machine learning theory.And the recognition and localization of many kinds of different growth of fruit were implemented by using software simulation and hardware test,which intended to develop a robust fruit recognition and localization method against the unstructured environment for the binocular vision system of multiple fruit picking robot,thus accurately guide the robot to complete the fruit picking.The specific contents are as follows:(1)Due to the problem that the visual system of fruit picking robot is easily subject to the change of illumination under unstructured environment,two models of fruit image illumination normalization were proposed.One model used the first-order wavelet transform to decompose the fruit image into the high and low-frequency domain,and then contrast enhancement and histogram equalization were implemented on these two domains,respectively.Thus,the fruit image illumination normalization based on wavelet transformation were completed after composing the processed images by using inverse wavelet transformation.Another model used the classical Retinex algorithm to decompose fruit image acquired by the visual system into illumination image and reflection image.Then wavelet transform-based illumination normalization were implemented on the reflection image.Thus,Retinex-Wavelet-based illumination normalization were completed.By investigating the visual evaluation indexes of the proposed two algorithms in the ideal picture simulation experiment and the fruit image experiment under natural environment,it can be seen that these two methods are suitable for the illumination normalization processing of fruit image.(2)The near circular fruit and non-circular fruit recognition models were constructed,respectively.After illumination normalization using the wavelet transform,the near circular fruit recognition model used color-based K means clustering and Hough circle transform(CHT)to detect the potential fruit area.And an AdaBoost classifier was trained by using the local binary pattern(LBP)feature value of fruit and non-fruit,which is used to detect false fruit area and non-fruit area.By composing the detection results,the recognition of near circular fruit was completed.After illumination normalization using the Retinex-Wavelet,non-circular fruit recognition model used the effective color component and six kinds of Tamura texture features to train the four basic classifiers.And then,fruit was recognized by using the four classifiers,respectively.By using logical ‘AND' operation to combine with the recognition results,non-circular fruit recognition was completed.The two methods were tested by using the citrus and grape images obtained under the natural environment,and the success rate of recognition was 85.6% and 86.86%,respectively.(3)After successful recognition of near circular fruit and non-circular fruit,a matching method based on single fruit label template and a matching method based on clustering fruit label template were proposed based on the different growth patterns of fruit under natural environment.The first matching method firstly extracted single fruit edge,and then,used OPTA algorithm to thin the edge.The minimum enclosing rectangle of the edge was used as the fruit label.The geometric center of the rectangle was used as the matching feature point.The normalized cross correlation(NCC)function was used as the similarity measure function.The optimal matching window was searched in the right image.The experimental results showed that the successful matching rate of litchi fruit pairs could reach 88.33% under different illumination and occlusion by using this method.The matching method based on clustering fruit label template improved the matching method based on single fruit label template.Because of inevitability of fruit occlusion each other under natural environment,label combination was implemented according to the pre-defined fruit clustering type.And then matching was implemented according to the single fruit label matching method.The experiment showed that this method improved the matching success rate of fruit.Under different illumination and occlusion conditions,the success rate of this method for litchi fruit pair was 93.33%.(4)In order to verify the feasibility and accuracy of the proposed method for fruit localization,a set of software for target localization of binocular vision system for multiple fruit picking robot was developed.The software integrated the illumination normalization algorithm of fruit image,fruit recognition algorithm and fruit localization algorithm proposed in this paper.Using the software cooperating the binocular camera hardware system,fruit localization experiment was implemented under natural environment.The data measured by the laser rangefinder was taken as the real value.The real-time localization experiment was carried out under the natural environment.In real-time localization experiment of citrus,the average error of all measured sample values were within the range of ±15mm.The different values of T-test analysis were greater than 0.05,which indicated that the proposed method was feasible and accurate and was robust against changes in illumination and occlusion.Thus,the method proposed in this paper could provide a visual basis for the picking operation of multiple fruit picking robot.