Research On Dual-mode Decision Fusion Of Garbage Sorting Based On Large Parallax Image

At present,the sorting work of garbage treatment station,lawn,residential road,and scenic spot is mostly carried out manually,which is characterized by harsh environment,tedious and repetitive work and high intensity.Using robot(manipulator)automated sorting can be very good to improve these problems.Most research on automatic garbage sorting is based on object detection technology with monocular vision.Generally,the monocular camera is placed directly above the recognition area to obtain one modal two-dimensional visual information.But garbage is a three-dimensional object,and the same object obtained in different views may have huge differences in the images.All the obtained dual-modal or multi-modal images from different views can be used as important basis for object classification.In this thesis,two cameras are used for simultaneous shooting with a large parallax angle,then we use the obtained two images for garbage classification.Drawing on the idea of the Bagging method in Ensemble Learning,we establish a Combined Symmetric Model.Input the visual information of the left and right channels into the two basic deep learning classifiers and divide the results into categories based on the consistency of the prediction results There are two types of results,one is the Consistent Pair of Prediction,and another is the Conflict Pair of Prediction.The Consistent Pair of Prediction means that the predicted categories of the two channels are the same.After the logarithmic average function is used to fuse the confidence of both channels,it can be output as the final prediction result.This logarithmic average function can solve the problems of the Combined Symmetric Model that the confidence level of each channel simply adds up may exceed [0,1],and the weighted average method cannot interpret the credibility problem well.The Conflict Pair of Prediction means that the prediction categories of the two channels are inconsistent.Then we use the Stacking method in Ensemble Learning for further distinguishing.Extracting features from the Conflict Pair of Prediction dataset and sending it to the ANN Fusion Classification model to obtain a more accurate garbage classifier.Drawing lessons from the idea of GIo U(Generalized Intersection over Union)in monomodal vision,find the smallest rectangular box that can cover the ground truth bounding boxes of the two-modal as the global ground truth bounding box.On this basis,an Io U of dual-modal visual projection is created to indicate the coincidence degree of the dual-modal prediction bounding box and the ground truth bounding box.Then redefine the judgment criteria of positive and negative cases under dual-modal vision.By integrating the optimized Bagging and Stacking methods above,3 basic classifiers(2 deep learning classifiers and 1 ANN classifier),we finally obtained a stronger garbage classifier.The performance of the ensembled classifier is verified in our experiment: In single modal experiment,the precision of the left and right camera is 75.00% and 74.87%respectively,the recall is 71.60% and 72.94% respectively.In dual-modal fusion classification,the precision has risen to 87.91%,and the recall has reached 78.33%,both of which have been greatly improved.The large parallax dual-modal vision with the ensemble learning fusion classification scheme can significantly improve the performance of the garbage classifier.