Research On Trolley State Detection And Pose Estimation Based On Monocular Vision

The Airport Luggage Collection Robot System is designed to recycle idle trolleys scattered at the airport and place them in designated areas to achieve unmanned collection of trolleys.The whole system of the airport baggage trolley collection robot is mainly composed of modules such as visual recognition,state detection,pose estimation,navigation and positioning.The state detection and pose estimation of the luggage trolley are the most basic and most critical parts of the whole system.The state of the trolley is divided into two states: occupied and unoccupied.It needs to be determined according to whether the trolley has goods and whether there are pedestrians around.The existing target detection algorithm only uses the information in the frame to classify the target.Considering the influence of surrounding information on the target classification,directly using the existing target detection algorithm will lead to inaccurate identification.In addition,the airport environment is complex,and the point cloud information of the luggage trolley has a lot of noise,and the use of the point cloud matching algorithm results in inaccurate results.In view of the above problems,this paper mainly develops the method of baggage trolley state detection and pose estimation based on monocular vision,and finally applies it to the airport baggage trolley collection robot system.Aiming at the problem of airport baggage trolley status detection,this paper proposes a baggage trolley state detection model based on context information,and uses the surrounding environment information of the baggage trolley to detect the state.The model is improved based on the two-stage target detection algorithm Faster RCNN,adding an extra branch.The branch judges the situation in which the pedestrian occupies the luggage trolley,and the original algorithm judges whether the luggage trolley has the goods,and finally integrates the two classification results to obtain the final state of the luggage trolley.In addition,there are a lot of noises in the point cloud information obtained in the airport environment.Based on a single rgb picture,this paper designs a method for estimating the position of luggage trolley based on semantic key points.The occlusion of the self is very common.The design model consists of two phases(DetecNet and RefineNet).DetecNet is a model structure of the encoder-decoder,and combines shallow and deep features to detect easy key points.RefineNet is a multi-volume.The block has an intermediate supervised model structure,fine-tuning the results of DetectNet.Inspired by the online difficult example mining and Focal Loss,we only select the top-K key point loss for back propagation,so that the model pays more attention to the difficult key points.Finally based on the detected 2D key points and the 3D key points of the model,the PNP algorithm is used to solve the posture of the luggage trolley.This paper collects a large number of images to verify the model of this paper,and compares the experimental data with the existing algorithms to verify the feasibility of this paper.Compared with the Faster RCNN algorithm,the designed context model has improved by 3 points.The designed semantic keypoint model reached 0.77 on the OKS evaluation index,compared with models such as hourglass and CPM,it was improved by 2 points,and the average position error was about 19 mm,and the average angle error was about 2.5.