Localization And Trajectory Prediction Of Spinning-Flying Ping-Pong Ball Based On Learning

Real-time detection,localization,and trajectory prediction of flying-spinning ob-jects is one of the research difficulties in robotics visual perception,with its significance and extensive practical application value in the fields of military,sports,aeronautics,and industry.For a robot to play spinning ball,this thesis conducts researches on the detection,localization,and trajectory prediction learning framework of a flying-spinning ping-pong ball based on deep learning methods,and further verifies the proposed frameworks and algorithms on self-built data sets.The main contributions of this thesis are as follows.(1)A ping-pong ball image data set under varieties of circumstances and a ping-pong ball flying trajectory data set with multiple spinning states are built,both with precise labels.Based on the humanoid robotic table tennis system "Wu&kong",we collect images of three kinds of ping-pong balls with several types of background ob-jects under various light conditions,and further increase the richness of the images by specific pre-process methods.By manually appending precise category and pixel posi-tion labels to the collected images,we build a ping-pong ball image data set containing 46000 images.Also based on the "Wu&kong" system,we build a ping-pong ball flying-spinning trajectory data set containing 2000 trajectories through stereo location method.The construction of the above two data sets lays the foundation of follow-up researches,and is of convenience for other researchers.(2)A learning framework based on convolutional neural network for detection and localization of a ping-pong ball is proposed,enhancing the adaptiveness of detec-tion and localization under varieties circumstances.This thesis constructs a two-branch neural network for both detection and localization,in which we use convolutional lay-ers for hierarchical feature extraction,and an Spatial Softmax layer for pixel position regression.Trained and tested by the self-built ping-pong ball image data set,the pro-posed method successfully detects and localizes three kinds of ping-pong ball under varieties of light conditions and backgrounds.In addition,we balance the accuracy and computation time of the proposed network by adjusting network structures properly,satisfying the demand on accuracy and effectiveness of robotic table tennis system at the same time.(3)A learning framework based on recurrent neural network for trajectory predic-tion of a flying-spinning ping-pong ball is proposed,improving the prediction accuracy of ping-pong balls trajectories under multiple spinning states.Without dependencies on any prior knowledge,we take the advantages of the non-linearity of LSTM(Long-Short Term Memory)units to approximate the real model of a flying-spinning ping-pong bal-1.Trained and tested by the self-built ping-pong ball trajectory data set,the proposed method successfully predicts the long-term trajectories of flying ping-pong balls with multiple spinning states,satisfying the demand on accuracy and effectiveness of robotic table tennis system at the same time.Using the proposed methods,the robotic table tennis system is enabled to conduct detection,localization,and trajectory prediction of a high-speed flying-spinning ping-pong ball under complicated circumstances.The accuracy of detection is more than 99%.The pixel position localization accuracy in u-axis and v-axis are 0.10 pixels and 0.58 pixels.The predicted accuracy at hitting point in x-axis and y-axis are 2.83mm and 2.97mm.