Object Detection And Localization In Dynamic Environments For Soccer Robots

The accurate and real-time object detection and localization problem under dynamic environment for soccer robots is studied in this thesis.Most RoboCup MSL teams used omnidirectional vision sensors as their robots' environment perception module sensors.The omnidirectional vision sensor has been widely used for object detection since it allows the robot to see in all directions simultaneously without rotating itself or its camera.But it captures images with distortion which has negative effects on the feature extraction,and cannot acquire accurate height information.To compensate these drawbacks,we design a RGB-D vision system which is composed of Kinect v2 and Jetson TX1 to assist the omnidirectional vision system to perceive the environment.This hardware platform provides a stable and reliable platform to test our algorithms.After the hardware platform has been constructed,we propose an algorithm for object detection and localization based on the parallel computing technique.It uses football's color information for detection and then combines with the depth information for the localization of the ball's position.For obstacles,we utilize its shape information in point clouds for detection and localization.All these algorithms are processed by CUDA parallel computing to improve the computing speed.The experimental results show that using the proposed algorithms,more accurate perception results can be obtained than omnidirectional vision system,and real-time requirements can be met.And then,we propose a novel approach for robot detection and localization based on the Convolutional Neural Networks(CNNs)for RoboCup MSL soccer robots.The approach is composed of two stages: robot detection using the RGB image,and robot localization using the depth point cloud.The high performance(mAP around 93.66%)and improved computing speed(from 9Hz to 18Hz)verify that the proposed method is suitable for robot detection and localization during the MSL competition,which will benefit the following strategy design and obstacle avoidance procedures.In summary,we propose an RGB-D vision system to assist omnidirectional vision system to improve the robots' environmental perception ability.By processing the combined RGB image and Depth image captured from Kinect v2,we can obtain accurate 3D information of objects and detect the arbitrary-colored obstacles which can compensate the omnidirectional vision system's drawbacks and improve the robustness of the robot's vision system.