Eye-in-hand visual servoing to improve accuracy in pool robotics

Deep Green is a robotic pool system that is currently under development at Queen's University. The objective of this system is to play the game of pool competitively with skillful human opponents. Primarily, the robot determines its position in placing a shot by detecting ball locations using a ceiling mounted camera. However, due to mechanical inaccuracies, the system is prone to positioning error.; To increase the accuracy of the system, an eye-in-hand system (a camera mounted on the robot wrist) is described, which provides final adjustment to the absolute positioning of the robot. The use of image information to correct a robotic motion is also called visual servoing. This eye-in-hand system, which provides positioning correction depending upon feedback from the local view of the table scene, is called the Local Vision System (LVS).; Challenges arise in making the LVS to work as functioning unit include correctly interpreting information from the LVS images and carrying out the corresponding motion correction. The solution proposed in this work depends on extracting only the cue ball and the object ball on the table. An ideal shot trajectory based on the image data of the balls was acquired and used as a reference to correct the positioning error. The accuracy of this system also depends on how accurately the camera and the robot is calibrated. In other words, the metric relationship between the last joint of the robot and the camera frame must be determined. Thus, in this work, we also present a novel technique to solve this calibration problem in the form of a homogeneous system. Experimental results showed that shots made by the robotic system, using our fully calibrated eye-in-hand visual servoing system, increased its shot accuracy by 3 times, with high consistency.