Stereo Vision Based Tracking And Detection System For Serial Robot

Serial robots have been widely used in the fields of modern industry, such as production of machine, welding, assembly, surface coating, handling and other operations. When the serial robot is working, the position and orientation of the manipulator reflects the important parameter of motion state. Therefore, it is essential to measure the information of working robot's manipulator. Moreover, in the robot vision control system, the position and orientation of the manipulator is the major parameter in the part of visual detection. The serial robot has the character of multiple degrees of freedom and the complex trajectory. The measurement must meet the requirements of multiple degrees of freedom angle. Compared with traditional measurement methods, computer vision based measurement system has the advantage of intelligence non-contact and fast. Therefore, in the thesis, we design the tracking and detection method for the robot system based on stereo vision theory. The system has completed tracking, locating and measuring the manipulator, meanwhile the results of 6-ROF robot state has been calculated and output, In the thesis, the main contributions have been obtained as follows.Firstly, we design the framework of stereo vision based tracking and detection system for 6-ROF robot. The system has been successfully applied to tracking and detection the position and orientation. The manipulator's real-time state information has been calculated and displayed by the system. It is composed of image gather and transfer, camera calibration, tracking the manipulator, feature point extraction and matching based on SURF, reconstruct of points, modeling and calculating the position and orientation.Secondly, CamShift algorithm is integrated into the framework of the tracking and detection system. Real-time tracking the series robot manipulator is realized by using OpenCV. Searching window can be locked to track manipulator and adaptively adjust the size. The system can track the serial robot in the huge space environment with multi-degree freedom and complex trajectory. Even if the object is blocked by something, the system still tracked in real-time.Thirdly, SURF algorithm is integrated into the feature point extraction and matching in the step of detection position and orientation of manipulator. The algorithm has not yet been applied in this field. SURF algorithm based feature extraction and matching should be the core of the system. Firstly, we detect the extreme point in the scale space, identify direction vectors of the key point, generate the SURF feature vectors, and match feature point by using SURF feature vectors. Then we obtain the feature point 2D coordinates in each camera system by using SURF algorithm.3D coordinates has been calculated in feature point restructuring model. Finally, according to position and orientation model,3D coordinates has been changed to resolve the objective optimization problem with variablesΦ,θ,Ψ. The result is optimized by an improved particle swarm optimization (PSO). It is for measurement position and orientation as feasible approach. The improved PSO algorithm has mainly been added to inertial coefficient and has substituted global extreme value by average of areas extreme global value. The method has reduced the possibility of algorithm falling into local extremes.Finally, the main results of experiments have been shown in this thesis, such as tracking the manipulator, feature point extraction, matching and so on. The 3D parameters(tx,ty,tz,θ,φ,Φ)simulated by MatLab have smaller difference between actual values in the workspace. All approaches in this thesis have been proven to be effective and feasible, particularly, suitable for serial robot tracking and detection with motion character of multi-degree freedom and space complexity.