Irregular3D Objects Tracking System Based On Matching With Keyframes

Tracking of3D objects is being used in many applications. These include robotproviding visual servo services and augmented reality systems which requires the real timeregistration of target objects. Computer vision offers cheap and practical solutions. Thefirst one are some algorithms that use some fiducials or markers. They can extract featureseasily, and reliable. But this method is invasive which means that we should attach thefiducials to the object all the time.The second approach is to use natural feature points toreplace the fiducials. And then tracking the target object uses the structure from motionalgorithm. But as for the lackness of robust extracting feature point, knowledge of3Dinformation of the object is used to help tracking. What’more, occlusion handling and driftare big challenges for this method. The third one is using tracking by detection. Howeverthe method is time consuming and will fall in jitter finally.We present a method that is based on matching with keyframes which is labelattached. At first, we extract the natural features from the image. And then matching themwith keyframes, during which the label will transfer from the keyframe to current frame.The strategy when calculating label is based on the label upon the keyframes and temporalcoherence contraints with previous label. Besides, this article proposed a concept of regioncompetition which means all the keyframes will compete for the chance of recovering thelabel in current frame. All these efforts lead to a robust label in every input frame. Finally,matching the points with label between consecutive frames to estimate camera matrix totrack.Using the keyframe with label to help track has five advantages which is list asfollows: at first, the preparation for the keyframe is easy to get compared to the complex CAD3d models；and the second the robustness of label in frame is much helping inmatching between consecutive frames to remove wrong matches which results in a bettertrack; the third advantage is good occlusion handling as for the label is calculated in everyframe; the fourth one is that we not only take the matching with keyframes account butalso for the previous one, which made us track object with much more less drift and jitter;at last, we have the label as a by-product in the tracking for many applications.