3D Object Detecting And Tracking System Based On Local Features

Augmented reality is a technology which integrate virtual objects into the real environment,and it makes users feel beyond reality.After decades of research and development,augmented reality technology has been widely used in automotive repair,cultural relic restoration and clinical medicine.3D object tracking is a technology which a 3D object in the real world is treated as a target,and the 3D model of this object is known.The system registers the target object in the video with the 3D model,and the 3D model is projected to the position of the target object in the image.Therefore,3D object tracking is also called model-based tracking,which can be used as a way to achieve augmented realityThere are many 3D objects with different shapes in the real world.According to different classification rules,it can be divided into rigid bodies and non-rigid bodies,textured objects and texturless objects,this paper only focuses on 3D rigid bodies with texture.Despite decades of development,there are still many challenges in the realization of 3D object tracking technology,such as motion blur,complex background and so on.In order to track a 3D object,camera pose estimation is needed.There are many methods on how to estimate camera pose.The method based on visual key points has many advantages.Because it is robust to occlusion,illumination,and scale changes,and easy to achieve in real timeIn view of the above reasons,we designs and implements a 3D object detecting and tracking system based on local features in this paper.This system is suitable for 3D object with texture,and it makes full use of the texture on the object surface.Its detection and tracking methods are both based on key points.Before the system launched,the system extracts features from the model of target object,and it generates a model preprocessed file.After the system is launched,the 3D object in the video will be detected automatically.Once the detection is successful,the system starts tracking When the system fails to track,the system will redetect the target object and initializes it.The system automatically completes the switch between the two modes.When the system detects and tracks the target object,the results are displayed in real time.It gives the user an intuitive feeling.Our system pursues the principle of simple operation.It need neither marker in the real environment nor Human-Computer Interaction.It only needs the 3D model and detection video of the target object uploaded by the user.Our systems do not require too excellent hardware.And it doesn't use GPU to accelerate.Its input device only uses monocular RGB camera.Currently,it runs in near real time on a computer.It is very hopeful to be transplanted to some embedded devices in future,such as AR glasses,industrial robot arms and so on.And it will generate great practical value.