Research And Implementation Of Augmented Reality Based On Depth Image Segmentation And Object Tracking

Augmented reality is that the virtual object or system hint information generated by the computer is superimposed into the real scene,thus realizing the "enhancement" of reality.With the release of the AR hardware system such as Google Glass,HoloLens,and the great heat of AR related applications such as Fantasy Kaka and Pokemon go,the augmented reality becomes one of the most popular technologies.Most of the current AR applications,mainly through the specific marker pattern or scene feature point detection to build the reference plane in the real scene,and calculate the camera relative to this reference plane space location,thus realizing the superposition of virtual objects and real scene.There is no more segmentation and recognition of objects in the scene,so it cannot realize the effect of shielding judgment and collision between virtual objects and real objects.In this paper,we propose segmentation and tracking of scene objects based on depth image,thus realizing the shielding and collision of virtual and real objects,and enhancing the effect of the interaction.We use the Kinect to acquire the depth image data of the scene,the segmentation of the scene object based on the depth image,and further achieve the dynamic tracking of the object segmentation.In the scene segmentation,the depth information is converted into point cloud data,and then based on the law to information and scene prior knowledge segmentation of the scene in the large plane(table,wall,etc.),finally to cluster the rest of point cloud data,thus realizing the scene segmentation.We also build the bounding box(known as the proxy geometry)for the segmented objects to facilitate the computation of the rear collision detection.When the object tracking is carried out,the segmented object is used as the tracking template,and the 3d tracking algorithm based on particle filter is applied to calculate the multi-objective real-time tracking.We use Unity3D as a platform to realize the development of AR application system.Firstly,the segmentation and tracking algorithm is encapsulated in a DLL,and the runtime DLL passes the segmentation and tracking object agent geometry into unity.Then,in unity,the fusion of the actual object,the shadow judgment of the actual object and the collision simulation work.The experimental results show that this paper can effectively divide the real scene in the simple indoor,and the collision,tracking and occlusion of virtual objects and real objects are realized compared with the traditional augmented reality system.It has certain significance and application value in the field of enhancement of reality-related research and application.