Scene Understanding Based Research For Indoor Augmented Reality Visualization

GIS technologies are becoming widely used in a growing number of application scenarios,thus more attention has focused on the display and visualization of spatial information.Traditional media for presenting spatial information,such as 2D or 3D maps,cannot meet growing user requirements for dynamic displays.Flexibility and realism in GIS visualizations are becoming ever more demanding,as the volume and complexity of this information expands;AR-GIS is a response to these challenges.High quality AR representations can help users better comprehend and interact with real world spatial information.In order to achieve realistic visual effects and coherent rendering,camera pose tracking techniques are necessary for accurate understanding of spatial relationships in AR-GIS.Precise tracking of the camera within an augmented environment is required to achieve proper alignment of the virtual objects to their real-world counterparts and create a rich user experience.The existing tracking methods obtain only a 6DOF camera pose.AR-GIS renderings,however,require exact positions of objects appearing in target scenes,consistent with the real world;for example,desks must be on the floor,or pictures must hang on the wall.Thus,the AR system must not only consider the correspondence between two images,but also recognize where the floor is located,and where a picture is superimposed on a wall.Therefore,the goal of this paper is to enable realistic augmented user experiences in 3D scenes though 3D scene understanding and indoor scene tracking that can properly integrate and deal with the limitations of AR-GIS visualization.In light of those,the main works and contributions if this paper were presented as follows:(1)We have summarized the key technology of augmented reality visualization,and concretely studied the 3-D registration based on computer vision.Especially,we have detailed the working process of 3-D registration method based on natural features,and have analyzed the problems existing in the current research augmented reality visualization.(2)This paper has been studied and summarized the existing indoor 3-D layout extraction technology.Through in-depth study of the concept of structured prediction,we have detailed the indoor layout extraction based on structured predictio.The extraction of indoor layout based on geometric information and FCN were described respectively.In addition,we realized and analysed the constraint effect of the rough layout estimated from FCN to the traditional layout extraction method.(3)A structured learning-based texture mapping method is proposed for automatic mapping a single still photo from a mobile phone onto an alreadyconstructed indoor 3-D model.The up-to-date texture is captured using a smart phone,and the indoor structural layout is extracted by incorporating per-pixel segmentation in the FCN algorithm and the line constraints into a structured learning algorithm.This enables real-time exture mapping according to parts of the model,based on the tructural layout.Furthermore,the rough camera pose is estimated by WiFi to determine here to map the texture.Based on this fusion approach,users can have a better experience in virtual indoor3-D applications.(4)We propose a novel method for fusing virtual objects and indoor scenes,based on indoor scene recognition technology-To accomplish scene fusion in AR,we first detect key points in reference images.Then,we performinterior layout extraction using a Fully Connected Networks algorithmto acquire layout coordinate points for the tracking targets.We detect and recognize the target scene in a video frame image to track targets and estimate the camera pose.In thismethod,virtual 3D objects are fused precisely to a real scene,according to the camera pose and the previously extracted layout coordinate points.Based on this fusion technique,users can better grasp virtual three-dimensional representations on an AR platform.(5)In this paper,indoor 3-D model texture based AR fusion visualization system was proposed.The system has been used texture with high sense of reality of as the reference image.Extracting the key points form the reference image for for tracking the camera pose and estimating the indoor 3-D layout to set an exact position for visual object drawing.The texture image with high realistic sense can not only be used to calculate the camera pose,unlike general reference images,it can also provide geometric information and spatial attributes,which are necessary for AR-GIS visualization.This method does not need to set up artificial signs and special processing for reference image beforehand that extremely simplified the preparation work of AR system.