Research And Application Of Key Technologies Of Augmented Reality For Water Scene

With the continuous development of augmented reality application scenes,water scenes have a certain development value as a common outdoor natural landscape.Water has complex physical optical properties and irregularity.To enhance the effect of the virtual-real fusion of this scene,the augmented reality system needs to have the cognition and identification ability of the real scene,provide scene information in real time and use computer technology to simulate and register to corresponding position,so as to complete the rendering effect of the surface of the virtual object consistent with the surrounding scene lighting.With the continuous improvement of software and hardware performance of mobile devices,mobile augmented reality is more suitable for outdoor use due to its portability.In order to enhance the augmented reality effect of outdoor water scenes,a new method of implementing the augmented reality systems is proposed in this paper,which is suitable for outdoor water scenes under solar light condition,and is transplanted to the mobile terminal for use.During the process of implementation,in view of the problems related to the two key modules of 3D registration and tracking technology and the virtual-real fusion,as well as the ultimate application goal in mobile devices,this paper has carried out the research and innovation of the following content:(1)A real-time and robust method of 3D semantic map optimization mapping is proposed.ORB-SLAM2 is selected as the basis for the construction of 3D scenes,and combined the deep learning semantic segmentation network ICNet with a multithreaded parallel structure to build a 3D semantic map in real time.In this process,the Bayesian method is used to optimize and update the semantic labels of the 3D map.A self-built data set is used for the training of semantic segmentation network.Experiments prove that the 3D tracking and registration technology combined with semantic information makes the augmented reality system have the ability to understand the scene,which accurately limits the placement of virtual objects in the real scene,and also helps to obtain scene information in the virtual-real fusion module.(2)Real-time virtual-real fusion rendering effect under outdoor water scene is realized.Firstly,the application requirements of augmented reality system in the water scene are summarized based on the requirements of virtual-real fusion.Secondly,the simulation of water surface reflection is performed.The virtual object is registered on the water surface which using the RANSAC plane fitting algorithm combined with the least square method to plane fitting with the feature points belonging to the water.Then simulate the Fresnel effect on the water surface and calculate the ratio of reflection and refraction.Finally,by analyzing the effect of the color of the water body on the reflection color,a reflection color mixing algorithm is proposed,which is calculated by combining the color data of the virtual object itself and the detected color data of the surrounding water surface,and finally rendering the corresponding reflection of the virtual object.The color of the reflection changes with the change of the viewing angle and the color of the surrounding water to achieve the visual coherence.(3)A mobile augmented reality application for outdoor water scenes has been implemented.Firstly,the system is tested on the PC side to verify the feasibility and effectiveness of the system and the implementation of various algorithms.Then the Android Studio development platform combined with JNI,Opencv4 Android,Open GL and other technologies and the Socket module are ued for porting to the mobile terminal.Aiming at the shortcomings of the mobile terminal's processing capability,a series of optimizations are performed on the mobile terminal.Finally,through some performance tests and experience evaluation,the final research results are verified.