Research On Outdoor Virtual--Real Shadow Interaction Technology For Mobile Augmented Reality

In recent years,with the development of augmented reality,especially various hardware devices and software systems represented by mobile augmented reality are widely used in the fields of science and technology education,game entertainment,industrial maintenance,map navigation and commercial consumption.At present,improving the realism of aug-mented reality system is the main research direction of scholars,in which light and shadow consistency is one of the most important ways to improve the realism.Light shadow con-sistency not only needs to pay attention to the generation of virtual object shadow and the estimation of scene lighting conditions,but also needs to study the influence of real object shadow and virtual object shadow on each other.In order to solve the problem of shadow interaction between virtual and real objects,this paper designs an outdoor virtual and real shadow interaction framework for mobile augmented reality.The main work and innovations of this paper are as follows:(1)The estimation of sunlight direction and intensity in outdoor scene is studied and realized.For direction estimation,firstly,the hardware sensor is used to obtain the current longitude and latitude information of the user,and then the altitude and azimuth of the sun are estimated according to the sun position positioning algorithm in the field of astronomy.For light intensity estimation,firstly,the absolute illuminance value of the scene is collected by the light intensity sensor,then the illuminance coefficients of direct sunlight and diffuse sky light are estimated respectively with reference to the outdoor lighting model,and finally the color depth of the shadow of the virtual object is set according to the illuminance coefficient.(2)Aiming at the problems of high false detection rate and poor edge continuity in the current video shadow detection algorithm under mobile viewpoint,a real-time shadow detection algorithm based on edge tracking and edge detection framework is proposed.Firstly,the shadow part overlapped by the front and back frames is tracked twice to screen out the points with large tracking error,and the accuracy of the tracking edge is guaranteed by Canny edge confidence? Then,the new region to be detected is obtained by the region division method based on optical flow? Secondly,aiming at the problems of texture edge false detection,soft shadow detection and dark area false detection,a seven dimensional feature vector is constructed,and the shadow edge feature vector is extracted to train SVM classifier to detect the shadow in the new area? Finally,a gradient direction consistency algorithm based on RGB color space is proposed to optimize the connection of broken edges.(3)Aiming at the problem of virtual real shadow interaction in the absence of complete geometric information of the scene,an outdoor virtual real shadow interaction framework for mobile augmented reality is proposed.The framework does not need to obtain the complete geometric structure of the scene,but only needs to obtain the depth information of the shadow area in the scene to complete the virtual real shadow interaction.Firstly,the three-dimensional pixels in the shadow area are transformed into the world coordinate system,and then the points in the world coordinate system are transformed into the clipping space of the virtual camera according to the internal and external parameters of the camera,so that the virtual shadow map of the light source perspective can be deduced.Then,the vertex coordinates of the virtual object are also transformed into the clipping space of the virtual camera to obtain the depth value of the corresponding position of the vertex in the virtual shadow map.Finally,whether to render the shadow effect on the vertex is determined according to the size of the vertex depth value and the corresponding position depth value in the shadow map.Experimental results show that the proposed video shadow detection algorithm under mobile viewpoint has the highest comprehensive performance compared with the latest research results,and is superior to the existing methods in detection accuracy and edge continuity.From the final virtual real fusion effect,it can be seen that the outdoor virtual real shadow interaction framework for mobile augmented reality designed in this paper can complete the outdoor virtual real shadow interaction without the complete geometric structure of the scene,and improve the realism of the augmented reality system.