Research On Virtual Real Fusion With Geometric Consistency Based On Two-dimensional Affine Transformation

Augmented reality technology,also referred to as AR technology for short,is a new technology that seamlessly integrates real-world information and virtual world information.It superimposes physical information(visual information,sound,taste,touch,etc.)that is difficult to experience in a certain time and space of the real world through computer,and integrates virtual information to the real world to be perceived by human senses,so as to achieve sensory experience beyond reality.Augmented reality(AR)can be widely used in industrial maintenance,military,medical,entertainment,education,tourism,online video communication and historic site restoration and cultural heritage protection etc.With the maturity of AR technology,the integration of AR and industry is more and more in-depth.Virtual-real fusion is the key technology of augmented reality and the effect of fusion directly affects the user experience of augmented reality.Most of the traditional virtual real fusion methods use the auxiliary marking method or the image feature point matching algorithm.These methods consider the occlusion relationship between the virtual object and the real world at the three-dimensional level.Due to the limitations of auxiliary marking method and less feature point matching between adjacent frames,the realism of virtual reality fusion will be reduced,so it is impossible to accurately achieve geometric consistency.Taking the human wrist wearing a virtual wristwatch in the real scene as an example,this thesis proposes a geometric consistency virtual reality fusion method based on two-dimensional affine transformation,which is as follows:(1)Firstly,the real scene is photographed by Kinect camera,and the scene is reconstructed by Kinect fusion algorithm.Then the scene is segmented to segment the human arm area that needs to accurately calculate the geometric consistency,and then the area is projected onto the two-dimensional plane by the method of threedimensional to two-dimensional,and the position and width of the wrist are determined on the two-dimensional plane.At the same time,the two cases of the arm facing the camera and not facing the camera are discussed.The position and width of the wrist in both cases are determined by two-dimensional affine transformation algorithm,so that the position and scaling of the virtual watch can also be determined.Finally,the rotation angle of the virtual watch is determined through the unification of the coordinate system.(2)The virtual wristwatch is rendered to the real scene through the differential rendering algorithm,and the above determined parameters such as position,scaling and rotation angle are substituted into the algorithm to obtain the final virtual real fusion results.The proposed geometric consistency virtual real fusion method uses the transformation from three-dimensional space to two-dimensional space to calculate the geometric relationship between the virtual object and the real scene in detail,and applies two-dimensional affine transformation to process two-dimensional images to achieve more accurate calculation.Finally,combined with the virtual reality fusion with illumination consistency,the virtual object is accurately inserted into the real scene through the differential rendering method,and a virtual reality fusion system with both geometric consistency and illumination consistency is realized.The experimental results show that the virtual object can be fused into the real scene completely according to the size and volume,which verifies the effectiveness of the proposed method.