Human-computer Interaction Method Based On Planar-aid For Virtual Reality Environment

With the development and application of virtual reality,how to interact with virtual environment naturally and efficiently has always been a hot issue of research.Currently,using the handle and joystick to interact with the virtual scene is the mainstream method of virtual reality interaction.However,the interaction range of the handle-interaction method is limited and cannot deal with the occlusion problem between objects;the joystick interaction method impairs the authenticity of virtual reality interaction and has a high learning cost.A Planar-aid is used to introduce the interactive paradigm of the smart tablet into a head-mounted virtual reality environment in this paper.And an efficient interaction system is implemented by tracking the user hand and Planar-aid efficiently with a single RGB camera.Main work of this dissertation is as follow:(1)A human-computer interaction system based on Planar-aid is designed and implemented with Unity3 D engine.And experiment and analysis are carried out based on the system.(2)A single RGB camera is used to estimate the user's hand posture.And for the defects of the model-driven hand posture estimation method,a RNN network is used to augment the result of it.(3)For the interaction error caused by the size difference between the virtual and the real interaction plane,an improved haptic retargeting solution is proposed.The solution enables the interaction operation to switch between adjacent targets continuously,and also can effectively prevent virtual hand from shaking during switching.