Research On Real-time Interaction Method In Virtual Reality

Virtual Reality(VR)technology is to produce a 3D virtual world with computer simulation,allowing users getting immersive feeling through visual,auditory,tactile and other senses.Immersion,interactivity and conception are the three basic characteristics of VR system.In recent years,with the fast improvement of smart phone performance,mobile phone-based VR system(we call it "mobile VR system")have been getting more and more favored because it is portability,ease to use,low price and its support the user's free move in the physical space.Thus,it is widely used in the fields of game,education and other field.However,due to its capacity of calculation,storage and other factors,the system still faces many challenges in real-time interaction.First of all,the VR system puts more emphasizes on the naturalness of interaction.Obviously,the traditional mouse-based and keyboard-based interaction cannot meet this requirement.A more natural way of real-time interaction is that the user leads the corresponding changes in the virtual scene with their own movement in physical space.Therefore,tracking the user's location is crucial for such a VR system.The accuracy of position tracking will directly affect the user immersive experience.In practical applications,taking into account factors like portability,cost,ease of deployment,we commonly use a single RGB-D camera(such as Kinect)for tracking.However,in the VR system which supports multi-person collaborative interaction,there must be a blocking problem for multi-person tracking when using a single camera.When the blocking is long and complete,there is no good solution.In this paper,we study how to solve the problem of long-term and complete blocking in order to accomplish multi-person tracking with a single RGB-D camera and support the real-time collaborative interaction between multiple people.In addition,the speed of virtual scene rendering is also an important factor affecting the real-time interactive experience.At present,the number of VR systems based on smart phones and other mobile devices become increasingly large.Users of applications such as virtual museum system may be different so are the equipments.Subject to different equipment performance constraints,the size of virtual scene showed by various types of system is still relatively limited.The current modeling approaches rarely consider the diversity of different equipments and the actual operating capacity,so the applicability is poor.To fulfill this kind of demand,this paper studies the virtual scene optimization strategy,which can adapt to the real-time interactive display of different devices,in order to realize the virtual scene of different scale in real-time interactive devices on different models and different running performance.We've also explored a new VR model that supports user's creative design and multi-channel natural interaction.At present,in the common virtual scene design system,the virtual scene is generally not designed by the ultimate user,but by the system provider in advance.However,encouraging users to express their own creativity and experience their own creation of virtual scenes become a new demand.To meet this end,this paper presents a new model of mobile VR that supports user's innovation design and multi-channel natural interaction experience.It provides users with support for virtual scene content design.Users can quickly realize the construction of the virtual world using daily marking tools(such as chalk and ribbon)and smart phones.They can also experience their own design of the virtual scene for natural interaction through the integration of physical space and virtual space.To conclude,the innovation and the main contributions of this paper include the following aspects:1.A multi-user real-time location tracking algorithm integrating shadow cues is proposedTo solve the problem of multi-person tracking by a single RGB-D camera in a VR system for multi-person collaborative interaction,this paper introduces the user shadow,which is often removed as a noise or interference factors in existing approaches,as an important clue to track the blocked user for the first time.A new multi-person location tracking algorithm based on RGB-D camera is proposed to realize real-time tracking of multiple users by combining the shadow information and skeleton data of users captured by RGB-D camera.When the RGB-D camera can detect the user skeleton,we use the skeleton data for user detection and tracking directly;when the camera can not,we will use the user shadow image information for location estimation.Without additional sensor equipment,this method can achieve real-time tracking of multi-person position with single RGB-D camera.The algorithm has been successfully applied to mobile VR systems,such as firefighting simulation training system and maze game system.Experimental results and application practice show that the proposed method can solve the problem of user tracking blocked circumstance of long time and complete blocking.In addition,the proposed method can also solve the user tracking loss problem not caused by blocking,so it enhances the Kinect's human body tracking ability.2.A virtual scene optimization strategy which can adapt to the real-time interactive display of different devices is proposedConsidering the diversity of equipment and the actual running ability,this paper proposes a virtual scene optimization strategy,which can rapidly draw indoor scenes such as maze on mobile devices,adapting to the real-time interactive display of different devices.The main features of this strategy are as follows:At the scene design stage,optimize the layout of virtual scene while taking the performance of mobile device as oneimportant factor in the design of 'virtual scene;set up special data structure and provide an accelerated drawing method based on the structure,to realize the real-time ability to draw large-scale scenes on different mobile devices.3.A new VR model is proposed to support user innovation design and multi-channel natural interactionThis paper also presents a new VR model that supports user innovation design and multi-channel natural interaction.It provides users with support for virtual scene content design,enabling users to use daily tagging tools(such as chalk and ribbon)to achieve rapid realization of virtual world structure,and experience their own design of the virtual scene through the natural integration of physical and virtual space.This model provides the user a new experience with good combination of both physical space and virtual spaceand innovative design and VR experience integration:fast and flexible VR content creation platform for the general user;real-time natural interaction of user and virtual world,in which the user's movement and posture will lead to virtual world changes;multi-person collaborative design and experience which enhance the experience of fun and participation,to ensure social interaction between users.In particular,this paper presents the concept of virtual scene creation in physical space with daily tools,which allows users to design virtual scenes quickly and flexibly.Our test results show that the new model proposed in this paper has the potential to promote the active participation of users,and can provide a richer,more interactive and immersive experience.