Research On Text Entry Technology Based On Gesture Interaction In Virtual Reality And Augmented Reality

With the gradual development of virtual reality and augmented reality technologies and the increased interest of major technology companies and the public,two types of immersive interaction are now used in various fields such as gaming,education and medicine.As the most basic interaction in use,text entry is often used by users to enter passwords to log in to systems,or communicate with players during games,or edit the content of documents.Although there is a lot of research on text input in both environments,there is still space for improvement in terms of efficiency and usability.Currently the most popular text entry method aim-and-shoot in the virtual reality environment requires the user to move the laser shot from the controller handle to select letters one by one on the virtual keyboard,which is tedious and inefficient,while the long hovering movement of the arm can easily lead to fatigue,and is not suitable for long time input scenarios.The most popular click entry method in the augmented reality environment also has a high load on the body,in addition to the virtual keyboard occupies most of the area in the center of the screen,blocking the physical environment and limiting the user’s perception of the surrounding environment.To solve the problems of text input in both environments,two word-level text entry techniques,RelativeSwipe and ImaginarySwipe,are proposed in this paper.The RelativeSwipe approach aims to solve the problems of low input efficiency and poor usability in the virtual reality environment,where the user draws a straight line in the corresponding direction based on the relative positions of two letters in the target word on the touchpad of the controller.The system recognizes the gesture and gives a list of candidate words,and the user can use the six-letter group disambiguation method to filter the candidate words so that the target word can be selected quickly.This method is scale and location independent,the user does not need to place the finger in a specific location.Second,the approach retains the benefits of gestural input,and users can gradually transition from relying on the visual guidance of the virtual keyboard to directly recalling gestures.The ImaginarySwipe approach aims to solve the problem of inefficient input and environmental occlusion in augmented reality environments.The user imagines the transparent box in front of him as an invisible keyboard,traces through the corresponding positions of the letters in the word in turn,and lifts his fingers to end the input.The gestures drawn by the user are matched with the template in shape and absolute position based on the SHARK~2 algorithm,and the user can click on the candidate word to replace the entered content directly.This method has the same advantages of gesture input,and a continuous stroke input has less physically load than a letter-by-letter click.Secondly,there is no obstruction of letter keys in the keyboard,so users can see their surroundings unobstructed.After completing the design of both,we experimentally evaluated the two approaches in terms of input speed,error rate,and interaction experience based on the guidelines for text input performance evaluation.The experimental results show that the RelativeSwipe method can reach 19.90WPM(Words Per Minute),which is faster than the aim-and-shoot method by 6.99WPM.The ImaginarySwipe method can reach16.95 WPM,which exceeds the click input by 6.92 WPM,and has a significant advantage in terms of physical demand and time demand.The experimental results have implications for text input techniques in virtual reality and augmented reality environments.