Research On Distant Object Selection Technique Based On Cross-device Interaction In Augmented Reality

Target selection is one of the fundamental tasks in human-computer interaction,and the most common ray selection methods in AR environment are gaze-based gesture and handbased ray-cast.However,these two methods have many drawbacks,especially in distant small target and dense target selection tasks with high fatigue and the Heisenberg Effect,resulting in a significant reduction in the speed and accuracy of target selection.To address these problems,this paper proposes two target selection techniques based on cross-device interaction to make up for the shortcomings of the gesture recognition scheme that comes with the AR glasses,combining multiple input channels to take advantage of the strength of each device in gesture perception.The techniques use the idea of progressive refinement and the cone-cast method,in which the user completes one selection through two stages of range selection and fine selection to improve the efficiency of small target selection at long distances.This paper also provides a fine selection stage target judgment algorithm designed for each technique,which can calculate the object the user wants to select by the user's hand position,movement direction and twist angle,while reducing the user's hand movement distance.Both techniques can achieve complete inside-out perception and can still be used in outdoor environments.The two techniques are based on the principles of improving efficiency,maintaining the naturalness of interaction and immersive experience,and are divided according to the ability to participate in interactive devices.One is Watch Cursor,based on AR glasses and smartwatches,in which users complete target selection by moving their hands in space.This can be done with one hand and has a lower requirement for equipment.The other is Eye Watch,based on eye tracking and smartwatches,in which users select targets by eye movement and wrist rotation,and use both hands to complete the operation,with the left and right hands controlling the rotation angle and performing confirmation gesture respectively.In the first technique,this paper determines key parameters and completes the final design by experimentally studying the jitter of objects of different depths selected by users using traditional hand-based ray-cast and the relationship between hand movement distance and object differentiation efficiency.In the second technique,this paper optimizes the selection technique and determines the corresponding parameters by examining the user's wrist rotation stability and eye-movement jitter.After the design is completed,this paper follows the basic principles of measuring target selection,refers to relevant studies on target selection in 3D environments,and designs experiments combining the characteristics of the techniques to evaluate the efficiency and user experience of the two selection techniques.The results show that the techniques presented in this paper has a significant improvement in the accuracy and average time of selecting distant small targets compared to the generic method.They have a certain improvement over the classical two-step selection method and have the advantage of maintaining immersion and are more suitable for AR environment,where the second technique is less fatiguing and has privacy benefits.The results are useful for the design of selection methods in AR environment.Finally,this paper also implements a modular communication platform for cross-device interaction,which supports arbitrary hardware and software extensions to fuse data from each input channel;some applications are proposed for the characteristics of the technique,such as sector menus and target operations,which are used to demonstrate the potential of combining this selection technique with other operations.