Research On Visuo-haptic Augmented Reality System For Upper Limb Functional Rehabilitation

The visuo-haptic augmented reality technology enables users to see and touch virtual objects in a real-world environment by delivering haptic sensations on the basis of augmented reality.With the technology,users can interact with virtual objects in a more realistic and natural way.Its application to upper limb rehabilitation offers not only advantages similar to those of the augmented reality technology in rehabilitation training,but also a variety of sensations and consciousnesses,which help re-integrate the high-level cortical centers of the brain and establish new channels of information receipt and processing.Therefore,the visuo-haptic augmented reality technology is of great research significance.By integrating visual and haptic functions in the augmented reality environment,the paper has designed and developed an augmented-reality upper limb rehabilitation training system based on vision and haptic.The paper is organized as follows:Firstly,existing literature on the key elements of the visuo-haptic augmented reality technology is reviewed.The paper uses the Geomagic Touch force feedback device to provide haptic feedback.To facilitate upper limb rehabilitation training,the paper proposes a threedimensional registration algorithm for the technology,which registers a haptic agent and virtual environment into the real-world environment.This realizes the integration between marker,camera,screen,and haptic device coordinate system.Visuo-haptic integration in an augmented reality environment is achieved through three-dimensional registration and force haptic rendering.Secondly,to provide patients with sensory retraining,two physiological and psychological experiments are designed based on the visuo-haptic augmented reality technology for the purpose of proving the advantages of visuo-haptic integration.This experiment innovatively combines the visuo-haptic augmented reality technology with the experimental psychology method to analyze visuo-haptic integration from the perspective of psychology.The effects of the haptic function on the accuracy and performance of the system are verified through fixedpoint experiments and trajectory experiments.Variance analysis is performed on the data using the one-factor repeated measures method.Experiments show that visuo-haptic integration helps improve the accuracy and performance of the system.Finally,the paper applies the visuo-haptic augmented reality technology to the field of upper limb rehabilitation,develops and designs a rehabilitation training system,and designs two training programs with different difficulty based on daily activities: trajectory training and obstacle training.The trajectory training involves force feedback and non-force feedback.The obstacle training can be conducted in three ways,based on the number of obstacles.The two training activities are designed mainly to improve the finger force,the flexibility of the wrist joint,and the range of motion of the elbow joint and shoulder joint.Patients can feel force feedback during the training.Patients' sensory retraining through multi-channel feedback facilitates recovery of their perceptive functions.At the end,the system is tested and a questionnaire is evaluated and analyzed through experiments to demonstrate the feasibility and practicability of the system.