Research On The Application Of Visual And Haptic Augmented Reality Technology In Rehabilitation Training

Augmented reality is a technology that enhances the real world by overlaying computergenerated perceptual information or virtual objects into the real world and provides an experience of interacting with the real world.Haptic feedback is achieved by using haptic device to interact with the virtual environment in the same way that humans perceive information.Visuo-haptic augmented reality combines augmented reality technology and haptic feedback,which enables operators to interact with virtual objects in real scenes through haptic devices by adding force and haptic perception in the augmented reality environment.Visuohaptic augmented reality,once applied to the rehabilitation of hand functions,can enable patients to do rehabilitation training under both visual and touch stimulations,and improve hand dyskinesia.The application of this technology not only provides patients with rich interaction scenarios,but also provides real feedback force,which improves the enthusiasm and initiative of patients to participate in the training.Based on the project of National Natural Science Foundation of China,this paper,combining computer vision and force/haptic rendering,develops a hand function rehabilitation training system based on visuo-haptic augmented reality,and designs rehabilitation training projects with different feedback forces.The main research contents of this paper include the following aspects:Firstly,the paper analyzes and summarizes the research status of visuo-haptic augmented reality,then proposes an improved ORB(Oriented Fast and Rotated BRIEF)and KLT(Kanade Lucas Tomasi)AR tracking registration algorithm,on which an AR environment is built.Compared with the current methods based on natural features,the algorithm in this paper improves the real-time performance of the system.The haptic device is integrated into the augmented reality environment,the 3D registration of the visuo-haptic workspace is proposed.Based on the rigid conversion of the visuo-haptic workspace,the position of the stylus of the haptic device is calibrated,and the virtual interactive tool is superimposed on the real stylus.At the same time,the direction of the stylus is calibrated,and the motion posture of the stylus is integrated to realize the coordinated movement of the real stylus and the virtual interactive tool.Haptic devices occupy a certain position in the scene,resulting in visual barriers and interference to the operators of haptic interaction.In order to achieve a more realistic and immersive augmented reality environment,this paper combines matting algorithm and image inpainting algorithm to remove and hide the haptic device.Secondly,the kinematics modeling of the 3D Systems Touch haptic device used in this paper is done to obtain the forward and inverse kinematics equations.The paper proposes improvement on calculation method of feedback force,offers a solution.to solve the collision problem between the haptic interaction point and the virtual object.At last,this paper,applying visuo-haptic augmented reality to the field of rehabilitation,,designs a hand function rehabilitation training system,which combines the haptic rendering based on the feedback force,and includes two different types of rehabilitation training programs:bouncing ball and writing.The bouncing ball training program is aimed at improving the free movement ability of the hand and wrist joints of the upper limb,and the training of the strength of the hand.The writing training mainly exercises the grasping ability of the finger and the ability of force and haptic perception.Finally,the system is tested through experiments to demonstrate the feasibility and practicability of the system.