Research On Key Tracking And Registration Technology In AR-Based Upper Limb Rehabilitation System

Augmented Reality(AR)is a new computer technology,which makes an exact fusion of virtual scenes and the reality to enhance real scenes.Tracking and Registration is one of the key technologies for AR to achieve the fusion of virtual and realities.AR technology in upper limb systems works by enhancing one’s current perception of reality,overlaying the computer-generated coaching information on the real exercise environment,which shows the patient steps clearly and correctly to adjust the training plan in time.Researches on the key technology in AR-Based upper limb rehabilitation system is further explorations for the methods and application of the technology,which provides a method for difficulties of applying AR in rehabilitation medicine and a solution of the key technology for the development of AR-Based upper limb systems.Therefore,the work of this thesis has important theoretical significance and application values.Markers and computer vision algorithms in existing AR-Based upper limb systems are able to track the movements of joints,but the related data cannot be obtained so that the training effect can’t be evaluated.Besides,a bad rehabilitation will be caused because of mental defensive led by putting markers on the upper limbs of patients.For these problems,on the basis of deep researches and analysis on AR tracking and registration,the main work of this thesis is as follows.(1)Analysis on the data for mix of virtual and reality.The positions of sensors are confirmed by analysis of body postures and the characteristics of motions.And the angles of motions are used as the data for mix of virtual and reality by combining ROM(Range of Motion)Test.(2)Implementation of tracking and registration for wrists and elbows with sensors.Because of the gradual intelligence and micromation,the sensors can be put on wrists and elbows,which is easy to accept.The ulnar deviation and radial deviation of wrists,flexion and extension of elbows can be modeled as the rotations around axes in three-dimensional space based on human mechanical structure.And the rotations of sensors with the motions of upper limb,the directions and angles of sensors are correspondence with the directions and angles of joints,which establishes a mathematical model for the angles.And the angles of motions will be calculated by the rotation angles.Finally,a tracking and registration algorithm for the motions of wrists and elbows is proposed by combing the repeatability and continuity of the rehabilitation training process.(3)Validation of effectiveness and usability.For one thing,the healthy,test patients and real patients are selected to evaluate the effectiveness of motion data from the aspects of sensitivity,accuracy,continuity and stability.The accuracy is evaluated by contrasting the data from sensors with the data of the same motion measured by a protractor used in clinic.The sensitivity,the continuity and the stability are evaluated by analyzing the data change curve produced by the prototype.For another,a prototype is designed and developed by oneself,and the experiments are conducted to validate the effectiveness further at the aspects of the usability of the hardware and the software in hospital.The usability of the hardware is evaluated at the aspects of ease of use,acceptability,comfort and satisfaction.And the usability of the software is evaluated at the aspects of ease to learn,ease to memorize,fault-tolerance,efficiency and satisfaction.The theoretical values of this thesis are as follows:(1)The method of modeling the motions of upper limb in three-dimensional space is proposed for the first time.(2)The work of this thesis is the result of the combinations of AR and rehabilitation medicine,which provides theoretical guidance and reference for AR training of lower limbs motor dysfunction to a certain extent.The practical values of this thesis are as follows:(1)The proposed algorithm is applied to AR upper limb rehabilitation system,which can support the fusion of virtual and reality and can used as the important basis for doctors and therapists to evaluate the recovery.(2)The designed AR upper limb rehabilitation system prototype based on Android platform shows the last training goal,the current training process and the current training goal,which is helpful to contrast clearly and to know the training situations for patients and doctors.The work of this thesis achieves the stable tracking of wrists that can support the registration further,but sometimes the motions of elbows are caused by the motions of wrists for some patients in experiments.Thus,the next work is to analyze the joints that perhaps cause motions of elbows,which can support the rehabilitation assessment and the fusion of virtual scenes and real scenes better.