Research On Upper Limb Rehabilitation Technology Based On Virtual Training And Visual Recognition Of Movement

The upper limb hemiplegia is a common stroke sequela which can be treated andrecovered through a long-term function training. In recent years, an extensive researchon the robot-aided upper limb rehabilitation technology was carried out since it canprovide a long-term and stable repetitive training practice and precisely record thetraining process, which overcame the traditional therapies’ shortcomings e.g.time-consuming and labor-consuming. However, an effective rehabilitation trainingshould involve the task-oriented motion and mobilize the patients’ initiatives. In thisthesis, the eventual goal is to establish an effective upper limb rehabilitation systembased on a5degrees of freedom (DOFs) arm rehabilitation robot, in which atask-oriented virtual training environment is developed and a bilateral mirroring trainingpattern based on visual recognition and motion mapping is also designed.In order to design an open3D virtual training scene with flexible perspectives, itwas explored to introduce VR technology to build up and run a robot system based onthe ROS-Gazebo software architecture. A vivid virtual human was used in the virtualenvironment as the controlled object in order to improve the sense of reality during thehuman-machine interaction. The virtual human has5DOFs and the same kinematicmodel with her upper limb corresponding to the real robot prototype so that they couldachieve completely synchronous motion. Since the virtual human was introduced to thevirtual world as an anthropomorphic robot, the position controller at each limb’s jointwas reinforced to enhance its motion consistency with the robot prototype. Moreover,the reaching movement was designed as the oriented-task, which mobilizes the patients’active motion efforts and practice interests.A Kinect depth camera was employed to implement the motion recognition of thehealthy side of the upper limbs via the vision capture, which can realize the bilateralarms’ synchronous movement by mirroring the motion of the healthy side to the otherside. Based on the Kinect sensor’s software and hardware system, the spatial positioninformation of the patient’s healthy arm’s joints was obtained according to the pipelineprocedure of Kinect skeleton tracking function. Since the healthy side and the affectedside under assistance of the rehabilitation robot own7and5DOFs respectively, ahealthy-affected arm motion mapping strategy based on the optimization concept wasproposed to reduce the asymmetry between the two arms. The optimization wasperformed respectively based on minimizing the position differences and minimizingthe direction approaching before and after mapping, which is validated by comparingthe experimental data.Finally, the architecture and implementation of the new upper limb rehabilitation system were studied, which consisted of four main components, i.e., the robot, thecontrol machine, the VR machine and the Kinect sensor. To ensure the fast and real-timeinformation exchange between the robot and the virtual human, the two host machineswere connected through Ethernet and communicated in ROS framework. For thepurpose of coordinating and managing each hardware component, the software systemof the control machine was also established, to publish the real-time robot position andcreate a user interface to control the rehabilitation movement. Furthermore, the newsystem’s overall performance and the active mirror training mode are evaluated inrelevant experiments, and the visual motion capture and identification method based onKinect was also tested, which validated the effectiveness and feasibility of the proposedmethodology.