Research On Compliant Control Of Medical Robots Based On Deep Reinforcement Learning

Medical robots refer to robots with autonomous control and automatic execution of medical tasks,which are widely used in the field of healthcare.With the continuous breakthrough of artificial intelligence technology and robot technology,medical robots are also constantly developing and improving,which can effectively help doctors to carry out a series of medical diagnosis and auxiliary treatment,and alleviate the current shortage of social medical resources.Considering that most medical tasks are complex and diverse,it is difficult to accurately model the environment,so it is very important to have the ability of autonomous learning,so that the manipulator can adjust its own state and complete the task according to the change of the environment.As a revolutionary technology for decision control,deep reinforcement learning can train the control strategy through continuous trial and error,and learn the optimal strategy through the guidance of the designed reward function.This method provides a new idea for robot control.Therefore,this thesis mainly focuses on the deep reinforcement learning algorithm to achieve the complete control process of the manipulator for the set medical tasks.Based on this,the main research contents of this thesis are as follows:(1)Aiming at the problem of controlling the manipulator to complete the medical auscultation task,a constant force tracking method based on deep reinforcement learning is proposed.In order to clearly obtain the sound signal in medical auscultation,it is necessary to consider the ups and downs of the body surface caused by breathing.Therefore,during the auscultation process,it is necessary to control the manipulator to continuously adjust the position of the end effector to track the changing contact position and maintain a constant contact force.Therefore,in order to simulate the real auscultation scene,a dynamic auscultation simulation environment is specially designed,and different changes can be simulated by changing the parameters of the state change function.At the same time,in order to achieve efficient training of control strategies in medical auscultation tasks,according to the specific requirements of auscultation tasks,the state function and reward function are optimally designed,and the proximal strategy optimization algorithm is used to complete the training of the manipulator.(2)Aiming at the problem of controlling the manipulator to complete the nursing massage task,a constant force tracking method based on deep reinforcement learning is proposed.Considering that the manipulator only involves one-dimensional motion in the medical auscultation task,in order to further extend the research content,the method is extended to the two-dimensional motion nursing massage task.Therefore,the nursing massage scene is constructed on the basis of auscultation simulation environment,and the purpose of training nursing massage control strategy is realized.At the same time,according to the specific requirements of the nursing massage task,the state function and reward function are optimized to realize the efficient training of the control strategy under the task.Finally,the training of constant force tracking control strategy in medical auscultation task and nursing massage task is completed in Pybullet,and a large number of tests and generalization experiments are carried out in the simulation environment.In addition,the algorithm is verified on the real manipulator.The experimental results show that the algorithm can achieve good constant force tracking effect.