Research On Key Technologies For Human-Robot Collaboration In Ultrasound Diagnosis

Currently,ultrasound diagnostic technology has become one of the most commonly used imaging technologies in the medical field.However,traditional ultrasound diagnosis is still limited by the skill level of doctors and equipment.Therefore,researchers have begun to explore ultrasound scanning robot systems.However,previous ultrasound robots could only perform either remote ultrasound scanning or automatic ultrasound scanning.To meet the ultrasound scanning needs in different scenarios,this thesis focuses on the topic of human-robot collaborative ultrasound scanning and designs a robot system that can perform both remote and automatic ultrasound scanning.Specifically,the work is as follows:Firstly,in order to perform remote ultrasound scanning,this thesis proposes a master-slave remote operation method based on the Leap Motion sensor and UR3 robot arm.The mechanical arm with the ultrasound probe tracks the operator’s left hand posture through Leap Motion,improving the naturalness and flexibility of human-machine interaction at the master end.To interact with the system more efficiently during remote ultrasound scanning,this thesis proposes a dynamic gesture recognition method based on Leap Motion and Bi LSTM neural network model,and establishes a set of dynamic gesture library.The system will collect real-time samples of the operator’s right hand dynamic gestures and recognize them.After successful recognition,it will automatically execute the corresponding function.The recognition accuracy of this method reaches 98.42% after testing.Secondly,in order to obtain the trajectory of the robot arm end effector during automatic ultrasound scanning,this thesis uses the Kinect depth camera to capture RGB-D images of the human body and establishes a human point cloud model.Due to the low accuracy of the Kinect depth camera,it is necessary to repair the holes in the depth image.Therefore,this thesis proposes a depth image repair method based on clustering color-guided images.The human body is extracted from the background using the random sample consensus algorithm and the DBSCAN clustering algorithm on the point cloud model established based on the depth image to prevent the background and other objects from affecting subsequent work.Thus,the human point cloud model is successfully established.In addition,this thesis also analyzes the inverse kinematics of the UR3 robot arm and proposes a control strategy for the robot arm during remote and automatic ultrasound scanning.Incremental space mapping will be used during remote ultrasound scanning to obtain a more flexible operating experience.During automatic ultrasound scanning,trajectory planning will be performed on the human point cloud model,and the normal vector on the trajectory will be obtained using the least squares local plane fitting method as the posture of the robot arm end effector during scanning.The feasibility of the system is verified by MATLAB simulation in this thesis.Finally,a human-robot collaborative ultrasound diagnosis system is built in this thesis,including hardware and upper computer software parts.The final experimental results show that the system can achieve the goal of human-machine collaborative medical assistance.