Research On Key Technologies Of Automatic Ultrasonic Robot System Based On Multi-mode Information Fusion

Ultrasonic imaging is one of the most widely clinical application of medical imaging technology,with a small damage to the human body,characteristics of operation process is simple,but with COVID-19(COVID-19)is popular all over the world,medical personnel in the ultrasound examination need to frequent contact with infected patients for interrogation,medical personnel to bring huge risk of infection.In order to reduce the incidence of COVID-19 infection among medical staff and reduce the workload of medical staff,it is urgent for intelligent ultrasonic robots to replace medical staff to perform ultrasonic scanning work on patients.However,the existing intelligent ultrasonic robot is often difficult to realize the automation of ultrasonic scanning,and only makes use of the single visual information of the environment,and fails to fully utilize and combine the modal information of vision and touch in the process of ultrasonic scanning.Based on the above background,this paper builds a set of automatic ultrasonic robot system based on multi-modal information fusion,which combines visual and tactile information to assist ultrasonic robot decision-making.Specific contributions are as follows:(1)In order to determine the scope of ultrasonic robot scanning,an object detection algorithm based on Anchor free,TTFNet-Lite,was proposed to detect the chest and abdomen of patients.The algorithm is improved based on TTFNet object detection model.TTFNet-lite object detection algorithm first changes the TTFNet backbone network from Darknet53 to Mobilenet V3-large,which greatly reduces the number of parameters and the size of the model.In addition,TTFNET-Lite also uses Cut Mix and other data enhancement algorithms and operations such as variable convolution.Experimental results show that TTFNET-Lite object detection network has shorter training time and higher detection accuracy than TTFNet object detection algorithm,and can complete the task of real-time detection of patients’ chest and abdomen objects.(2)In order to judge the state of the ultrasonic probe grasped by the ultrasonic robot in real time and control the stability of the probe in the ultrasonic scanning process,a real-time haptic perception and classification method based on the broad learning system is proposed.In this paper,the robot terminal tactile data is collected and preprocessed,the grasping state data set is made,and the broad learning system is introduced to train the data set.The trained broad learning system achieves the expected goal of real-time detection,provides tactile modal feedback for the system,and further enhances the robustness of the automatic ultrasonic robot.(3)A set of multi-mode information fusion automatic ultrasonic robot system is built.Firstly,the scanning system of ultrasonic robot is studied,including scanning range determination and scanning trajectory planning.In order to determine the scanning range,filtering,registration and loophole filling are carried out on the original depth image.Meanwhile,scanning strategy is designed and scanning trajectory is planned.Then,the three subsystems of the automatic ultrasonic robot system are integrated,and network and data transmission standards are configured.Finally,after the software and hardware platform is deployed,the functional test is carried out.The experimental results show that the system can complete the task of automatic ultrasonic scanning of robot.