Research On Monocular Reconstruction And Voice Control For Robot-assisted Minimally Invasive Surgery

In minimally invasive surgery,it is usually necessary for the assistant doctor to hold the endoscope to provide the surgeon with the surgical field of view to complete the surgical task.On the one hand,the two-dimensional image transmitted by the endoscope lacks depth information,and the doctor cannot intuitively obtain a good surgical scene.On the other hand,the inconsistent actions of the assistant doctor and the chief surgeon will increase the difficulty of the operation and even cause a safety accident.With the development of robot technology,surgical robots applied in the medical field are becoming more and more popular.Surgical robots can replace doctors to perform some tedious and labor-intensive operations,reduce the difficulty of surgery and improve the safety of surgery.Based on the research background of robot-assisted minimally invasive surgery,a three-dimensional imaging system for a surgical robot based on voice control is established in this paper,where the monocular image sequence is used to restore the three-dimensional structure of the soft tissue to provide the doctor with an intuitive surgical scene and a surgical robot control method based on voice command solves the problem of uncoordinated cooperation between the assistant doctor and the chief surgeon,as well as improves the safety of the operation to a certain extent.Restoring the three-dimensional model of soft tissue using image sequences acquired by monocular endoscopes is the core task of the 3D imaging system of surgical robots.This paper uses a monocular camera to capture external soft tissue images from multiple locations,analyzes soft tissue image feature information,and proposes a low-resolution feature screening and cascade hash search image feature matching strategy,which improves the efficiency of feature matching operations.After obtaining the 3D point cloud model of soft tissue,the paper further densifies and meshes the point cloud model,and transforms the disordered point cloud into a surface mesh model with topological structure.In order to improve the visual effect of the soft tissue reconstruction model,this paper uses the texture mapping method to further process the mesh model,and proposes an improved gap repair model for the phenomenon of texture gap,and has achieved significant improvement effects.Realizing the interaction between the doctor and the surgical robot can avoid the problem of coordination with the assistant staff.This paper proposes to use the doctor's voice command signal to control the surgical robot arm to complete the corresponding action.This kind of human-computer interaction can reduce the difficulty of doctor's surgery to a certain extent.According to the requirements of the actual scene,this paper designs a speech recognition system based on deep neural network,and combines the actual robot arm control module to build a kind of voice control system of surgical robot.The input of the speech recognition system is the speech signal spectrogram feature which is extracted artificially,and the output is the nine motion commands of the surgical robot arm.This paper further combines the voice control module and 3D imaging system of the surgical robot,builds the AUBO robotic arm experimental platform,verifies the feasibility of the voice controlled surgical robot arm,and launches the 3D reconstruction test experiment under the voice control robotic arm mode.The results show the feasibility of the 3D reconstruction system of surgical robot based on human-computer interaction mode of voice signal.