Research On Mixed Reality Based Training System Of Minimally Invasive Surgical Robot

Robot-assisted minimally invasive surgery has been widely adopted in hospitals due to its advantages such as small wound,mild pain and short hospital stay.The training system is indispensable for cultivating a qualified minimally invasive surgeon.However,the traditional training process is operated on animals or dead bodies,which leads to costly and ethical problems.The current virtual reality training system could solve these problems,but it does not provide enough immersion for the trained doctors.Therefore,we innovatively invent a training system based on mixed reality to further improve the immersion of the training system.In order to obtain the virtual operation objects,the three-dimensional reconstruction technique of soft tissue is studied.First of all,the CT image data of patients before the surgery are preprocessed with threshold segmentation,equalization and median filtering in succession,so as to improve the image contrast and filter out the noise.Secondly,the active contour model method and the region growing method are used to segment the soft tissue,and we conclude that the region growing method is more suitable for complex multi-contour image segmentation from the segmentation results.Finally,the threedimensional model of soft tissue is reconstructed by marching cubes algorithm.Virtual and real collision detection is the key to realize virtual and real interaction in mixed reality.In this paper,a virtual and real collision detection algorithm is designed to detect whether the actual surgical instrument is in contact with the virtual soft tissue.The algorithm is divided into two steps: first,create a virtual surgical instrument and register it with the real surgical instrument in space;second,replace the real surgical instrument with it and complete the collision detection in the virtual environment.Since the virtual and real space registration require to solve the kinematics model of the master manipulator,and the master manipulator is a decoupling structure of the series and parallel mechanism,the position solution and attitude solution are conducted separately.The closed-loop vector equation method is used for the position solution,while D-H parameter method is used for the attitude solution.In the virtual environment,an octree based collision detection algorithm is designed to adapt to the feature of soft tissue deformation,which greatly improves the speed of collision detection under the condition of ensuring the accuracy of collision detection.Finally,the training system of minimally invasive surgical robot is gradually improved and integrated,and relevant experimental verification is carried out.Firstly,in order to solve the elastic equation of soft tissue quickly,a neural network model is established to predict the deformation of soft tissue,which ensures the deformation accuracy and real-time performance.Secondly,in order to realize the information sharing between the master manipulator control program and the virtual environment,a data transmission channel is established by using Socket communication.Then,three demensional registration technique is used to seamlessly blend the virtual soft tissue with the real environment.Finally,mixed reality system is integrated and the relevant mixed reality experiment is carried out.