Research On Magnetic Levitation Visuo-haptic Interaction In Virtual Surgery

With the development of virtual reality and augmented reality technology,the interaction between human and machine is becoming more and more frequent.Visual interaction and haptic interaction are important components of human-computer interaction.The unprecedented explosion of current visual interaction researches has brought significant breakthroughs in the fields of medicine,entertainment,and industrial manufacturing.As an important sensory channel,haptic sensation plays an indispensable role in the interaction of human daily life.However,researches about haptic interaction in the field of human-computer interaction are far behind visual interaction,which leads to the lack of realism and interactivity of virtual reality applications.Especially for virtual surgery applications that require high visual and haptic interaction immersion,such as personalized preoperative planning,high-precision intraoperative navigation,etc.,it is urgent to employ realistic,stable and real-time haptic interaction methods.Compared with mechanical haptic interaction,magnetic levitation(maglev)haptic interaction has the advantages of no mechanical friction and natural interaction manner.However,the existing maglev haptic interaction methods lack reasonable design theory and have poor scalability.Moreover,the current virtual surgery applications usually adopt a general model,lacking a personalized soft tissue visual model and precise soft tissue parameters,which makes it impossible to establish a realistic soft tissue interactive virtual simulation scene for the maglev visuo-haptic interaction application.In addition,in the maglev visuo-haptic interaction system,the existing method cannot simultaneously achieve high frequency,high precision,strong stability and cost-effective method for position and orientation measurement,affecting the quality of information interaction between the haptic interaction device and the virtual scene,and making the visuo-haptic interaction lack realism and may even cause the problem of unstable interactions.In view of the above three problems,in order to successfully apply the maglev visuo-haptic interaction to the virtual surgery,it is urgent to study: high-precision maglev haptic feedback method,personalized soft tissue modeling method,and real-time accurate and stable high-quality method for position and orientation measurement.Therefore,this dissertation has carried out the research on the key methods of maglev visuo-haptic interaction in virtual surgery,including high-precision maglev haptic feedback method for enhancing soft tissue stiffness perception,an accurate segmentation method for soft tissue,a nonlinear parameter estimation method of soft tissue,and a visual-inertial fusion method for position and orientation measurement for improving the realism and stability of the maglev visuo-haptic interaction.The main researches are as follows:There is mechanical friction in traditional mechanical haptic feedback,which affectes the stiffness perception of soft tissue.The existing maglev haptic feedback method does not consider the rationality and scalability of the coil array design.To tackle these two issues,we propose a novel high-precision maglev haptic feedback method.According to the simulation analysis of the effective magnetic field region for the coil array,we propose a maglev haptic feedback method with an adjustable coil array to adapt to different requirements of soft tissue haptic interaction.Aiming at the problem that the current of each coil can be easily affected by the external environment and coil temperature,which will affect the stiffness perception of soft tissue,we propose an adaptive fuzzy PID algorithm to control the coil current.The experimental results show that the maglev haptic interaction device,which is based on the proposed maglev haptic feedback method with an adjustable coil array and coil current intelligent control algorithm,has advantages of no mechanical friction,high precision,low power consumption and strong expandability,and it can effectively enhance the stiffness perception of soft tissue.Because individual differences,lesions,etc.can cause changes in soft tissue structure and appearance,it is particularly critical to achieve precise segmentation of soft tissue and establish a personalized soft tissue model for immersive soft tissue visuo-haptic interaction simulation.Aiming at the problem that the existing convolutional neural network is prone to produce anatomically impossible segmentation results which will seriously affect the accuracy of soft tissue visualization model,we propose a recurrent interleaved attention network(RIANet).RIANet introduces the recurrent block between different layers with the same resolution,which greatly improves the ability of feature representation with a small number of parameters.Besides,the interlaced attention mechanism is proposed to fuse different-level features during the decoding stage,so that more distinctive features can be used for the reconstruction of segmentation results.Furthermore,the weighted deep supervision mechanism is utilized to train our network for further improving the segmentation accuracy.Experimental results show that RIANet can effectively reduce the anatomically impossible segmentation results and improve the segmentation precision and reconstruction accuracy of soft tissue.In the visuo-haptic interaction simulation of soft tissue,accurate estimation of soft tissue elastic parameters is essential for the realistic visualization of soft tissue deformation and immersive haptic rendering.The computational process of nonlinear finite element modeling is quite complicated and the soft tissue is usually modeled as the linear material,which leads to low precision.We propose a nonlinear parameter estimation model for soft tissue based on a parameter substitution strategy.We firstly use the weighted combination forecasting model based on support vector machine(WCFM_SVM)to correct the stress data so as to provide high quality data for the nonlinear parameter estimation.Secondly,we establish a finite element model for estimating nonlinear parameters of soft tissue(Young's modulus and Poisson's ratio)based on the idea of local linearization of nonlinear materials.During the modeling,a parameter substitution strategy is adopted to avoid solving complex nonlinear solving problems.Besides,the initial parameters of the linear finite element model are introduced to improve the robustness of the nonlinear model and avoid local minima.Finally,we propose a self-adapting Levenberg-Marquardt(LM)algorithm that can adaptively adjust the iterative parameters to solve our nonlinear parameter estimation model.The low-frequency method for position and orientation measurement in the maglev visuo-haptic interaction system will lead to the lack of realism and stability of the haptic feedback,which affects the immersion of maglev visuo-haptic interaction.In view of this problem,we propose a ?-increment learning method based on cascaded network,which constructs a cascaded network to predict the increment of variable ? in a small time step.We propose a visual-inertial fusion method for position and orientation measurement using the ?-incremental learning method,and construct a cascaded network OCasNet and a nested cascade network PCasNet to perform high-frequency estimation of the increments for orientation and position data,respectively.Experimental results show that the proposed ?-incremental learning method based on cascaded network can improve the measurement frequency while maintaining high precision.The research results provide a key basis for improving the immersion of maglev visuo-haptic interaction simulation.In conclusion,the key problems in the maglev visuo-haptic interaction system for virtual surgery are studied in depth,and some research results are obtained.These research results provide necessary theoretical and methodological support for the establishment of the immersive maglev visuo-haptic interaction system,and have potential application value in personalized virtual surgery interactive simulation.