Microvision-based Sensing For Robotic Micromanipulation

As one of the most advanced equipment in the high-tech industry,the robotic micromanipulation system has the characteristics of knowledge-intensiveness,high added value,and high social benefits.The robotic micromanipulation system also involves multidisciplinary that has the potential to lead the breakthrough for both advanced equipment manufacturing and cuttingedge medical service.With the properties of non-contact,high flexibility,abundant information,and intuitive visualization,the microvision-based motion measurement system plays a significant role in the process of automating robotic micromanipulation.However,the trend of more complex,precise,and efficient micromanipulation also push forward higher performance requirements for the vision-based systems.Currently,the microvision-based systems still have many shortcomings for micromanipulation,such as low absolute measurement accuracy,small microscope field of view,and small depth of field with the optical microscope platform.Thus,this dissertation focuses on precise vision-based sensing to the micromanipulation positioning stages,micromanipulators,and manipulated objects,which all are the key components in the micromanipulation systems.Based on machine vision technology,the dissertation aims to study high-precision,large-scale,high-dimensional,and efficient vision-based sensing theories and methods.The main research contents of this dissertation are as follows:(1)Due to the inaccurate modeling,the absolute measurement accuracy in the field of view of the vision-based system by the analytical calibration is still low for micromanipulation.To address this issue,a machine learning calibration method is proposed to improve the accuracy of the imaging model.Firstly,the geometric model of the vision-based system for macro-micro manipulation is introduced,and the deficiencies of the analytical calibration method are analyzed;secondly,the strategy of integrating the analytical method and the machine learning method is proposed,where an artificial neural network model and Gaussian process model is designed separately;Thirdly,by simulations and experiments,the validity of the proposed calibration method is verified.Simulation and experimental results show that the proposed calibration method can significantly improve the accuracy of the vision-based system model,reduce residual errors,and improve measurement stability.(2)To solve the issue that the field of view in the microvision-based motion measurement system is too small to meet the requirements of micromanipulation,a multi-scale motion measurement method for positioning stages is proposed.By constructing a global map,the effective measurement range of the microvision-based motion measurement system is expanded.While improving the measurement range,an automatic micro-motion tracking method that combines feature matching and phase correlation is designed,in order to maintain high-precision measurement capabilities.Therefore,measurement of both the macro-motion and micro-motion of the micromanipulation positioning stage can be simultaneously achieved.By simulation and online experiments,the performances of the proposed method are fully testified,and the effectiveness of the method for measuring the multi-scale of motions is verified.(3)The motion detection method of the micromanipulator in the robotic micromanipulation system is studied.Based on feature extraction,an optical flow estimation method is proposed to achieve the motion detection of micromanipulators in the entire field of view.Meanwhile,the practicability of the proposed method is enhanced by designing the feature image pyramid and the motion measurement process of the target of interest.A simulation study with simulated displacement and Gaussian noise interference is presented to verify the robustness of the method.Then,through a series of experimental studies,the accuracy and stability of the method are tested,and finally the method is used to detect the motion of the compliant microgripper.(4)In order to meet the requirements for three-dimensional(3D)visual sensing of manipulated objects in the complex and precise micromanipulations,and 3D imaging of cells based on conventional brightfield optical microvision-based systems is proposed,since the cell is a common manipulated object in micromanipulation.Based on the optical sectioning technique,the sensing capability of the optical microvision-based system in the depth direction is expanded through image stacking.An automatic process for cell manipulation is designed to realize the3 D imaging of manipulated objects.Both the cell and the intracellular material are identified and localized in 3D.In the vision-based robotic micromanipulation platform,experiments are conducted on early-stage embryos,and the accuracy and practicability of the method are validated.Thus,The proposed method enables low-cost 3D imaging based on the conventional optical microvision-based system for cell micromanipulation.The summary and outlook of this dissertation are presented in the last chapter.This dissertation provides practical ideas and methods for the precise vision-based sensing of each key component of the robotic micromanipulation systems,thereby contributing to the development of the automation of the micromanipulation system.