Visual Feedback And Localization Of Micro-Manipulation System Oriented Complex Task

Micro-manipulation system, which relieves operators from heavy and complicated manual operation, is the extension of robot technology to micro-manipulation field. With the development of micro-manipulation technology and research, multi-target operation and batch-target operation become the focus in micro-manipulation. In order to solve the problems in batch-target operation, this paper introduces the concept of complex task into micro-manipulation field, and does in-depth studies in visual feedback and location in complex task. The research contents are as follows.The narrow field of view (FOV) of microscope causes various difficulties in complex task in micro-manipulation. In other word, it is the major factor which limits huge-range micro-manipulation. A method to extend microscopic FOV based on image stitching is put forward in this paper. This method combining stitching methods based on image region and image features, can accomplish extending microscopic FOV online with high stitching precision. At the same time, hierarchical structure is utilized to establish the global FOV image with centimeter range and sub-micron resolution. The method provides wide visual feedback for complex task in micro-manipulation.Based on the analysis of microscopic imaging features and microscopic imaging model, a semi-blind image restoration approach is presented to restore blurred microscopic images in this paper. The point spread function of the imaging system is described as Gaussian function, and Canny edge detection is introduced into image restoration field. Results prove that this approach is effective and provides clear visual feedback for complex task in micro-manipulation.Based on the target location methods offered, a method on batch-target hierarchical global location is addressed in this paper. This method divides microscopic global FOV image into a series of sub-images based on hierarchical strategy, and shortens the whole time for target location. Moreover, hierarchical information can also be utilized to simplify the sort problem of the targets, which is one of NP-complete problems. This method completes the preparation for complex task in micro-manipulation. On the other hand, In order to solve re-location problem for batch biologic targets, a relocation method is put forward. Using configuration of a group of adjacent targets, this method can find the same target in each observation, and lays the foundation of quantitative statistics in micro-manipulation of batch biologic targets.In the direction of optical axis, microscope suffers from limited depth-of-field: clear image can only be obtained in small range around focusing plane, while in defocused plane, just blurred microscopic images will be obtained. Micro-manipulation can not be done manually with defocused visual feedback. However, the defocused microscopic image is a kind of regular blurring. If this rule is understood, the location problem of micro-manipulation tools can be solved, which makes defocused micro-manipulation possible. Depending on the analysis of microscopic imaging principle, a location approach in depth direction based on defocused optical transfer function is put forward in this paper. This approach has high location precision and good linearity, since lens parameters and diffraction effect have been taken account. This approach achieves on-line location of micro-manipulation tools, and can be applied in complex task in micro-manipulation with high precision.Finally, as for the modern problems in gene engineering, experiments, including plane FOV extension and location of batch biologic targets, and location in depth direction based on defocused microscopic images, are designed and accomplished in this paper. The experiment results demonstrate that all the methods in this paper are effective.