The Key Algorithms For Manual Real-time Stitching Based On Microscope Video Stream

The microscope is a common optical instrument.It has a wide range of applications in remote sensing image processing,medical image analysis,computer vision,computer graphics,and virtual reality technology.Conventional microscopes have contradictions in the field of view range and resolution,and the image stitching technology can stitch the images of different fields into an image with large field of view,which increases the field of view and improves the resolution of the observed image.This thesis studies the key algorithms in the manual real-time stitching system for video streaming under the microscope.By manually moving the slices on the stage,the integrated stitching system can perform real-time stitching and display the image in the current field of view.The key issues of the stitching system include image filtering,image registration,image fusion,brightness correction,and image preprocessing.The thesis will focus on these key issues for research.This thesis designs a framework suitable for image stitching of video stream.The system can perform stitching operations in real time and greatly improve the stitching accuracy.In addition,a registration feedback mechanism is provided to provide a feedback signal that adjusts the scanning speed,making the manual stitching process smoother.In the image registration part,the paper proposes a three-step registration method,coarse registration,precise registration,and re-registration,to achieve the transformation matrix between two images within 30ms,which meets the requirement of real-time stitching.An improved ORB feature point registration algorithm is proposed,which greatly improves the accuracy of the registration algorithm.The re-registration part achieves rapid registration by subtle selection of background frames and the use of feature points to construct sparse optical flow,eliminating cumulative offset errors.In the image fusion part,the paper proposes a fusion algorithm combining local optimal seam line and multi-band,which can effectively eliminate the problems of image ghosting and brightness difference,and obtain high-quality fused images.An improved dynamic programming algorithm is proposed for searching optimal seam line.The algorithm expands the search direction to realize the search of an optimal seam line between the specified starting point and the end point,accelerating the search process.What’s more,the algorithm can be extended to more general cases such as irregular overlapping regions by segmenting the seam line.In the pre-processing part of stitching,the paper studies image pre-processing,such as image white balance and brightness equalization methods,and sharpness evaluation mechanisms.Finally,the paper integrates the algorithm into the software system of microscope camera as a stitching module.Experiments show that the system can realize real-time stitching and preview of microscopic images without the electric scanning platform,and generate super-large stitching images.