Key Techniques Of Multi-spectral Dynamic Optical Imaging System

Multispectral imaging technology is an emerging technology based on imaging and spectroscopy.It combines the advantages of spatial imaging and spectral imaging which can simultaneously acquire and fuse spatial as well as spectral information of targets,thereby integrating the effective information in a single image.As for clinical research,the traditional method of tumor surgery is the most direct and effective way to remove tumors.However,since the judgment of tumor tissue depends on the doctor's experience,it is easy for tumor to reborn after surgery or normal tissue could be removed in the surgery.Multi-spectral optical imaging system based on multi-spectral imaging technology can image tiny tumor lesions in biological tissues,reflect the true boundary of tumor tissue and other information to improve the accuracy and rate of success of surgeries.In this paper,a multi-spectral dynamic optical imaging system is designed and built to capture color images and near-infrared images of tumors based on fluorescent tracers of tumors and other targets.The fusion images of tumor markers are displayed in real time after fusion by multispectral image fusion algorithm for subsequent analysis and processing.The main work of this paper is as follows:(1)A multi-spectral image fusion algorithm based on undecimated wavelet transform and frequency-domain U-Net is proposed for fusing the color image and near-infrared image of the tumor labeled with indocyanine green(ICG).The algorithm obtains the frequency domain sub-bands of the green channel and fluorescence image by undecimated wavelet transform,and the pre-trained frequency-domain U-Net is used to segment the low-frequency sub-band of the fluorescence image for regions of tumor to remove noises.Then based on the region of interest,the high-low frequency sub-band of the source image is fused by the fusion rule of high-frequency taking large absolute value and low-frequency weighted addition.At last the inverse undecimated wavelet transform is used to obtain the fused image to improve the quality of the spectrally fused image.(2)For the image captured by the system and processed by JPEG algorithm for high compression ratio,there are severe blocking and edge effects.This paper proposes a dual-residual-based convolutional neural network to process compressed JPEG images.The network takes the decompressed image with artifacts such as blocking and ringing effect as input,and outputs an image with more details such as boundary and texture to improve the image quality after decompression.(3)Designed and built a multi-spectral dynamic optical imaging hardware system.After the tumor tissue is marked with green,the fluorescence is excited by the near-infrared light source and two independent charge-coupled devices(CCDs)are used to capture the color image of the target and the near-infrared fluorescence image simultaneously.Due to the different positions of the CCD,it is necessary to design the optical path to convert the field of view of two CCDs to the same.Then the captured image is transmitted to the computer for processing and displaying.(4)Designed and implemented a multi-spectral dynamic optical imaging software system.The parameters of the CCD cameras such as exposure time,gain,and image format are controlled by C++.The difference between the image frame rates of the two CCD cameras causes the captured images to be asynchronous so a buffer is used for synchronization,then the images are fused and displayed.Besides,functions like saving images,saving videos,dynamic curves,2D analysis,and more are provided.(5)The experiments of pork,hearts of pig as well as mice are designed to verify the fusion performance of multi-spectral image fusion algorithm and the JPEG image processing algorithm by using various image evaluation indexes.Also the experiments proved the effectiveness and stability of the dual-channel high-definition dynamic optical imaging system.