Multispectral Image Super-Resolution Algorithms Using Image Fusion

Multispectral imaging records spectral information of scenes,and is of wide application in biomedicine,remote sensing,color reproduction,and etc.Meanwhile,multispectral imaging has also been a promising tool for many emerging computer vision tasks,including the tracking,face recognition and segmentation.Due to hardware restrictions,the multispectral image can not be acquired with both high efficiency and high quality.To address the issue,this thesis carried out the following innovative reasearches:1.A fast multispectral imaging framework based on the image sensor pixel-binning and spec-tral unmixing technique is proposed to improve the imaging efficiency.The framework comprises a fast imaging stage and a computational reconstruction stage.In the imaging stage,only a few spectral images are acquired in high resolution,while most spectral images are acquired in low resolution.The low resolution images are captured by applying pixel binning on the image sensor such that the exposure time can be greatly reduced.In the reconstruction stage,an optimal number of basis spectra are computed and the signal-dependent noise statistics are estimated.Then the un-known high resolution images are efficiently reconstructed by solving a closed-form cost function that models the spatial and spectral degradations.The effectiveness of the proposed framework is evaluated using real-scene multispectral images.Experimental results validate that our method can well reduce the imaging time,and outperforms the state-of-the-arts in terms of reconstruction accuracy,with additional 20 x or more improvement in computational efficiency.2.A multispectral image super-resolution algorithm is proposed to improve the spatial reso-lution of multispectral image by fusing with a high resolution RGB image.It deals with the general circumstance that image intensity is linear to scene radiance for multispectral imaging devices while is nonlinear and unknown for most RGB cameras.Our algorithm first solves the inverse camera response function and spectral sensitivity function of RGB camera,and establishes the linear rela-tionship between multispectral and RGB images.Then the unknown high resolution multispectral image is efficiently reconstructed according to the linear image degradation models.Meanwhile,the edge structure of the reconstructed high resolution multispectral image is kept in accordance with that of the RGB image using a weighted total variation regularizer.The effectiveness of our algorithm is evaluated on both public datasets and our image set.Experimental results validate that our algorithm can well improve the spatial resolution,and outperforms the state-of-the-arts in terms of both reconstruction accuracy and computational efficiency.3.A practical multispectral image super-resolution algorithm is proposed based on proba-bilistic superpixel matching and spectral reconstruction.The required low resolution multispectral image and high resolution RGB image have no need for alignment.Our algorithm first defines the superpixel correspondence between images according to the dense approximate nearest neigh-bor field,and extracts superpixel level training samples along with probabilistic confidence.Then a Bayesian weighted regression model with heteroscedastic noise is built to learn the mapping from RGB space to multispectral space.This probabilistic regression is treated as an Expectation-Maximization learning problem and has no open parameters needing to be tuned.The unknown high resolution multispectral image is finally reconstructed by applying the mapping function to the RGB image with kernel trick.Experiments on both public datasets and our image set demonstrate the validation of our algorithm in a more practical scenarios,and the superiority of our algorithm over current state-of-the-arts.