Compressvie Sensing Based Dual Camera Spectral Imaging System

With the rapid development of science and technology,the imaging methodology is increasingly playing the crucial role in the process of controlling the known world and exploiting the unknown world.Both the human eye and the computer sensors are sensitive to the light information.In turn,the resolve of the light information drives the development trends of the vision sensors.Generally,the light information contains seven dimensions,which is called Plenoptic Function.However,due to the limitation of imaging devices,conventional cameras can only resolve two spatial and one temporal dimensions.Up to this point,exploiting the other dimensions of light has become a popular point in the field of optics,signal processing and computer vision.Wherein,hyperspectral imaging is one of the representative directions.Hyperspectral imaging can reveal the essential nature of light,which is regarded as the”gene” of light.Since Hyperspectral imaging has the ability of detailed description about the target scene,it shows very wide application prospect in various fields,such as meteorological observation,land allocation,medical diagnosis,vegetation classification,military reconnaissance,resource exploration,seismic surveillance,and disaster preparedness,etc.However,due to the limitation of current device,material and craftsmanship,traditional hyperspectral imaging still suffers from many practical problems.They mainly lie in:1.The spatial resolution depends on the pixel density of the detector.It necessitates higher pixel density to improve the spatial resolution.However in some special scenarios,such as remote sensing and medical diagnosis,the detector with high pixel density is very expensive or directly unavailable.2.High spectral resolution is usually contradictory with high spatial resolution.In traditional hyperspectral imaging systems,the slit's width directly affects the spatial resolution.Although decreasing the slit's width can lead to improvement of the spatial resolution,the light throughput would be severely limited,which results in low signal-to-noise ratio.Meanwhile,to increase the spectral resolution,it needs to decrease the bandwidth of each spectral band.However,with certain light throughput,the radiance energy of each band would be diminished,which cannot guarantee the effective information acquisition.3.Traditional hyperspectral imaging usually employs the time-division multiple-scanning strategy.The goal is to construct the direct mapping relationship between the three dimensional spectral information and the two dimensional measurement.The core idea is to sacrifice temporal resolution and spatial resolution for the improvement in the spectral dimension,so it is not suitable for dynamic scenes theoretically.4.The data throughput of the spectral information is rather huge,while the efficiency of the direct measurement strategy is rather low,which ignores the intrinsic structure characteristic of the spectral information.Besides,there are usually precision optical machinery within the spectral imager.This kind of special machinery cannot be easily lightened in weight and miniature in size,and further is hard to remain stable due to the moving components,which would inevitably results in low imaging accuracy.This thesis,based on the compressive sensing theory,focuses on the problems of the traditional hyperspectral methodology.Through exploiting the fundamental knowledge in computational imaging,this thesis develops a series of research about the computational hyperspectral imaging methodology,which is characterized by high spatial resolution,high temporal resolution,and high spectral resolution.This thesis proposes a theoretically complete imaging methodology and builds a practical hardware system.With the interaction of imaging theory and practical system,the goal of this thesis is to introduce a new spectral imaging strategy and remedy the drawbacks of traditional methods.The main contributions of this thesis are as follows:1.Aiming at the contradiction between high spatial resolution and high spectral in traditional hyperspectral imaging systems,this thesis,based on the compressive sensing theory,proposes a dual-camera compressive hyperspectral imaging(DCCHI)system.DCCHI is compose of two branches: compressive hyperspectral branch and panchromatic branch,which are integrated by a beam splitter.In compressive hyperspectral branch,the scene information are captured in three procedures: spatial modulation,spectral dispersion and spectral integration,which,in together,is equivalent to project the three dimensional spectral information onto the two dimensional detector.Panchromatic branch,which contains a standard panchromatic camera,directly project the three dimensional spectral information onto the two dimensional detector without any additional procedure.With distinct sampling strategies in two branches,DCCHI is characterized by diverse and complementary measurement,which is in accordance with the basic requirement of compressive sensing theory.Through exploiting the prior knowledge of scene information,in addition to the measurement from two branches,DCCHI can reconstruct the underlying three dimensional spectral information.The proposed system successfully solve the contradiction between high spatial resolution and high spectral in traditional hyperspectral imaging systems.2.Aiming at the problem of capturing hyperspectral video,this thesis proposes a dualcamera system for high speed(temporal resolution)hyperspectral video acquisition.By exploiting the characteristic of two branches,this thesis proposes a temporal acceleration framework,which achieve the state-of-the-art hyperspectral video frame rate: 100 FPS.Meanwhile,by exploiting the structure similarity between panchromatic and hyperspectral image,this thesis proposes to train an adaptive dictionary from the panchromatic image and a reconstruction algorithm based on the adaptive dictionary.The proposed system and algorithm are verified through theoretical simulation and hardware system,both of which significantly boost the frame rate and reconstruction quality of hyperspectral video.3.Aiming at the low reconstruction quality in computational hyperspectral imaging,this thesis proposes a efficient reconstruction algorithm based on adaptive three dimensional nonlocal sparse representation.Firstly,this thesis proposes a theoretical framework which incorporates the three dimensional sparse representation into the hyperspectral reconstruction,and propose a solution for best parameter selection.Then,by exploiting the relation between panchromatic and hyperspectral image,this thesis propose a joint metric for non-local similarity.By further analyzing the reconstruction error,this thesis proposes an adaptive solution for estimating the non-local similarity.Finally,the proposed algorithm is validated through theoretical simulation and hardware experiment,both of which demonstrate the superiority of the proposed algorithm.4.By exploiting the heterogeneous property between compressive hyperspectral branch and the panchromatic branch,this thesis proposes a cross-modal system which can simultaneously capture depth information and spectral information of the scene.By exploiting the correlation of depth information and spectral information,this thesis proposes an algorithm based iterative and mutual-beneficial algorithm,which can jointly optimize the reconstruction accuracy of depth information and hyperspectral information.Meanwhile,this thesis successfully builds a hardware prototype,and demonstrates the efficiency of the proposed methodology,which,for the first time,achieve the joint acquisition of depth information and hyperspectral information in a single shot.