Research On Depth Map Super-resolution Reconstruction

Super-resolution is to increase the resolution of the original image by hardwareor software. The process to obtain a high-resolution image by a series oflow-resolution images is named super-resolution reconstruction. The core idea ofsuper-resolution reconstruction is exchanging spatial resolution with temporalbandpass which is an access to multi-frame image sequence of a scene, realizing theconversion from temporal resolution to spatial resolution. With the improvement ofimaging technology, the recent launch of depth camera makes a breakthrough on thelimitation of depth imaging in the traditional laser scanning and stereo matching,which can obtain a real-time dynamic three-dimensional scene more easily. However,with the limited resolution, its application in computer vision becomes very narrow.Therefore, it is necessary to find an approach to enhancing the current resolution ofdepth map.To increase the density of acquisition device sensors at present is the moststraightforward solution to improve the resolution. Nevertheless, the image sensor ofhigh density is quite expensive, which is generally unbearable. On the other hand,imaging system has been close to the limitation with the constraint of the density ofits inherent sensor. Another method of increasing the resolution is to boost the size ofchips, leading to an increase of capacitance and a decrease of the speed of chargetransfer, which is not generally considered to be effective. To obtain high-resolutionfrom low-resolution by a signal processing method is a promising approach.Therefore, the thesis presents a novel method of super-resolution depth map. Themethod based on Eulerian video magnification uses the Laplacian pyramid to do thespatial filtering, while using two first-order lowpass IIR filters with the differentcutoff frequencies to construct an IIR bandpass filter for temporal filtering of eachspatial band, then applies amplification factor to amplify or weaken the bandpasssignal and add it back to original signal, generating the final amplified signal, namelysuper-resolution signal. But in the amplification process of the signal, the noiseexisted is also amplified. Thus it is necessary for denoising and deblurring of the super-resolution signal. Block-matching and3D filtering algorithm is a recentdenoising method. The algorithm has two main steps, namely basic estimate and finalestimate. These two steps are very similar, firstly defining a2D Kaiser window,grouping similar2D image patches into3D groups, and finally applying filtering andaggregating to the3D groups. However, there are two different points in the two steps.One is that final estimate deals with filtered blocks generated from basic estimate.The other is wiener filtering for final estimate, but hard threshold filtering for basicestimate. Finally the depth map is enhanced in detail and deblurred by guidancefiltering algorithm. The experiments show that the novel depth map super-resolutionalgorithm can get better results whether visual effects or objective evaluationmeasurements.