Research On The Depth Image Acquisition And Restoration

Depth data have enormous applications,which include manufacturing,medical care,education industry,entertainment,etc..We need methods that could acquire high resolution and high accuracy depth m aps in various scenarios.However,the depth data which are directly attained by present depth sensing devices have limited quality.We also have to research on how to restore high resolution and high accuracy depth map from the low quality depth data.This dissertation firstly reviews the related depth acquisition methods.Secondly,one active depth sensing approach: structured light is introduced.Thirdly,some depth restoration methods are summ arized.Then three propos ed depth sensing m ethods and one depth completion technique are described.The summaries of the works are as follows:1.A coprime theorem based temporal structured light method is proposed.This method adds a square wave in each pattern of the traditional three-step phase shifting approach.The wrapped phase of the fringe pattern is detected by the phase shifting technique.And the phase unwrapping is conducted with the help of the wrapped phase of the encoded square wave.Because the periods of the sinusoidal wave and the square wave are coprime,coprime theorem is utilized to attain reliable absolute phase.Results of experiments demonstrate that the proposed temporal structured light method could acquire more accurate depth map than the current off-the-shelf depth sensing devices.2.A frequency-division multiplexing based spatial structured light is proposed.This method tries to design one single spatial en coding structured light method to combine the advantages of the temporal and spatial encoding structured light approaches.In this proposed method,two fringe patterns are modulated into one combined fringe pattern.Therefore,the phase information of the two fringe patterns can be transmitted simultaneously.When the camera shots the im age,two deformed fringe images are required to be demodulated from the image.The Gabor filter is employed to extract the wrapped phase of each fringe pattern.Then the absolute phase is computed by the coprime theorem.Results of experiments show that this proposed method could acquire high resolution and high accuracy depth image with less time consumption.3.A De Bruijn sequence based spatial structured light is proposed.When there are multiple objects in the scene,the depth map of the scene always conta ins discontinuities and sharp changes.The present phase unwrapping methods tend to fail to calculate reliable unwrapped phase in this situation.In order to tackle this problem,this approach modulates the fringe pattern and the period-code pattern into one-shot structured light pattern.The continuous region detection is accomplished from the wrapped phase of the defor med fringe pattern.Then,the period of the encoded fringe pattern is determ ined by a statistic and the De Bruijn sequence in each continu ous region.The proposed three criteria guarantee the reliability of the decoded periods further.Results of various experiments show that this proposed method could acquire reliable depth map of complex scenes and this approach is robust to the color,texture and albedo of objects.4.A joint temporal-spatial modeling depth map completion from very sparse LIDAR measurements is proposed.This approach leverages both the tem poral information and the availability of corresponding color im ages by presenting the joint temporal-spatial modeling solution.The scene m odel is adopted to describe the three dimensional structure of the scene,and the auto-regressive model is used to present the local and non-local information between the depth map and the corresponding color image.The proposed method enforces both the structural constraint in 3D and the color-depth consistency under one unified fram ework.The resultant optimization is solved as a discrete optimization problem.Quantitative and qualitative evaluations on existing benchmarking datasets demonstrate that our proposed method outperforms existing state-of-the-art methods with a margin.In the end,the dissertation summarizes all the works and discusses the drawbacks of the works.