LiDAR data manipulation: Object detection, segmentation, and inpainting

Light Detection and Ranging (LiDAR) is a data acquisition method that uses time-of-flight calculations on laser pulses to obtain 3D point samples of an environment. These data sets contain information about a scene that is impossible or very difficult to accurately obtain with standard optical imaging, such as the absolute scale of objects, the position of heavily occluded objects, and surface texture information. In this thesis we investigate several interesting uses of such data sets.;First, we propose a strategy for confirming the proposed positions of objects in a scene, independent of a specific detection or registration method. A dual metric, consistency and confidence, is computed to produce a human-interpretable metric of the certainty that an object has been correctly identified. We show that the proposed method works well for scenes containing partially visible or heavily occluded objects.;Next, we propose a technique to interactively segment objects from the background in LiDAR scans. First, we create a conservative segmentation of the object based on user-drawn lines and LiDAR depths, where they are reliable. Second, this segmentation is refined using a traditional graph-cut method based on colors and depths. The resulting segmentations are sharp and cannot be easily achieved with either color-only or depth-only algorithms.;Third, we study how to fill large holes in LiDAR data, typically due to occlusions of the scene background by foreground objects and by removing objects from the scene. Our approach generalizes exemplar-based image inpainting and uses both the color and depth information present in a LiDAR scan. A novel gradient-domain algorithm forms the core of the approach. We show that by combining an inpainting algorithm with the proposed algorithm we can quickly and convincingly remove unwanted objects from LiDAR scans and render the scene from different perspectives.;Finally, we address several important problems in patch-based image inpainting. We describe techniques to verify the correctness of selected source patches, explain a procedure to prevent unwanted repeated patch artifacts, and demonstrate a user interface to allow interactive refinement during the inpainting procedure. We show that these modifications can result in significantly better in painted images than traditional greedy algorithms.