Point Cloud Based3D Reconstruction And Morphological Event Analysis

Recent advances in3D scanning technologies open a new era with scanners mounted overairborne or street level vehicles for fast scanning of large scenes, as well as those emerging cheapdevices that are affordable by large population. The increasing capability and ubiquity of suchdevicesenableeasieracquisitionoflargeramountof3Dmeasurementdataofrealworld,whichwecallPointCloud. Togetherwiththesignificantprogressesmadeinmulti-viewstereothatcomputespoint cloud from images, point cloud has been an important data source for digitizing the realworld. However, unlike those widely used in reverse engineering, the point cloud data acquiredby these devices or methods under much less constraint environments often incurs imperfections:large missing regions, significant variation in sampling density, as well as noises and outliers.We propose algorithms on denoising, consolidation and primitive fitting of such data for3Dreconstruction, and analyzing the morphological events in the time-lapse point cloud that recordsnot only the static frames of the scanned objects, but also the dynamics of them, which we call4DPoint Cloud. The proposed algorithms all work on point cloud and share the idea of incorporatinghigh level shape priors for regularization. The output of our algorithms can possibly be usedfor reconstruction and procedural synthesis, shape understanding and sparse encoding, as well asquantitative study of objects with changing geometry.In the first part of the thesis, we propose algorithms for consolidation and3D reconstructionfromimperfectpointclouddata. Morespecifically,weworkonpointcloudconsolidationofurbanbuildings, and3D reconstruction of the more generalized category, man-made shapes.Buildings often exhibit large scale repetitions and self-similarities. Detecting, extracting,and utilizing such large scale repetitions provide powerful means to consolidate the imperfectdata. We present a novel algorithm for non-local consolidation of urban building point cloud byclustering corresponding surfaces to fuse the multiplicity of the geometry to a base-geometry, andapply off-plane and in-plane denoising there. We observed that3D point cloud and2D imageshave complementary characteristics and present a novel approach to fuse the two complementarydata modals and transfer repetition information detected in2D images for consolidation of pointcloud, which can even recover structures completely missing in point cloud as long as seen inimages. Our urban building point cloud consolidation is an important pre-processing step forreconstruction of urban models, which is gaining increasing attention these days, motivated byambitious applications that aim to build digital copies of real cities. In man-made objects, the more generalized category than urban buildings, the repetitions areless commonly seen, instead, they are characterized with more general global relations such asprecise alignment and precise equality of attributes between the simple primitives that the man-madeobjectsarecomposedof. Wepresentanovelalgorithmthatcouplesthelocalprimitivefittingwiththeglobalrelations. StartingwithasetofinitialRANSACbasedlocallyfittedprimitives, ouralgorithm progressively learns and infers orientation, placement, and equality relations. In eachstage, a set of feasible relations are extracted among the candidate relations, and then conformedto while best fitting to the input data. The approach not only recovers the geometry of man-madeobjects from point cloud, but also extracts the global relations that are likely to significantly assistin shape understanding and sparse encoding.In the second part, we study temporal morphological events in4D point cloud data that arenot in-compressibleandthusinvalidatesthein-compressibleassumptionthatiswidelyused. Morespecifically, we introduce a framework to study plant growth, particularly focusing on accuratelylocating and tracking morphological events like budding and bifurcation. Studying growth anddevelopment of plants is of central importance in botany. Current methods for quantitative anal-ysis of such growth processes are either limited to tedious and sparse manual measurements, orcoarse2D measurements. Automating this measurement process by3D scanning will give bi-ologists access to data in volumes and accuracy currently inconceivable. However, during theirdevelopment, plants grow new parts and bifurcate to different components—violating the centralin-compressibility assumption made by state-of-the-art3D acquisition algorithms and renderingthem unsuited to analyzing growth. We address the problem by a novel forward-backward analy-sis approach, wherein we track robustly detected plant components back in time to ensure correctspatial-temporal detection of morphological events.To the best of our knowledge, we present the first attempt to focus on morphological eventdetection in4D point clouds, both in time and space, in the context of growing (and decaying)plants. The framework we proposed has a unique feature of tracking branching and budding ofnew organs, even when there are organ touching or overlapping situations, enables accurate andautomatedmeasurementofplantgrowth,whichcanbecomeakeytoolforthebiologiststoperformanalysis at a scale currently simply impossible using manual means, and is considered by worldleading biology experts to have great potential in their field.