Automatic Three-dimensional Modeling from Reality

In this dissertation, we develop techniques to fully automate the process of constructing a digital, three-dimensional (3D) model from a set of 3D views of a static scene obtained from unknown viewpoints. Since 3D sensors only capture the scene structure as seen from a single viewpoint, multiple views must be combined to create a complete model. Current modeling methods are limited in that they require measurement of the sensor viewpoints or manual registration of the views. We have developed an alternative approach that fully automates the modeling process without without any knowledge the sensor viewpoints and without manual intervention. Given a set of 3D views of a scene, the challenge is to align all the views in a common coordinate system without knowing the original sensor viewpoints or even which views overlap one another. This problem, which we call multi-view surface matching, is analogous to assembling a jigsaw puzzle in 3D. The views are the puzzle pieces, and the problem is to correctly assemble the pieces without even knowing what the puzzle is supposed to look like. To solve the multi-view surface matching problem, we begin by using pair-wise surface matching to align pairs of views. The resulting relative pose estimates (matches), some of which may be incorrect, are stored as edges in a graph data structure called the local registration model graph (GLR). Unfortunately, it is generally not possible to distinguish all the correct matches from incorrect ones at the local (pair-wise) level; however, by considering the surface consistency constraints of a sequence of matches, we can identify incorrect, but locally consistent matches. GLR contains a node for each view and an edge for each match from pair-wise surface matching. The sub-graphs of GLR represent the set of discrete model hypotheses – potential solutions to the multi-view surface matching problem. If we can find a model hypothesis that connects all the views and contains only correct matches, we can use the relative poses to place the views in a common coordinate system, thereby correctly solving this multi-view surface matching instance. This discrete search space also contains a continuous aspect, since the view poses we are estimating are continuousvalued. We therefore formulate this search as a mixed discrete/continuous optimization over the discrete hypothesis space and continuous pose parameters. With this formulation, we have reduced the multi-view surface matching problem to two fundamental questions: 1) What constitutes a “good” model hypothesis? and 2) How do we robustly and efficiently search for the best model hypothesis in an exponentially large hypothesis space? We answer the first question with a method for learning a probabilistic model of the local quality of pair-wise matches and the global quality of model hypotheses. The method provides a principled way to combine multiple, disparate measures of registration quality. We answer the second question with two classes of search algorithms. One class searches deterministically for a globally consistent model hypothesis by repeatedly adding edges from GLR to an initially empty hypothesis. The second class uses a stochastic edge swapping approach to search for the best hypothesis according to the given global quality measure. Our multi-view surface matching framework is sensor independent and can be applied to 3D data at any scale. We show results using three different range scanners to automatically model scenes varying in size from from 20 centimeters to 200 meters. We model a variety of scene types, including small, “desktop” objects, building interiors, and terrain. For small objects, we have developed an application called hand-held modeling, in which the user holds an object and scans it from various viewpoints. This is an easy modeling method, requiring no specialized hardware, minimal training, and only a few minutes to model an average object.