Model-Based 3D Face Capture with Shape-from-Silhouettes

We present a method for 3D face acquisition using a set or sequence of 2D binary silhouettes. Since silhouette images depend only on the shape and pose of an object, they are immune to lighting and/or texture variations (unlike feature or texturebased shape-from-correspondence). Our prior 3D face model is a linear combination of ”eigenheads” obtained by applying PCA to a training set of laser-scanned 3D faces. These shape coefficients are the parameters for a near-automatic system for capturing the 3D shape as well as the 2D texturemap of a novel input face. Specifically, we use back-projection and a boundary-weighted XOR-based cost function for binary silhouette matching, coupled with a probabilistic ”downhill-simplex” optimization for shape estimation and refinement. Experiments with a multi-camera rig as well as monocular video sequences demonstrate the advantages of our 3D modeling framework and ultimately, its utility for robust face recognition with built-in invariance to pose and illumination. IEEE International Workshop on Analysis and Modeling of Faces & Gestures (AMFG’03) In conjunction with ICCV’03, Nice, France, October 2003. This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Copyright c ©Mitsubishi Electric Research Laboratories, Inc., 2003 201 Broadway, Cambridge, Massachusetts 02139 ∗ MERL Research Laboratory † The Ohio State University Submitted to: Analysis and Modeling of Faces & Gestures (AMFG’03), October 2003. Model-Based 3D Face Capture with Shape-from-Silhouettes Baback Moghaddam Jinho Lee Hanspeter Pfister Raghu Machiraju Mitsubishi Electric Research Laboratory The Ohio State University 201 Broadway, Cambridge MA 02139 USA 2015 Neil Avenue, Columbus OH 43210 USA baback,pfister @merl.com leeji,raghu @cis.ohio-state.edu