3D Bilinear Face Model Fitting from Multiple Cameras

3D facial analysis attracts much interest recently due to the fact that it provides solutions for mitigating confounding factors in 2D image analysis, such as pose, illumination. On the other hand, it also provides enriched representation with more discriminative depth information for applications such as expression or identity analysis. In this paper, we investigate 3D face reconstruction based on sets of extracted facial features in a multi-view camera setup. The reconstruction is done using Bilinear Face Models where identity and expression are modelled independently in different modes. A novel algorithm based on full-perspective projection model is introduced. We validate our reconstruction method on synthetic data in this study. Experiments show that the reconstruction performance is significantly improved with multi-view inputs in terms of point-to-point error, normal error, as well as errors in model coefficients. We also show that the reconstruction method based on full-perspective projection model produces superior reconstruction accuracy comparing to weak-perspective model in multi-view reconstruction. The robustness of reconstruction against noise in the feature data is discussed and show the proposed method is applicable on real data.