Automatic extraction of facial interest points based on 2D and 3D data

Facial feature points are one of the most important clues for many computer vision applications such as face normalization, registration and model-based human face coding. Hence, automating the extraction of these points would have a wide range of usage. In this paper, we aim to detect a subset of Facial Definition Parameters (FDPs) defined in MPEG-4 automatically by utilizing both 2D and 3D face data. The main assumption in this work is that the 2D images and the corresponding 3D scans are taken for frontal faces with neutral expressions. This limitation is realistic with respect to our scenario, in which the enrollment is done in a controlled environment and the detected FDP points are to be used for the warping and animation of the enrolled faces [1] where the choice of MPEG-4 FDP is justified. For the extraction of the points, 2D, 3D data or both is used according to the distinctive information they carry in that particular facial region. As a result, total number of 29 interest points is detected. The method is tested on the neutral set of Bosphorus database that includes 105 subjects with registered 3D scans and color images.