Author's Personal Copy Multi-kernel Appearance Model ☆

In most cases authors are permitted to post their version of the article (e.g. in Word or Tex form) to their personal website or institutional repository. Authors requiring further information regarding Elsevier's archiving and manuscript policies are encouraged to visit: a b s t r a c t a r t i c l e i n f o Keywords: Facial feature localization Multiple-kernel learning Two-stage classifiers SIFT descriptor Deformable model alignment Gauss–Newton optimization Automatic facial landmarking is a crucial prerequisite of many applications dedicated to face analysis. In this paper we describe a two-step method. In a first step, each landmark position in the image is predicted independently. To achieve fast and accurate localizations, we implement detectors based on a two-stage classifier and we use multiple kernel learning algorithms to combine multi-scale features. In a second step, to increase the robustness of the system, we introduce spatial constraints between landmarks. To this end, parameters of a deformable shape model are optimized using the first step outputs through a Gauss–Newton algorithm. Extensive experiments have been carried out on different databases (PIE, LFPW, Cohn-Kanade, Face Pix and BioID), assessing the accuracy and the robustness of the proposed approach. They show that the proposed algorithm is not significantly affected by small rotations, facial expressions or natural occlusions and can be favorably compared with the current state of the art landmarking systems. Automatic salient point localization is a fundamental building block in many computer vision-based applications. In facial analysis, these salient points (e.g., eye and mouth corners, nose and chin tips) are called fiducial points or landmarks. Their appearance and spatial arrangement are important for most facial analysis tasks, including face recognition, expression recognition or face animation. In human spontaneous behavior analysis, facial landmark configurations are indicative of deformations caused by expressions. Whatever the application, automatic facial land-marking is the second step (after face detection) of a larger face processing task. Many methods have been introduced over the last decade to solve this problem. Locating facial landmarks remains challenging for applications that must operate under a wide range of conditions, including illumination variations, occlusions, poses or expressions. In general, related methods use two types of information, both essential. One is facial landmark appearance, also called texture information. The other is the spatial relationship among different facial landmarks, also called shape information. By looking at the latter and the degree of spatial constraint it …