Estimating Head Orientation with Stereo Vision

Interpretation of human behaviors in video data is essential for natural and intuitive human-computer interfaces. In this context, the estimation of a person’s head pose plays a major role, since heads and faces are continuously used in interaction between people. In this work we present a method for estimating a person’s head pose with a stereo camera. Our approach focuses on the application of humanrobot interaction, where people may be further away from the camera and may move freely around in a room. First, the 3D scene is reconstructed from the images of a stereo camera by calculating depth information. Subsequently, the face is extracted with a color-based face tracking approach. Finally, the resulting 3D face model is preprocessed by a number of normalization algorithms. The estimation is based on neural networks, which are trained to compute the head pose from gray scale and depth information. We show that depth information not only helps improving the accuracy of the pose estimation, but also improves the robustness of the system when the lighting conditions change. The system can handle pan and tilt rotations from −90◦ to +90◦ and achieves high accuracy in a realistic environment. It doesn’t require any manual initialization and doesn’t suffer from drift during an image sequence. Moreover, the system is capable of real-time processing. Acknowledgements This work was conducted at the Interactive Systems Labs as part of my studies at the Universität Karlsruhe (TH). I would like to thank all members of the laboratory for participating in the various data collections performed during this work. I am particularly grateful for the help of Kai Nickel who was always there for advice regarding the stereo camera and implementation details. Furthermore I thank my advisor Rainer Stiefelhagen for his constant support.