Articulated-Pose Estimation Using Brightness and Depth-Constancy Constraints

This paper explores several approaches for articulated-pose estimation, assuming that video-rate depth information is available, from either stereo cameras or other sensors. We use these depth measurements in the traditional linear brightness constraint equation, as well as in a depth constraint equation. To capture the joint constraints, we combine the brightness and depth constraints with twist mathematics. We address several important issues in the formation of the constraint equations, including updating the body rotation matrix without using a first-order matrix approximation and removing the coupling between the rotation and translation updates. The resulting constraint equations are linear on a modified parameter set. After solving these linear constraints, there is a single closed-form non-linear transformation to return the updates to the original pose parameters. We show results for tracking body pose in oblique views of synthetic walking sequences and in moving-camera views of synthetic jumping-jack sequences. We also show results for tracking body pose in side views of a real walking sequence. 1 Introduction In this paper, we extend the head-pose tracking of Har-ville et al. [1] to articulated-pose tracking. We assume that we have video-rate depth images, from either stereo cameras or from other sensors. The depth images allow us to use depth-constancy constraint equations (ZCCE) that are similar to the classic brightness-constancy constraint equations (BCCE). The depth images also give us linear constraints , even when we use a perspective-camera model. In Section 3, we review these constraint equations and use twist mathematics [2] to capture the motion constraints imposed by the articulated joints. Our basic twist derivations are similar in spirit to the derivations of Bregler et al. [3]. The primary differences trace back to the approximations made within the derivations: Bregler approximates perspective constraints using scaled-orthographic constraints and he approximates the body-rotation matrix using an extra first-order Taylor-series expansion. We avoid this first-order approximation by solving our constraints on a transformed parameter set and by remapping our results into the original parameter set using a closed-form non-linear function (Section 3.5). Throughout Section 3, we assume that we know which limb each pixel corresponds to. To get this information, we