Action Recognition From Weak Alignment of Body Parts

The objective of this paper is to recognize human actions in still images. The contribution of this work is a novel framework for obtaining weak alignment of human body-parts to improve the recognition performance. Our framework implicitly exploits physical constraints of human body parts (e.g., heads are above necks, hands are attached to forearms). It uses the locations of some detected body parts to aid the alignment of some others. Specifically, we demonstrate the benefit of our framework for computing registered feature descriptors from automatically detected upper bodies and silhouettes. Fig. 1 illustrates the benefits of our approach over the grid-alignment approach. Given the bounding box of a human, we approximate the human body by a set of deformable rectangular parts, which is similar to a DPM [1]. The goal is to align these rectangular parts between two images, referred to as reference and probe images. We formulate the problem as a minimization of a deformation energy between the parts of the reference (which are fixed as a default grid formation) and those of the probe (which deform to best match those of the reference). The energy encourages the parts to overlap the silhouette and upper body in a consistent way (between reference and probe) whilst penalizing severe deformations. The energy is defined for a configuration of parts, and it is formulated as the sum of unary and pairwise terms. Consider aligning a human specified by a bounding box b in the probe image I to another human specified by the bounding b f in the reference image I f . Let p re f 1 , · · ·p re f k be the default configuration of parts for the reference image at the bounding box b f . We consider the following energy function for a configuration of parts p1, · · · ,pk of a probe image I: