Joint Non-rigid Motion Estimation and Segmentation

Usually object segmentation and motion estimation are considered (and modelled) as different tasks. For motion estimation this leads to problems arising especially at the boundary of an object moving in front of another if e.g. prior assumptions about continuity of the motion field are made. Thus we expect that a good segmentation will improve the motion estimation and vice versa. To demonstrate this we consider the simple task of joint segmentation and motion estimation of an arbitrary (non-rigid) object moving in front of a still background. We propose a statistical model which represents the moving object as a triangular (hexagonal) mesh of pairs of corresponding points and introduce an provably correct iterative scheme, which simultaneously finds the optimal segmentation and corresponding motion field.