Occlusion robust pose and model estimation using Gaussian Process Latent Variable Model on GPU

In this project, we would like to develop multi-object pose and model estimation which uses the 3D voxelized model and foreground background segmentation to detect 3D pose and model variation. In the relevant literature, [4], the authors used gradient descent to minimize the optimization energy function which is the integration over image multiplied by the smoothed step function of SDF of 3D model (silhouette) to jointly optimize the 6 degree location and rotations and deformation of model. The system is generally stable given prior mask and pose and works faster than state-of-the-art 3D pose estimation algorithm such as Zia and Stark [7]. However, the system requires prior mask which is hard to get automatically. In this project, we propose automatic segmentation mechanism that make use of object relation and alternating multi-object newton step. Thus, we improve the system by making it robust to texture and background color and especially occlusion from other object. The pipeline is first, we use [2] object detector to find location of objects and use these bounding boxes for contour detection techniques such as hierarchical image segmentation [8] or Grab Cut to capture the contour of objects which is essential for correct pose estimation. Then we run newton step for pose and model estimation for multiple objects.