Use of projector-camera system for human-computer interaction

The use of a projector in place of traditional display device would dissociate display size from device size, making portability much less an issue. Associated with camera, the projector-camera system allows simultaneous video display and 3D acquisition through imperceptible structured light sensing, providing a vivid and immersed platform for natural human-computer interaction. Key issues involved in the approach include: (1) Simultaneous Display and Acquisition: how to make normal video projector not only a display device but also a 3D sensor even with the prerequisite of incurring minimum disturbance to the original projection; (2) 3D Information Interpretation: how to interpret the spare depth information with the assistance of some additional cues to enhance the system performance; (3) Segmentation: how to acquire accurate segmentation in the presence of the incessant variation of the projected video content; (4) Posture Recognition: how to infer 3D posture from single image. This thesis aims at providing improved solutions to each of these issues. To address the conflict between imperceptibility of the embedded codes and the robustness of code retrieval, noise-tolerant schemes to both the coding and decoding stages are introduced. At the coding end, specifically designed primitive shapes and large Hamming distance are employed to enhance tolerance toward noise. At the decoding end, pre-trained primitive shape detectors are used to detect and identify the embedded codes – a task difficult to achieve by segmentation that