Online Submission ID: Image-space correction of AR registration errors using graphics hardware

Many Mixed Reality applications rely on drawing virtual imagery directly on top of physical objects in a video scene. Registration accuracy is a serious problem in these cases since any imprecisions are immediately apparent as virtual and physical edges and features coincide. We present a hardware-accelerated image-based post-processing technique that adjusts rendering of virtual geometry to better match edges present in images of a physical scene, reducing the visual effect of registration errors from both inaccurate tracking and oversimplified modeling. We detect intensity edges in an image of the scene captured by a camera, and search for these edges around the boundary of projected polygons. These detected edges are used to clip the boundaries of the rendered polygons, making virtual geometry edges match strong image features. Our algorithm is easily integrable with existing AR applications, having no dependency on the underlying tracking technique. We use the advanced programmable capabilities of modern graphics hardware to achieve high performance without burdening the CPU. CR Categories: I.3.3 [Computer Graphics]: Picture/Image Generation—Display Algorithms I.3.7 [Computer Graphics]: Three-Dimensional Graphics and Realism—Virtuality Reality I.4.6 [Image Processing]: Segmentation—Edge and Feature Detection