FormPad: A Camera-Assisted Digital Notepad

A camera-assisted digital writing tablet was invented recently. It preserves the familiar experience of filling out a paper form while allowing automatic conversion of relevant handwritten field entries into electronic form, without explicit form scanning. In this paper, we focus on two key computer vision problems associated with the invention of this device, namely, form indexing and field projection. These are needed for accurate association of tablet writing with corresponding entries in the electronic form. Form indexing is modeled as the problem of shape-based content retrieval using the perspectively-distorted form appearances seen from the tablet camera. Fast form indexing is achieved using geometric hashing based on projective invariants. The invariants derived from curve and line features reduce the basis search space considerably while still providing for robust localization. We derive field projection as a sequence of projective transformations between the tablet, the camera and the original electronic form coordinates. Results of extensive testing on a medical form database are reported.