We describe a trainable, multi-stroke symbol recognizer for pen-based user interfaces. The approach is insensitive to orientation, nonuniform scaling, and drawing order. Symbols are represented internally as attributed relational graphs describing both the geometry and topology of the symbols. Symbol definitions are statistical models, which makes the approach robust to variations common in han...

As pen-based devices become more popular, it has motivated several novel research issues in the area of pen computing. One interesting and practical problem is the storage and retrieval of hand-drawn sketches. A sketch can consist of handwritten notes, symbols, free-form hand-drawings, annotations on a document etc. It will be very useful to store the sketches in a database and then retrieve th...

Objects in freely-drawn sketches often have no spatial or temporal separation, making object recognition difficult. We present a two-step stroke-grouping algorithm that first classifies individual strokes according to the type of object to which they belong, then groups strokes with like classifications into clusters representing individual objects. The first step facilitates clustering by natu...

We introduce a model that learns to convert simple hand drawings into graphics programs written in a subset of LATEX. The model combines techniques from deep learning and program synthesis. We learn a convolutional neural network that proposes plausible drawing primitives that explain an image. This set of drawing primitives is like an execution trace for a graphics program. From this trace we ...

We describe an application of the novel Support Vector Machined Kernel (SVM’ed Kernel) to the Recognition of hand-drawn shapes. The SVM’ed kernel function is itself a support vector machine classifier that is learned statistically from data using an automatically generated training set. We show that the new kernel manages to change the classical methodology of defining a feature vector for each...

In this paper we propose an exploratory study for the application of Computational Linguistics techniques such as the Semantic Role Labeling of natural languages (SRL) to the case of sketch understanding. Our approach originated from a preliminary psychological experiment in which participants were asked to produce hand-drawn sketches from sentences embedding various semantic features; the effe...

This paper deals with the incremental interpretation of on-line hand-drawn structured documents. We recall the bases of the CD-CMG visual grammar we have presented previously, but we mainly focus on its associated parser. The formalism context-sensitiveness allows to exploit a parser with predictive capacities which couples bottomup and top-down strategies. It also helps targeting consistent ca...

tanding in front of Cinderella’s castle at Walt Disney World is a statue of two figures, one tall and one short, holding hands like parent and child. The tall figure is a representation of Walt Disney himself, evoking memories for baby boomers who watched him as the genial narrator of Disney’s Sunday evening television show. The smaller figure, the child, is Mickey Mouse, including those tradem...

Words often fail when teaching science. Although some scientific concepts lend themselves to easily articulated descriptions and definitions, others consist mostly of a language of symbols only decipherable by knowledgeable experts. Diagrams can help. However, some researchers and science educators believe another medium, known more for its entertainment value than its educational value, is oft...