Uni cation-based Multimodal Integration

Unification-based Multimodal Integration

Recent empirical research has shown conclusive advantages of multimodal interaction over speech-only interaction for mapbased tasks. This paper describes a multimodal language processing architecture which supports interfaces allowing simultaneous input from speech and gesture recognition. Integration of spoken and gestural input is driven by uni cation of typed feature structures representing ...

Information Integration through Uni cation of Feature Structures

Complementarity and redundancy in multimodal user inputs with speech and pen gestures

We present a comparative analysis of multi-modal user inputs with speech and pen gestures, together with their semantically equivalent uni-modal (speech only) counterparts. The multimodal interactions are derived from a corpus collected with a Pocket PC emulator in the context of navigation around Beijing. We devise a cross-modality integration methodology that interprets a multi-modal input an...

A Universal Unification Algorithm Based on Unification-Driven Leftmost Outermost Narrowing

We formalize a universal uni cation algorithm for the class of equational theories which is induced by the class of canonical totally de ned not strictly subuni able term rewriting systems for short ctn trs For a ctn trs R and for two terms t and s the algorithm computes a ground complete set of ER uni ers of t and s where ER is the set of rewrite rules of R viewed as equations The algorithm is...

Maintaining Cross Viewpoint Consistency using Z

This paper discusses the use and integration of formal techniques in particular Z into the Open Distributed Processing ODP standardization initiative One of the cornerstones of the ODP framework is a model of multiple viewpoints During the development process it is important to maintain the consistency of di erent viewpoints of the same ODP speci cation In addition there must be some way to com...