Our group is interested in creating human machine interfaces which use natural modalities such as vision and speech to sense and interpret a user's actions [6]. In this paper we describe recent work on multimodal adaptive interfaces which combine automatic speech recognition, computer vision for gesture tracking, and machine learning techniques. Speech is the primary mode of communication betwe...