Multi-modal communication system for mobile robot
نویسندگان
چکیده
This paper presents a cloud-based multi-modal human-robot interaction architecture. The architecture consists of parts running on the edge (mobile) platform and cluster services. Software mobile is represented primarily by Robot Operating System, an MQTT broker client side SpeechCloud architecture, which provides communication with computing cluster. Its detailed description part paper. main contribution design based standard frameworks, protocols, validated JSON messages. also focuses tested equipped NVIDIA Jetson family computer information about tasks that robot will be able to perform using proposed
منابع مشابه
A multi-modal dialog system for a mobile robot
A challenging domain for dialog systems is their use for the communication with robotic assistants. In contrast to the classical use of spoken language for information retrieval, on a mobile robot multimodal dialogs and the dynamic interaction of the robot system with its environment have to be considered. In this paper we will present the dialog system developed for BIRON – the Bielefeld Robot...
متن کاملWearable Telepresence System using Multi-modal Communication with Humanoid Robot
This paper presents a new type of wearable telepresence system that is applicable to the control of a humanoid robot. The system consists of self-contained computing hardware with a stereo HMD, a microphone, a headphone and a wireless LAN, an arm and head motion tracking mechanism that uses several types of sensors to get the motion data of an operator, and a simple force reflection mechanism t...
متن کاملSpontaneous Speech Understanding for Robust Multi-Modal Human-Robot Communication
This paper presents a speech understanding component for enabling robust situated human-robot communication. The aim is to gain semantic interpretations of utterances that serve as a basis for multi-modal dialog management also in cases where the recognized word-stream is not grammatically correct. For the understanding process, we designed semantic processable units, which are adapted to the d...
متن کاملMulti-modal human-machine communication for instructing robot grasping tasks
A major challenge for the realization of intelligent robots is to supply them with cognitive abilities in order to allow ordinary users to program them easily and intuitively. One way of such programming is teaching work tasks by interactive demonstration. To make this effective and convenient for the user, the machine must be capable to establish a common focus of attention and be able to use ...
متن کاملA multi-modal system for tracking and analyzing faces on a mobile robot
This paper describes a user detection system which employs a saliency system working on an omnidirectional camera delivering a rough and fast estimate of the position of a potential user. It consists of a vision (skin color) and a sonar based component, which are combined to make the estimate more reliable. To make the skin color detection robust under varying illumination conditions, it is sup...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IFAC-PapersOnLine
سال: 2022
ISSN: ['2405-8963', '2405-8971']
DOI: https://doi.org/10.1016/j.ifacol.2022.06.022