Multi-Modal Chatbot in Intelligent Manufacturing

Artificial intelligence (AI) has been widely used in various industries. In this work, we concentrate on what AI is capable of doing manufacturing, the form a chatbot. We designed chatbot that helps users complete an assembly task simulates those manufacturing settings. order to recreate setting, have assemble Meccanoid robot through multiple stages, with help interactive dialogue system. Based classifying users' intent, able provide answers or instructions user when encounters problems during process. Our goal improve our system so it can capture needs by detecting their intent and therefore relevant helpful information user. However, multiple-step task, cannot rely classification question utterance as only input, questions raised from different steps may share same but require responses. paper, proposed two methods address problem. One not textual features also visual YOLO-based Masker CNN (YMC) model. Another usage Autoencoder encode multi-modal for classification. By incorporating information, significantly improved chatbot's performance experiments conducted dataset.