Problematic Situation Analysis and Automatic Recognition for Chinese Online Conversational System

Automatic problematic situation recognition (PSR) is important for an online conversational system to constantly improve its performance. A PSR module is responsible of automatically identifying users’ un-satisfactions and then sending feedbacks to conversation managers. In this paper, we collect dialogues from a Chinese online chatbot, annotate the problematic situations and propose a framework to predict utterance-level problematic situations by integrating intent and sentiment factors. Different from previous work, the research field is set as open-domain in which very few domain specific textual features could be used and the method is easy to be adapted to other domains. Experimental results show that integrating both intent and sentiment factors gains the best performance.