Managing, Mining and Visualizing Multi-Modal Data for Stress Awareness Master Thesis

The stress problem has become the major dilemma affecting many people’s lives and professions. Nowadays, stress is often unrecognized, and people accept stress as a normal condition. Short term stress is not necessarily bad, as on some occasions it can help us to meet challenges. On the other hand, prolonged stress should be avoided because it has been shown to cause physical breakdowns and makes our body vulnerable to diseases. In this thesis, we propose a framework for stress analytics, which focuses on management and analysis of multi-modal affective data captured in text, speech, facial expression and physiological signals, such as Galvanic Skin Response (GSR). The framework allows for automatic stress detection based on multimodal data, for instance, from GSR and speech. We investigate the discriminating power of speech and GSR in distinguishing two different stress levels in the controlled experiment environment. A collective of 10 subjects voluntarily participated in the psychological study for stress elicitation. The stress was induced by using the Stroop-Word color test and solving mental arithmetic problems. During the experiment, the speech was recorded and the homemade GSR device was used to monitor the skin conductance. Four different machine learning classifiers were investigated regarding their ability to discriminate between two different stress levels. The state-of-the-art classifier, Support Vector Machine (SVM), outperformed the other classifiers. The reasonable accuracy of 70% was achieved by using individual GSR data as an input to SVM classifier. On the other hand, using the speech signal as an input to an SVM classifier yields a maximum accuracy of 92%. Furthermore, combining both GSR and speech models does not improve the performance in significant ways.