Transactions on Computer - Human Interaction

Nowadays, more and more people play games on touch screen mobile phones. This phenomenon raises a very interesting question: does touch behavior reflect the player’s emotional state? If possible, this would be a valuable evaluation indicator for game designers but also for realtime personalization of the game experience. Psychology studies on acted touch behaviour show the existence of discriminative affective profiles. In this paper, finger stroke features during gameplay on an iPod were extracted and their discriminative power analysed. A system was built and tested to recognize four emotional states (Excited, Relaxed, Frustrated and Bored) and 2 levels of arousal and two of valence. Discriminant Analysis of the collected data shows that pressure features discriminate frustration states from the other three states. Stroke speed and directness features discriminate between different levels of arousal whilst stroke length features discriminate mainly boredom from a relaxed state. Three machine learning algorithms were used to build a person-independent automatic emotion recognition system based on touch behaviour. All three algorithms produced very interesting results in discriminating between 4 emotional states reaching between 69% and 77% of correct recognition. Higher results (~89%) were obtained for discriminating between two levels of arousal and between two levels of valence. These results highlight the potential of using touch behaviour as a non-obstructive way to measure users’ emotional states in contexts where touch-based devices are used.