Using Hashtags as Labels for Supervised Learning of Emotions in Twitter Messages

Many college students experience depression or anxiety but do not seek help due to the social stigma associated with psychological counseling services. Automatic techniques to classify social media messages based on the emotions they express can assist in the early detection of students in need of counseling. Supervised machine learning methods yield accurate results but require training datasets of text messages that have been labelled with the classes of emotions they express. Manually labeling a large corpus of Twitter messages is labor-intensive, error prone and time-consuming. Hashtags are keywords inserted into social media messages by their authors. In this paper, we investigate using hashtags as emotion labels and evaluate them through two user studies, one with psychology experts and the other with the general crowd. The study showed that the labels created by general crowd was inconsistent and unreliable. However, the labels generated by experts matched with hashtag labels in over 87% of Twitter messages, which indicates that hashtags are indeed good emotion labels. Leveraging the concept of hashtags as emotion labels, we develop Emotex, a supervised learning approach that classifies Twitter messages into the emotion classes they express. We show that Emotex correctly classifies the emotions expressed in over 90% of text messages.