Evaluating stance annotation of Twitter data

Taking stance towards any topic, event or idea is a common phenomenon on Twitter and social media in general. users express their opinions about different matters assess other people’s various discursive ways. The identification analysis of the linguistic ways that people use to take stances leads better understanding language user behaviour Twitter. Stance multidimensional concept involving broad range related notions such as modality, evaluation sentiment. In this study, we annotate data from using six notional categories ––contrariety, hypotheticality, necessity, prediction, source knowledge uncertainty––¬¬ following comprehensive annotation protocol including inter-coder reliability measurements. relatively low agreement between annotators highlighted challenges task entailed, which made us question inter-annotator score reliable measurement quality categories. nature data, difficulty type are discussed, potential solutions suggested