Leveraging Textual Features for Best Answer Prediction in Community-based Question Answering

One of the intriguing problems in Community-based Question Answering (CQA) research is the automatic identification of the best answer, which is expected to benefit various stakeholders. First of all, since several answers are provided for each question, the readers of these websites will be able to process the candidate answers more efficiently and mitigate the “information overload” phenomenon. Secondly, a mechanism that identifies high quality answers will increase awareness within the community and will help to put more effort into questions that remain poorly answered. For instance, in StackOverflow(SO) alone, as of September 2013, we found that approximately 33% of the questions have yet to be marked as resolved (i.e., out of the 5 million, 1.7 million questions have no answer marked as “accepted”). Researchers in related fields have used lexical, syntactic, and discourse features to produce a predictive model of readers’ judgments [3]. In several cases, the use of shallow features, i.e. features that do not employ semantic or syntactic parsing such as sentence length or word length, have been shown to be effective in assessing properties such as ease of reading or usefulness. However, with respect to CQA, research efforts towards the exploitation of shallow features report relatively low results. To improve the efficacy of their models, researchers refer to more contextual information, such as the score of each answer, the comments received or the reputation of the user [1]. However, these features may not be readily available since a) comments and scores introduce an inherent delay, and b) features based on reputation may not be applicable on a newly formed community or pose a threat to its development (i.e. preferential attachment) and result in the reinforcement of the pre-existing community hierarchy. In our approach, we revisit the case of shallow linguistic features and use features found in [3]. Figure 1 shows the average feature values for the accepted answers together with the non-accepted ones of SO using a one-month window time frame. As seen from the figure, the linguistic features clearly differentiate the accepted from the non-accepted answers. More specifically, accepted answers tend to be longer, use a less common vocabulary, contain longer words, more words per sentence and the longest sentences are lengthier. Even though the above remarks look promising concerning best answer prediction, when training a binary classifier prediction remains weak (58% precision and 0.56 F-Measure