Automatic machine-learning based identification of diagnostic test accuracy studies for systematic reviews

This study evaluates several machine-learning based approaches for identifying diagnostic test accuracy studies within a large set of scientific articles for the purpose of reducing the workload of researchers when manually screening abstracts to be utilized in systematic reviews. The relative performance was measured between three documents representation models, with different degrees of random undersampling of the data set and with two different algorithms. This includes document representations in the form of TF-IDF bag-of-words, word2vec and Latent Dirichlet Allocation, and the Support Vector Machine algorithm as well as a Logistic Model Tree algorithm. The results seem to indicate that undersampling does not lead to increased performance of this particular data set, that the word2vec representation outperforms the others in a ranking approach, and while the most frequently used TF-IDF model is more robust in a classification approach.