CESSM : Collaborative Evaluation of Semantic Similarity Measures

The application of semantic similarity measures to proteins annotated with Gene Ontology terms has become a common method in bioinformatics. However, the evaluation of these measures is still challenging, since no common standard of evaluation exists. We present an online tool for the automated evaluation of GO-based semantic similarity measures, CESSM, that enables the comparison of new measures against previously published ones considering their relation to sequence, Pfam and EC similarity. The tool also has a collaborative component, by which the authors of published measures can contribute to the enrichment of the evaluation by providing their own results. CESSM is freely available at http://xldb.di.fc.ul.pt/tools/cessm/ BACKGROUND The creation of the Gene Ontology (GO) [1], a controlled vocabulary for the description of gene product functions, triggered the development of computational methods that take advantage of its structured information. One such method is the application of semantic similarity measures to GO terms, whereby the similarity between two terms is calculated according to their relationship in the ontology. Likewise, semantic similarity measures can also be used to calculate the similarity between gene products, provided they are annotated with GO terms. Several semantic similarity measures based on GO have been proposed in recent years [2-12], but the evaluation of their performance has been identified as a relevant problem in the field [13]. Various evaluation strategies have been proposed, including the investigation of the relation between the semantic similarity measure and other gene product or protein similarities (such as sequence[2-7], family [12,7] or expression similarity [8,14,15]); and of the feasibility to use semantic similarity measures in such distinct scenarios as the prediction of subnuclear location [16], the ability to characterize human regulatory pathways [17], or the performance in gene clustering [9,10]. This multiplicity of evaluation strategies arises from the lack of a gold standard suitable to this scenario, driving researchers to use diverse data sets, to which they apply distinct evaluation strategies, thus rendering comparison among different works unfeasible. We present an online tool CESSM (Collaborative Evaluation of Semantic Similarity Measures) for the collaborative and automated evaluation of semantic similarity measures in the context of GO. CESSM allows researchers to compare the performance of their novel semantic similarity measures against several existing ones, using the same protein and annotation dataset and according to three distinct aspects: relation with sequence, EC class and Pfam family similarities.