The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization

Abstract We present a novel benchmark and associated evaluation metrics for assessing the performance of text anonymization methods. Text anonymization, defined as task editing document to prevent disclosure personal information, currently suffers from shortage privacy-oriented annotated resources, making it difficult properly evaluate level privacy protection offered by various This paper presents TAB (Text Anonymization Benchmark), new, open-source corpus developed address this shortage. The comprises 1,268 English-language court cases European Court Human Rights (ECHR) enriched with comprehensive annotations about information appearing in each document, including their semantic category, identifier type, confidential attributes, co-reference relations. Compared previous work, is designed go beyond traditional de-identification (which limited detection predefined categories), explicitly marks which spans ought be masked order conceal identity person protected. Along presenting its annotation layers, we also propose set that are specifically tailored toward measuring both terms utility preservation. illustrate use proposed empirical several baseline models. full along guidelines, scripts, models available on: https://github.com/NorskRegnesentral/text-anonymization-benchmark.