Unsupervised Text Segmentation for Automated Error Reduction

Challenging the assumption that traditional whitespace/punctuation-based tokenisation is the best solution for any NLP application, I propose an alternative approach to segmenting text into processable units. The proposed approach is nearly knowledge-free, in that it does not rely on languagedependent, man-made resources. The text segmentation approach is applied to the task of automated error reduction in texts with high noise. The results are compared to conventional tokenisation. Posted at the Zurich Open Repository and Archive, University of Zurich ZORA URL: https://doi.org/10.5167/uzh-101471 Accepted Version Originally published at: Furrer, Lenz (2014). Unsupervised Text Segmentation for Automated Error Reduction. In: KONVENS 2014, Hildesheim, 8 October 2014 10 October 2014, 178-185. Unsupervised Text Segmentation for Automated Error Reduction Lenz Furrer University of Zurich Binzmühlestr. 14, CH-8050 Zürich