We propose a simple and general method to regularize the fine-tuning of Transformer-based encoders for text classification tasks. Specifically, during we generate adversarial examples by perturbing word embedding matrix model perform contrastive learning on clean in order teach learn noise-invariant representations. By training both along with additional objective, observe consistent improvemen...

This paper explores the question of how predictive uncertainty methods perform in practice Natural Language Processing, specifically multi-class and multi-label text classification. We conduct benchmarking experiments with 1-D convolutional neural networks pre-trained transformers on six real-world classification datasets which we empirically investigate why popular scalable estimation strategi...

Abstract Extreme Multilabel Text Classification (XMTC) is a text classification problem in which, (i) the output space extremely large, (ii) each data point may have multiple positive labels, and (iii) follows strongly imbalanced distribution. With applications recommendation systems automatic tagging of web-scale documents, research on XMTC has been focused improving prediction accuracy dealin...

The first step in any NLP pipeline is to split the text into individual tokens. most obvious and straightforward approach use words as However, given a large corpus, representing all not efficient terms of vocabulary size. In literature, many tokenization algorithms have emerged tackle this problem by creating subwords, which turn limits size corpus. Most techniques are language-agnostic, i.e.,...

Consider a supervised learning problem in which examples contain both numericaland text-valued features. One common approach to this problem would be to treat the presence or absence of a word as a Boolean feature, which when combined with the other numerical features enables the application of a range of traditional feature-vector-based learning methods. This paper presents an alternative appr...

Consider a supervised learning problem in which examples contain both numericaland textvalued features. To use traditional feature-vector-based learning methods, one could treat the presence or absence of a word as a Boolean feature and use these binary-valued features together with the numerical features. However, the use of a text-classification system on this is a bit more problematic—in the...