From General Language Understanding to Noisy Text Comprehension

Obtaining meaning-rich representations of social media inputs, such as Tweets (unstructured and noisy text), from general-purpose pre-trained language models has become challenging, these inputs typically deviate mainstream English usage. The proposed research establishes effective methods for improving the comprehension texts. For this, we propose a new generic methodology to derive diverse set sentence vectors combining extracting various linguistic characteristics latent multi-layer, models. Further, clearly establish how BERT, state-of-the-art model, comprehends attributes identify appropriate representations. Five probing tasks are developed Tweets, which can serve benchmark study text comprehension. Experiments carried out classification accuracy by deriving GloVe-based Sentence-BERT, using different hidden layers BERT model. We show that initial middle have better capability capturing key texts than its latter layers. With complex predictive models, further vector length lesser importance capture information, perform existing vectors.