Abstract We present a memory-augmented approach to condition an autoregressive language model on knowledge graph. represent the graph as collection of relation triples and retrieve relevant relations for given context improve text generation. Experiments WikiText-103, WMT19, enwik8 English datasets demonstrate that our produces better in terms perplexity bits per character. also show relational...

A language model assigns a probability to a piece of unseen text, based on some training data. For example, a language model based on a big English newspaper archive is expected to assign a higher probability to “a bit of text” than to “aw pit tov tags”, because the words in the former phrase (or word pairs or word triples if so-called N -GRAM MODELS are used) occur more frequently in the data ...

Generative language models are promising for assisting human writing in various domains. This manuscript aims to build generative the patent domain and evaluate model performance from a human-centric perspective. The perspective is measure ratio of keystrokes that can be saved by autocompletion based on models. A higher means more effective which save keystrokes. metric used benchmark performan...

In statistically based natural language processing a language model is used in the generation of output strings to assess likelihood that a given string of words is a sentence in a particular language. If the information about the ordering of the words in the sentences in a texts were discarded so that each sentence was regarded simply as a bag of words, then the better of two language models w...

This paper describes the uses of computer models in studying the evolution of language. Language is a complex dynamic system that can be studied at two fold one at the level of biological evolution of some innate linguistic ability, and other at the level of cultural evolution of specific languages and language families. The dynamics of language evolution or rather language acquisition occurs b...