Zipf’s law states that the frequency of word tokens in a large corpus of natural language is inversely proportional to the rank. The law is investigated for two languages English and Mandarin and for ngram word phrases as well as for single words. The law for single words is shown to be valid only for high frequency words. However, when single word and n-gram phrases are combined together in on...

Statistical language models should improve as the size of the n-grams increases from 3 to 5 or higher. However, the number of parameters and calculations, and the storage requirement increase very rapidly if we attempt to store all possible combinations of n-grams. To avoid these problems, the reduced n-grams’ approach previously developed by O’Boyle (1993) can be applied. A reduced n-gram lang...

We discuss the application of the Zipf–Mandelbrot law on musical pieces encoded in MIDI. Our hypothesis is that this will allow us to computationally identify and emphasize aesthetic aspects of music. Specifically, we have identified an initial set of attributes (metrics) of music pieces on which to apply the Zipf–Mandelbrot law. These metrics include pitch of musical events, duration of musica...

In this paper the Zipf-Mandelbrot law is revisited in the context of linguistics. Despite its widespread popularity the Zipf–Mandelbrot law can only describe the statistical behaviour of a rather restricted fraction of the total number of words contained in some given corpus. In particular, we focus our attention on the important deviations that become statistically relevant as larger corpora a...

Zipf's law seems to be ubiquitous in human languages and appears to be a universal property of complex communicating systems. Following the early proposal made by Zipf concerning the presence of a tension between the efforts of speaker and hearer in a communication system, we introduce evolution by means of a variational approach to the problem based on Kullback's Minimum Discrimination of Info...

A study on a 220-piece corpus (baroque, classical, romantic, 12tone, jazz, rock, DNA strings, and random music) reveals that aesthetically pleasing music may be describable under the Zipf-Mandelbrot law. Various Zipf-based metrics have been developed and evaluated. Some focus on musictheoretic attributes such as pitch, pitch and duration, melodic intervals, and harmonic intervals. Others focus ...

We discuss the application of the Zipf–Mandelbrot law to musical pieces encoded in MIDI. Specifically, we have identified a set of metrics on which to apply this law. These metrics include pitch of musical events, duration of musical events, the combination of pitch and duration of musical events, and several others. Our hypothesis is that these metrics will allow us to computationally identify...