In n-tuple Zipf analysis, ‘‘words’’ are defined as strings of n digits, and their normalized frequency of occurrence v is measured for a given ‘‘text’’ ~sequence of digits!. In the case of various non-Markovian sequences, the probability density of the frequencies P(v) has a power-law tail. Here we argue that a broad class of unbiased binary texts exhibiting a nonexponential distribution of clu...

In Phys. Rev. Letters, 73:2, 5 Dec. 94, Mantegna et al. conclude on the basis of Zipf rank frequency data that noncoding DNA sequence regions are more like natural languages than coding regions. We argue on the contrary that an empirical t to Zipf's \law" cannot be used as a criterion for similarity to natural languages. Although DNA is a presumably an \organized system of signs" in Mandelbrot'...

From a grammar point of view, the role of punctuation marks in a sentence is formally defined and well understood. In semantic analysis punctuation plays also a crucial role as a method of avoiding ambiguity of the meaning. A different situation can be observed in the statistical analyses of language samples, where the decision on whether the punctuation marks should be considered or should be ...

Corpora, large bodies of text, are of great importance to the field of Natural Language Processing. They are for instance used to train systems on specific tasks through machine learning. When a system is trained on a corpus of low quality, it will not provide reliable results. We search for a metric of corpus quality by comparing the vocabulary growth data, the Zipf slopes and the Pareto expon...