Unbiased Estimators for Entropy and Class Number

We introduce unbiased estimators for the Shannon entropy and the class number, in the situation that we are able to take sequences of independent samples of arbitrary length. Introduction. This paper supposes that we may pick a sequence of arbitrary length of independent samples w1, w2, . . . from an infinite population. Each sample belongs to one of M classes C1, C2, . . . , CM , and the probability that a sample belongs to class Ci is pi. So these probabilities satisfy the constraints 0 ≤ pi ≤ 1 and ∑M i=1 pi = 1. The goal of this paper is to present methods to estimate the Shannon entropy H = − ∑M i=1 pi log(pi) (see [8]). An obvious method is to take a sample of size n, and compute the estimators p̂i = ki/n, where ki are the number of samples from the class Ci. However this is known to systematically underestimate the entropy, and it can be significantly biased [2][4][5][6]. Recently there have been more advanced estimators for the entropy which have smaller bias [2][3][4][5][6][7]. In this paper we introduce new entropy estimators that have bias identically zero. We also introduce an unbiased estimator for the class number M , a problem of interest to ecologists (see for example the review article [1]). The disadvantage of all our methods is that there is no a priori estimate of the sample size. For this reason, we postpone rigorous analysis of variance and other measures of confidence until it becomes clear that these estimators are of more than theoretical value. We will use the following power series. Define the harmonic number by hn = ∑n k=1 1/k, h0 = 0. Then for |x| < 1 1 (1− x)2 = ∞ ∑