Entropy Versus Pairwise Independence

We give lower bounds on the joint entropy of n pairwise independent random variables. We show that if the variables have no dominant value (their min-entropies are bounded away from zero) then this joint entropy grows as Ω(log n). This rate of growth is known to be best possible. If k-wise independence is assumed, we obtain an optimal Ω(k log n) lower bound for not too large k. We also show that the joint entropy of an arbitrary family of pairwise independent random variables grows as Ω(min(L, p log(2 + L)) where L is the sum of the entropies of the variables in the family. We expect that the √ log in this expression can be improved to log. We also prove a tight Ω(log log n) lower bound on the joint entropy of n balanced Bernoulli trials with pairwise correlation bounded away from 1.