where pi ≡ p(x = i). If only M of the K states have non-zero probability and that probability is 1/M then S = logM . In this special case the entropy is the log of the number of occupied states. An analog of this special case in the continuous domain occurs in Liouville’s Theorem. This states that if the volume of phase space (i.e. of the x variable, but now multivariate and continuous) is some...