Inequalities for Sums of Joint Entropy Based on the Strong Subadditivity

In what follows, V = {1, . . . , n} is the set of indices of given random variables X1, . . . , Xn, and B = {B1, . . . , Bm} is a set of subsets (possibly with repeat) of V . Furthermore, for S = {i1, . . . , il} ⊆ V , XS and H(XS) denote the random vector (Xi1 , . . . , Xil) and its Shannon entropy H(Xi1 , . . . , Xil) (H(X∅) = 0). The power set (the set of all subsets) and the set of all l-subsets of V are written as 2 and ( V l ) , respectively. For simplicity, we state results only for discrete random variables with finite alphabets for which the entropy functions are always well-defined. The following entropy inequality, which is called Shearer’s inequality, is given in [1] as a key lemma used in certain combinatorial argument.