Relative $\alpha$-Entropy Minimizers Subject to Linear Statistical Constraints

We study minimization of a parametric family of relative entropies, termed relative α-entropies (denoted Iα(P,Q)). These arise as redundancies under mismatched compression when cumulants of compressed lengths are considered instead of expected compressed lengths. These parametric relative entropies are a generalization of the usual relative entropy (KullbackLeibler divergence). Just like relative entropy, these relative αentropies behave like squared Euclidean distance and satisfy the Pythagorean property. Minimization of Iα(P,Q) over the first argument on a set of probability distributions that constitutes a linear family is studied. Such a minimization generalizes the maximum Rényi or Tsallis entropy principle. The minimizing probability distribution (termed Iα-projection) for a linear family is shown to have a power-law.