Computer-aided methods, based on the entropic linear program framework, have been shown to be effective in assisting study of information theoretic fundamental limits systems. One key element that significantly impacts their computation efficiency and applicability is reduction variables, problem-specific symmetry dependence relations. In this work, we propose using disjoint-set data structure ...

Abstract We consider generalized linear models in regimes where the number of nonzero components signal and accessible data points are sublinear with respect to size signal. prove a variational formula for asymptotic mutual information per sample when system grows infinity. This result allows us derive an expression minimum mean-square error (MMSE) Bayesian estimator entries have discrete distr...

This paper is concerned with the information-theoretical limits of density estimation for Gaussian random variables with data drawn independently and with identical distributions. We apply Fano’s inequality to the space of densities and an arbitrary estimator. We derive necessary conditions on the sample size for reliable density recovery and for reliable density estimation. These conditions ar...