Minimum MS. E. Gerber's Lemma
نویسندگان
چکیده
—Mrs. Gerber's Lemma lower bounds the entropy at the output of a binary symmetric channel in terms of the entropy of the input process. In this paper, we lower bound the output entropy via a different measure of input uncertainty, pertaining to the minimum mean squared error (MMSE) prediction cost of the input process. We show that in many cases our bound is tighter than the one obtained from Mrs. Gerber's Lemma. As an application, we evaluate the bound for binary hidden Markov processes, and obtain new estimates for the entropy rate.
منابع مشابه
The Entropy Power Inequality and Mrs. Gerber's Lemma for groups of order 2n
Shannon’s Entropy Power Inequality (EPI) can be viewed as characterizing the minimum differential entropy achievable by the sum of two independent random variables with fixed differential entropies. The EPI is a powerful tool and has been used to resolve a number of problems in information theory. In this paper we examine the existence of a similar entropy inequality for discrete random variabl...
متن کاملThe Entropy Power Inequality and Mrs. Gerber's Lemma for Abelian Groups of Order 2^n
Shannon’s Entropy Power Inequality can be viewed as characterizing the minimum differential entropy achievable by the sum of two independent random variables with fixed differential entropies. The entropy power inequality has played a key role in resolving a number of problems in information theory. It is therefore interesting to examine the existence of a similar inequality for discrete random...
متن کاملGeneralization of Mrs. Gerber's Lemma
Mrs. Gerber’s Lemma (MGL) hinges on the convexity of H(p ∗ H(u)), where H(u) is the binary entropy function. In this work, we prove thatH(p∗f(u)) is convex in u for every p ∈ [0, 1] provided H(f(u)) is convex in u, where f(u) : (a, b) → [0, 1 2 ]. Moreover, our result subsumes MGL and simplifies the original proof. We show that the generalized MGL can be applied in binary broadcast channel to s...
متن کاملRandom Walks with the Minimum Degree Local Rule Have O(N2) Cover Time
For a simple (unbiased) random walk on a connected graph with n vertices, the cover time (the expected number of steps it takes to visit all vertices) is at most O(n). We consider locally biased random walks, in which the probability of traversing an edge depends on the degrees of its endpoints. We confirm a conjecture of Abdullah, Cooper and Draief [2015] that the min-degree local bias rule en...
متن کاملMatchings and Tilings in Hypergraphs
We consider two extremal problems in hypergraphs. First, given k ≥ 3 and k-partite k-uniform hypergraphs, as a generalization of graph (k = 2) matchings, we determine the partite minimum codegree threshold for matchings with at most one vertex left in each part, thereby answering a problem asked by Rödl and Ruciński. We further improve the partite minimum codegree conditions to sum of all k par...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- IEEE Trans. Information Theory
دوره 61 شماره
صفحات -
تاریخ انتشار 2015