We provide a new result that links two crucial entropy notions: Shannon Entropy H1 and collision entropy H2. Our formula gives the worst possible amount of collision entropy in a probability distribution, when its Shannon Entropy is fixed. Our results and techniques used in the proof immediately imply many quantitatively tight separations between Shannon and smooth Renyi entropy, which were pre...

This thesis focuses on the Shannon capacity of a graph. Suppose we want to send a message across a channel to a receiver. The general question is about the effective size of an alphabet in a model such that the receiver may recover the original message without errors. To answer this question Shannon defined the Shannon capacity of a graph, Θ(G), and stated Shannon’s Theorem. We study an article...

In this note we revisit the famous result of Shannon [Sha49] stating that any encryption scheme with perfect security against computationally unbounded attackers must have a secret key as long as the message. This result motivated the introduction of modern encryption schemes, which are secure only against a computationally bounded attacker, and allow some small (negligible) advantage to such a...

entity; it is always tied to a physical representation. It is represented by engraving on a stone tablet, a spin, a charge, a hole in a punched card, a mark on a paper, or some other equivalent.” (1996, p. 188; see also Landauer 1991). This view is also adopted by some philosophers of science; for instance, Peter Kosso states that “information is transferred between states through interaction.”...

The Shannon upper bound places a limit on the error-free information transmission rate (capacity) of a noisy channel. It has stood for over sixty years, and underlies both theoretical and practical work in the telecommunications industry. This upper bound arises from the Shannon-Hartley law, which has two parameters: the available bandwidth and the signal-to-noise power ratio. However, aside fr...

We consider the problem of communicating the state of a dynamical system via a Shannon Gaussian channel with a given power constraint and no feedback. The transmitter observes a possibly noisy measurement of the state. These measurements are then used to encode the message to be transmitted over a noisy Gaussian channel, where a power constraint is imposed on the transmitted message. The receiv...

Claude Shannon founded information theory in the 1940s. The theory has long been known to be closely related to thermodynamics and physics through the similarity of Shannon's uncertainty measure to the entropy function. Recent work using information theory to understand molecular biology has unearthed a curious fact: Shannon's channel capacity theorem only applies to living organisms and their ...