The notion of entropy was introduced by Clausius in 1850, and some of the main steps towards the consolidation of the concept were taken by Boltzmann and Gibbs. Since then several extensions and reformulations have been developed in various disciplines with motivations and applications in different subjects, such as statistical mechanics, information theory, and dynamical systems. Fujii and Kam...

In information theory, systems are modeled by a transmitter, channel, and receiver. The transmitter produces messages that are sent through the channel. The channel modifies the message in some way. The receiver attempts to infer which message was sent. In this context, entropy (more specifically, Shannon entropy) is the expected value (average) of the information contained in each message. 'Me...

The general problem of inductive inference is to update from a prior probability distribution to a posterior distribution when new information becomes available. Bayes' rule is the natural way to update when the new information is in the form of data while Jaynes’ method of maximum entropy, MaxEnt, is designed to handle information in the form of constraints. However, the range of applicability...

“If only once only once no matter where, no matter before what audience I could better the record of the great Rastelli and juggle with thirteen balls, instead of my usual twelve, I would feel that I had truly accomplished something for my country. But I am not getting any younger, and although I am still at the peak of my powers there are moments why deny it? when I begin to doubt and there is...

This paper studies bivariate distributions with fixed marginals from an information-theoretic perspective. In particular, continuity and related properties of various information measures (Shannon entropy, conditional entropy, mutual information, Rényi entropy) on the set of all such distributions are investigated. The notion of minimum entropy coupling is introduced, and it is shown that it de...

Only recently have approaches to quantitative information flow started to challenge the presumption that all leaks involving a given number of bits are equally harmful. This paper proposes a framework to capture the semantics of information, making the quantification of leakage independent of the syntactic representation of secrets. Secrets are defined in terms of fields, which are combined to ...

Entropy is a useful concept that has been used to describe the structure and behavior of different systems. We summarize its multifaceted character with regard to its implications for urban sprawl, and propose a framework to apply the concept of entropy to urban sprawl for monitoring and management.

Some general considerations on the notion of entropy in physics are presented. An attempt is made to clarify the question of the differentiation between physical entropy (the Clausius-Boltzmann one) and quantities called entropies associated to Information Theory, which are in fact generating functionals for the derivation of probability distributions and not thermodynamic functions of state. T...

Information-theoretic quantities, such as entropy and mutual information (MI), can be used to quantify the amount of information needed to describe a dataset or the information shared between two datasets. In the case of a dynamical system, the behavior of the relevant variables can be tightly coupled, such that information about one variable at a given instance in time may provide information ...

Recently, Brukner and Zeilinger have presented a number of arguments suggesting that the Shannon information is not well defined as a measure of information in quantum mechanics. If established, this result would be highly significant, as the Shannon information is fundamental to the way we think about information not only in classical but also in quantum information theory. On consideration, h...