What is information? What role does information entropy play in this information exploding age, especially in understanding emergent behaviors of complex systems? To answer these questions, we discuss the origin of information entropy, the difference between information entropy and thermodynamic entropy, the role of information entropy in complexity theories, including chaos theory and fractal ...

Developing a lesson plan (or syllabus) for a course can be a very time consuming process. Once the instructor has a pool of topics to be taught in the course, the organization of those topics in a form that facilitates understandability by the students as well as efficiency with respect to the order of presentation can be an elusive task – many times considered to be an art that distinguishes g...

We give a survey of the basic statistical ideas underlying the definition of entropy in information theory and their connections with the entropy in the theory of dynamical systems and in statistical mechanics.

For Dr. Martin Harwit, whose course on information theory rst exposed me to the concept of entropy. 1. Introduction In this paper, I study \entropy" as a quantity deened axiomatically by certain formal properties, independently of any particular mathematical context. In particular , unlike all previous attempts to axiomatize information theory which are known to me, the concept of probability w...

This is an introduction to Shannon’s Information Theory. It is more like a long note so that it is by no means a complete survey or completely mathematically rigorous. It covers two main topics: entropy and channel capacity. All related concepts will be developed in a totally combinatorial flavor. Some issues usually not addressed in the literature will be discussed here as well. Hopefully, it ...

This article proposes a quantitative risk assessment for security systems which have multiple protected assets and a risk-based benefit-cost analysis for decision makers. The proposed methodology consists of five phases: identification of assets, security unit and intrusion path, security unit effectiveness estimation, intrusion path effectiveness estimation, security system risk assessment and...

Only recently have approaches to quantitative information flow started to challenge the presumption that all leaks involving a given number of bits are equally harmful. This paper proposes a framework to capture the semantics of information, making the quantification of leakage independent of the syntactic representation of secrets. Secrets are defined in terms of fields, which are combined to ...

Recently, Brukner and Zeilinger have presented a number of arguments suggesting that the Shannon information is not well defined as a measure of information in quantum mechanics. If established, this result would be highly significant, as the Shannon information is fundamental to the way we think about information not only in classical but also in quantum information theory. On consideration, h...