This is an introduction to Shannon’s Information Theory. It is more like a long note so that it is by no means a complete survey or completely mathematically rigorous. It covers two main topics: entropy and channel capacity. All related concepts will be developed in a totally combinatorial flavor. Some issues usually not addressed in the literature will be discussed here as well. Hopefully, it ...

The rapid development of communication systems with mobile receivers at higher data rates has lead to the importance of studies on information transfer over highly time varying channels. Under such circumstances, the channel variations become fast and the receiver is unable to track the channel during the predefined block length. Here existing results for the channel capacity and the optimal in...

The DNA sequencing efforts of the past years together with rapid progress in sequencing technology have generated a huge amount of sequence data available in public molecular databases. This recent development makes it statistically feasible to apply universal concepts from Shannon’s information theory to problems in molecular biology, e.g to use mutual information for gene mapping and phylogen...

The most important communication signal is human speech. It is helpful to think of speech communication in terms of Claude Shannon’s information theory channel model. When thus viewed, it immediately becomes clear that the most complex part of speech communication channel is in auditory system (the receiver). In my opinion, even after years of work, relatively little is know about how the human...

OF THE DISSERTATION Information theory methods in communication complexity by Nikolaos Leonardos Dissertation Director: Michael Saks This dissertation is concerned with the application of notions and methods from the field of information theory to the field of communication complexity. It consists of two main parts. In the first part of the dissertation, we prove lower bounds on the randomized ...

3 The randomized communication complexity of Disjointness 8 3.1 High level idea of the proof reduction to internal information of AND . . . 8 3.2 The Internal Information of AND . . . . . . . . . . . . . . . . . . . . . . . . 8 3.2.1 Divergence and Statistical Distance . . . . . . . . . . . . . . . . . . . 8 3.2.2 High level idea of the proof . . . . . . . . . . . . . . . . . . . . . . . 10 3.2...

Optical communication channels are ultimately quantum-mechanical in nature, and we must therefore look beyond classical information theory to determine their communication capacity as well as to find efficient encoding and decoding schemes of the highest rates. Thermal channels, which arise from linear coupling of the field to a thermal environment, are of particular practical relevance; their ...

Human communication is based on a dynamic information exchange of the communication channels facial expressions, prosody, and speech content. This fMRI study elucidated the impact of multimodal emotion processing and the specific contribution of each channel on behavioral empathy and its prerequisites. Ninety-six video clips displaying actors who told self-related stories were presented to 27 h...

Two Multi-Terminal Communication Problems: Distributed Estimation and Source-Channel Broadcast by Vinod Malathi Prabhakaran Doctor of Philosophy in Engineering-Electrical Engineering and Computer Sciences Designated Emphasis in Communication, Computation, and Statistics University of California, Berkeley Professors Kannan Ramchandran and David Tse, Co-chairs A major success of information theor...