Review of 'Information Theory' (Ash, Robert; 1965)

Professor Ash's remark in the Preface that this book was written ". .. in a style suitable for first-year graduate students in mathematics and the physical sciences.. s " should not be taken too seriously. The material is indeed largely self-contained, but the pace and depth of treatment are such that a heavy dose of mathematical maturity is required of the reader. It must be added that this is a very scholarly treatise which will generously reward the reader for his time spent in its mastery. Theorems are precisely stated and rigorously proved (except for a few minor lapses such as on page 112, where it is erroneously argued that pairwise independence of components implies pairwise independence of the entire code words), and historical credits are carefully traced. It is striking evidence to the genius of Claude Shannon that the broad coverage provided by the eight chapters of this book fails to exhaust the concepts introduced in Shannon's 1948 paper; e.g., there is no mention of Shannon's important concept of the rate-distortion function for an information source. In Chapter One, " A Measure of Information, " Ash derives the uncertainty function from a set of intuitive axioms and gives several interpretations of H(X). The interpretation in terms of typical sequences is particularly well done. In Chapter Two, " Noiseless Coding, " Ash treats the main features of the noiseless coding theorem including Huffman's construction for optimal codes. This chapter is complicated by Ash's formidable proof of the validity of an algorith-mic test for unique decipherability. This material could have been relegated to an appendix (or omitted altogether) with little loss, since a major conclusion of the chapter is that the conditions for unique decipherability coincide with those for the existence of instantaneous (or prefix) codes. Thus codes that are uniquely de-cipherable but noninstantaneous would seem to be of little theoretical or practical interest. Chapter Three, " The Discrete Memoryless Channel, " is an elegant presentation of the noisy coding theorem for the DMC. Ash isolates the important aspects in his " Fundamental Lemma, " which lies at the root of all the coding theorems proved in this book. As has become standard, the coding theorems in this and the later chapters are all stated in terms of word error probabilities for block codes. (Tree codes are nowhere considered in this book-a significant omission, since Ash co-authored one of the fundamental …