Information Theory, Predictability and Disequilibrium. Lecture 11: An information theoretic approach to statistical disequilibrium 1 Introduction

In the last two lectures we have seen that the relative entropy between a prediction density and an equilibrium density is a natural measure of the predictability of a statistical system. Furthermore in Lecture 5 we have seen that in a continuous Markov random process governed by the Fokker Planck equation (FPE), the relative entropy between an arbitrary density and the equilibrium density declines monotonically and can therefore be used as a measure of the disequilibrium of such systems. This close relationship between disequilibrium, relative entropy and predictability suggests that non-equilibrium statistical physics is a subject worthy of very close attention. Unfortunately it is also a subject which is far from being in a final state as a complete physical theory. Many different approaches have been attempted with varying degrees of elegance and success. This situation is in stark contrast to the power, elegance and completeness of equilibrium statistical mechanics. A unifying principle in the latter area derives from information theory due to the maximum entropy principle identified by Jaynes. This suggests that a similar phisiophical approach may be productive for the non-equilibrium case. In this lecture we present a recent such effort by the instructor and co-workers.