Machine learning techniques are becoming increasingly useful primarily with the rapid development of Internet. A variety of machine learning methods have drawn inspirations or borrowed ideas from information theory. In this paper, we present a survey of such interactions between machine learning and information theory. Four important areas of machine learning are examined from the perspective o...