Statistical and Inductive Inference by Minimum Message Length (2005)

Since 1965, Prof. Wallace and others have been developing an approach tostatistical estimation, hypothesis testing, model selection and their applications in the Artificial Intelligence field of Machine Learning. The approach is based on Information Theory, using concepts from classical Shannon theory and more recent work on Algorithmic Complexity. The new approach has come to be called the Minimum Message Length principle, since it is based on the idea of constructing a message which concisely encodes the available data. Although a range of journal and conference papers has been published on the principle and its application, and several computer programs applying it have been shown to perform well and have been fairly widely used, there is no text providing a thorough treatment of the principle or giving general guidance for its application.