SMML estimators for linear regression and tessellations of hyperbolic space

The strict minimum message length (SMML) principle links data compression with inductive inference. The corresponding estimators have many useful properties but they can be hard to calculate. We investigate SMML estimators for linear regression models and we show that they have close connections to hyperbolic geometry. When equipped with the Fisher information metric, the linear regression model with p covariates and a sample size of n becomes a Riemannian manifold, and we show that this is isometric to (p+1)-dimensional hyperbolic space H equipped with a metric tensor which is 2n times the usual metric tensor on H. A natural identification then allows us to also view the set of sufficient statistics for the linear regression model as a hyperbolic space. We show that the partition of an SMML estimator corresponds to a tessellation of this hyperbolic space. 1 The linear regression model To establish our notation we briefly recall some details of the linear regression model. The linear regression model is a statistical model for observed data y ∈ R (thought of as a column matrix) which is a realization of an n-dimensional, normally-distributed random variable Y with mean Aβ and variance-covariance matrix σIn, i.e., Y ∼ Nn(Aβ, σIn), where A is a full-rank n× p matrix called the design matrix, β ∈ R is a column matrix, σ > 0 and In is the n × n identity matrix. Here β and σ are unknown and are to be estimated in terms of y and A. In this paper, we will always require p ≤ n though for certain results (indicated in the text) we will also require p < n. The probability density function (PDF) of Y given values of the unknown model parameters β and σ is therefore (2πσ) exp ( −‖y − Aβ‖ 2 2σ2 )