Twice Universal Linear Prediction of Individual Sequences

We present a linear prediction algorithm which is \twice universal," over parameters and model orders, for individual sequences under the square-error loss function. The sequentially accumulated mean-square prediction error is as good as any linear predictor of order up to some M. Following an approach taken in many prediction problems we transform the linear prediction problem into a sequential probability assignment problem from universal coding theory. We derive an upper bound on the excess prediction error which can be identiied with the excess coding redundancy in the assigned universal probability. The bound holds for all individual sequences of all lengths, not only for asymptotically long sequences. The two terms in the bound correspond to a parameter redundancy term, which is proportional to (p=2)n ?1 ln(n), and a model order redundancy term which is proportional to n ?1 ln(p), where n is the data length, and p is the best model order.