Recent approaches to collaborative filtering have concentrated on estimating an algebraic or statistical model, and using the model for predicting missing ratings. In this paper we observe that different models have relative advantages in different regions of the input space. This motivates our approach of using stagewise linear combinations of collaborative filtering algorithms, with non-const...

Many statistical machine learning algorithms minimize either an empirical loss function as in AdaBoost, or a penalized empirical loss as in Lasso or SVM. A single regularization tuning parameter controls the trade-off between fidelity to the data and generalizability, or equivalently between bias and variance. When this tuning parameter changes, a regularization “path” of solutions to the minim...

Ordinal classification problems with monotonicity constraints (also referred to as multicriteria classification problems) often appear in real-life applications, however, they are considered relatively less frequently in theoretical studies than regular classification problems. We introduce a rule induction algorithm based on the statistical learning approach that is tailored for this type of p...

We develop an algorithm to learn Bernoulli Mixture Models based on the principle that some variables are more informative than others. Working from an information-theoretic perspective, we propose both backward and forward schemes for selecting the informative ’active’ variables and using them to guide EM. The result is a stagewise EM algorithm, analogous to stagewise approaches to linear regre...

Lasso is a regularization method for parameter estimation in linear models. It optimizes the model parameters with respect to a loss function subject to model complexities. This paper explores the use of lasso for statistical language modeling for text input. Owing to the very large number of parameters, directly optimizing the penalized lasso loss function is impossible. Therefore, we investig...

Most of the existing simultaneous approaches for heat exchanger network synthesis assume that only one type of hot/cold utility is available to adjust the final temperatures. In this work, we propose a simultaneous mixed-integer nonlinear programming formulation based on a more generalized stagewise superstructure. We allow utilities to adjust the temperature of process streams in each stage. W...

Discrete-time optimal control (DTOC) problems are large-scale optimization problems with a dynamic structure. In previous work this structure has been exploited to provide very fast and eecient local procedures. Two examples are the diierential dynamic programming algorithm (DDP) and the stagewise Newton procedure { both require only O(N) operations per iteration, where N is the number of times...