Improved variable selection with Forward-Lasso adaptive shrinkage
نویسندگان
چکیده
منابع مشابه
Improved Variable Selection with Forward - Lasso Adaptive Shrinkage
Recently, considerable interest has focused on variable selection methods in regression situations where the number of predictors, p, is large relative to the number of observations, n. Two commonly applied variable selection approaches are the Lasso, which computes highly shrunk regression coefficients, and Forward Selection, which uses no shrinkage. We propose a new approach, “Forward-Lasso A...
متن کاملForward-LASSO with Adaptive Shrinkage
Both classical Forward Selection and the more modern Lasso provide computationally feasible methods for performing variable selection in high dimensional regression problems involving many predictors. We note that although the Lasso is the solution to an optimization problem while Forward Selection is purely algorithmic, the two methods turn out to operate in surprisingly similar fashions. Our ...
متن کاملSupplementary material for Forward-LASSO with Adaptive Shrinkage
Note that νj([2S−1]) = 1 and νj(ρ) is a strictly increasing function for ρ < [S−1]−1. The proof of Theorem 2 in Wainwright (2009) establishes that for each value of the tuning parameter λ the necessary condition for the signed support recovery is |νj + Z̃j| ≤ 1 with Z̃j = λ−1XT j ΠX⊥ K ( ). Note that Zj follows a non-degenerate zeromean gaussian distribution for S < n, thus the probability of the...
متن کاملBayesian Shrinkage Variable Selection
We introduce a new Bayesian approach to the variable selection problem which we term Bayesian Shrinkage Variable Selection (BSVS ). This approach is inspired by the Relevance Vector Machine (RVM ), which uses a Bayesian hierarchical linear setup to do variable selection and model estimation. RVM is typically applied in the context of kernel regression although it is also suitable in the standar...
متن کاملThe Adaptive Lasso Method for Instrumental Variable Selection
Adaptive lasso is a weighted `1 penalization method for simultaneous estimation and model selection. It has oracle properties of asymptotic normality with optimal convergence rate and model selection consistency. Instrumental variable selection has become the focus of much research in areas of application for which datasets with both strong and weak instruments are available. This paper develop...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: The Annals of Applied Statistics
سال: 2011
ISSN: 1932-6157
DOI: 10.1214/10-aoas375