We propose a Bayesian framework for learning the optimal regularization parameter in the L1-norm penalized least-mean-square (LMS) problem, also known as LASSO [1] or basis pursuit [2]. The setting of the regularization parameter is critical for deriving a correct solution. In most existing methods, the scalar regularization parameter is often determined in a heuristic manner; in contrast, our ...

Regularization techniques have attracted many researches in the past decades. Most focus on designing the regularization term, and few on the optimal regularization parameter selection, especially for faulty neural networks. As is known that in the real world, the node faults often inevitably take place, which would lead to many faulty network patterns. If employing the conventional method, i.e...

Total Variation (TV) is an effective method of removing noise in digital image processing while preserving edges [23]. The choice of scaling or regularization parameter in the TV process defines the amount of denoising, with value of zero giving a result equivalent to the input signal. Here we explore three algorithms for specifying this parameter based on the statistics of the signal in the to...

Ill posed problems constitute the mathematical model of a large variety of applications. Aim of this paper is to define an iterative algorithm finding the solution of a regularization problem. The method minimizes a function constituted by a least squares term and a generally nonlinear regularization term, weighted by a regularization parameter. The proposed method computes a sequence of iterat...

This article discusses the problem of choosing a regularization parameter in the group Lasso proposed by Yuan and Lin (2006), an l1-regularization approach for producing a block-wise sparse model that has been attracted a lot of interests in statistics, machine learning, and data mining. It is important to choose an appropriate regularization parameter from a set of candidate values, because it...

We propose two new affine projection algorithms (APA) with variable regularization parameter. The proposed algorithms dynamically update the regularization parameter that is fixed in the conventional regularized APA (R-APA) using a gradient descent based approach. By introducing the normalized gradient, the proposed algorithms give birth to an efficient and a robust update scheme for the regula...

Straightforward solution of discrete ill-posed linear systems of equations or leastsquares problems with error-contaminated data does not, in general, give meaningful results, because propagated error destroys the computed solution. The problems have to be modified to reduce their sensitivity to the error in the data. The amount of modification is determined by a regularization parameter. It ca...