Enhanced Sparsity by Non-Separable Regularization

Regularization With Non-convex Separable Constraints

We consider regularization of nonlinear ill-posed problems with constraints which are non-convex. As a special case we consider separable constraints, i.e. the regularization takes place in a sequence space and the constraint acts on each sequence element with a possibly non-convex function. We derive conditions under which such a constraint provides a regularization. Moreover, we derive estima...

Sparsity Reconstruction by the Standard Tikhonov Regularization

It is a common belief that Tikhonov scheme with ‖ · ‖L2 -penalty fails to reconstruct a sparse structure with respect to a given system {φi}. However, in this paper we present a procedure for sparsity reconstruction, which is totally based on the standard Tikhonov method. This procedure consists of two steps. At first Tikhonov scheme is used as a sieve to find the coefficients near φi, which ar...

Nonparametric sparsity and regularization

In this work we are interested in the problems of supervised learning and variable selection when the input-output dependence is described by a nonlinear function depending on a few variables. Our goal is to consider a sparse nonparametric model, hence avoiding linear or additive models. The key idea is to measure the importance of each variable in the model by making use of partial derivatives...

Sparsity Based Regularization

In previous lectures, we saw how regularization can be used to restore the well-posedness of the empirical risk minimization (ERM) problem. We also derived algorithms that use regularization to impose smoothness assumptions on the solution space (as in the case of Tikhonov regularization) or introduce additional structure by confining the solution space to low dimensional manifolds (manifold re...

Supplementary Material: Non-Convex Rank/Sparsity Regularization and Local Minima