Regularization is one of the crucial ingredients of deep learning, yet the term regularization has various definitions, and regularization methods are often studied separately from each other. In our work we present a systematic, unifying taxonomy to categorize existing methods. We distinguish methods that affect data, network architectures, error terms, regularization terms, and optimization p...

In this paper we derive a generalizing concept of G-norms, which we call G-values, which is used to characterize minimizers of nondifferentiable regularization functionals. Moreover, the concept is closely related to the definition of slopes as published in a recent book by Ambrosio, Gigli, Savaré. A paradigm of regularization models fitting in this framework is robust bounded variation regular...

We consider the local regularization problem for integral equations of the first kind, generalizing previous work which applied only to problems of Volterra type. Our approach allows for local control of the regularization process, allowing for resolution of fine/sharp features of solutions without having to resort to nondifferentiable optimization techniques. In addition we present examples il...

To learn reliable rules that can generalize to novel situations, the brain must be capable of imposing some form of regularization. Here we suggest, through theoretical and computational arguments, that the combination of noise with synchronization provides a plausible mechanism for regularization in the nervous system. The functional role of regularization is considered in a general context in...

Intrigued by some recent results on impulse response estimation by kernel and nonparametric techniques, we revisit the old problem of transfer function estimation from input-output measurements. We formulate a classical regularization approach, focused on finite impulse response (FIR) models, and find that regularization is necessary to cope with the high variance problem. This basic, regulariz...

In recent studies on sparse modeling, the nonconvex regularization approaches (particularly, Lq regularization with q ∈ (0, 1)) have been demonstrated to possess capability of gaining much benefit in sparsity-inducing and efficiency. As compared with the convex regularization approaches (say, L1 regularization), however, the convergence issue of the corresponding algorithms are more difficult t...

In this paper, we investigate sparsity regularization for electrical impedance tomography (EIT). Here, we combine sparsity regularization with the energy functional approach. The main results of our paper is the well-posedness and convergence rates of the sparsity regularization method.

A regularization for effective field theory with two propagating heavy particles is constructed. This regularization preserves the low-energy analytic structure, implements a low-energy power counting for the one-loop diagrams, and preserves symmetries respected by dimensional regularization.

In this paper we show that regularization methods form a scale-space where the regularization parameter serves as scale. In analogy to nonlinear diiusion ltering we establish continuity with respect to scale, causality in terms of a maximum{minimum principle, simpliica-tion properties by means of Lyapunov functionals and convergence to a constant steady-state. We identify nonlinear regularizati...