A learning machine--or a model--is usually trained by minimizing a given criterion (the expectation of the cost function), measuring the discrepancy between the model output and the desired output. As is already well known, the choice of the cost function has a profound impact on the probabilistic interpretation of the output of the model, after training. In this work, we use the calculus of va...

Pedomodels have become a popular topic in soil science and environmentalresearch. They are predictive functions of certain soil properties based on other easily orcheaply measured properties. The common method for fitting pedomodels is to use classicalregression analysis, based on the assumptions of data crispness and deterministic relationsamong variables. In modeling natural systems such as s...

During the last decades, the use of information theoretic criteria (ITC) for selecting the order of autoregressive (AR) models has increased constantly. Because the ITC are derived under the strong assumption that the measured signals are stationary, it is not straightforward to employ them in combination with the forgetting factor least-squares algorithms. In the previous literature, the attem...

We address the problem of joint Schur decomposition (JSD) of several matrices. This problem is of great importance for many signal processing applications such as sonar, biomedicine, and mobile communications. We rst present a least-squares (LS) approach for computing the JSD. The LS approach is shown to coincide with that proposed intuitively by Haardt et al, thus establishing the optimality o...

In this paper, we treat a design problem for IIR digital filters described by rational transfer function in discrete space. First, we form the filter design problem using the modified least-squares (MLS) criterion and express it as the quadratic form with respect to the numerator and denominator coefficients. Next, we show the relaxation method using the Lagrange multiplier method in order to s...

A novel approach for signal parameter estimation, named the Non-Linear Instantaneous Least Squares (NILS) estimator, is proposed and a high SNR statistical analysis of the estimates is presented. The algorithm is generally applicable to deterministic signal in noise models. However, it is of particular interest in applications where the “conventional” non-linear least squares criterion suffers ...

Feature selection is an essential problem in many fields such as computer vision. In this paper we introduce a supervised feature selection criterion based on Partial Least Squares regression (PLS). We find an optimal feature subset by applying the theory of Optimal Experiment Design to optimize the eigenvalues of the loadings matrix obtained from PLS. Since PLS extracts components such that th...

This paper firstly presents an extended ambiguity resolution model that deals with an ill-posed problem and constraints between the estimated parameters. In the extended model, the regularization criterion is used instead of the traditional least squares in order to estimate the float ambiguities better. The existing models can be derived from the general model. Secondly, the paper examines the...

Sparse representations techniques have become an active domain of research in signal processing with numerous applications in compression and coding, for instance. They are mostly based on a combined `2 − `1 criterion, where the least-squares-part ensures closeness to the observations and the `1-part sparsity. We replace the least-square-part by a `∞-part and investigate the recovery conditions...