This paper deals with the problem of multi-class classification in machine learning. Various techniques have been successfully proposed to solve such problems, with a computation cost often much higher than techniques dedicated to binary classification. To address this problem, we propose a novel formulation for designing multi-class classifiers, with essentially the same computational complexi...

Linear discriminant analysis (LDA) is a popular dimensionality reduction and classification method that simultaneously maximizes between-class scatter and minimizes within-class scatter. In this paper, we verify the equivalence of LDA and least squares (LS) with a set of dependent variable matrices. The equivalence is in the sense that the LDA solution matrix and the LS solution matrix have the...

The convergence properties of adaptive least squares (LS) and stochastic gradient (SG) algorithms are studied in the context of echo cancellation of voiceband data signals. The algorithms considered are the SG transversal, SG lattice, LS transversal (fast Kalman), and LS lattice. It is shown that for the channel estimation problem considered here, LS algorithms converge in approximately 2N iter...

This paper presents the application of least squares support vector machines (LS-SVMs) for automatic detection of alterations in the human electroencephalogram (EEG) activities during hypopnoea episodes. The obstructive sleep apnoea hypopnoea syndrome (OSAH) means ‘‘cessation of breath” during the sleep hours and the sufferers often experience related changes in the electrical activity of the b...

In least squares support vector machines (LS-SVMs), the optimal separating hyperplane is obtained by solving a set of linear equations instead of solving a quadratic programming problem. But since SVMs and LS-SVMs are formulated for two-class problems, unclassifiable regions exist when they are extended to multiclass problems. In this paper, we discuss fuzzy LS-SVMs that resolve unclassifiable ...

For a small sample problem with a large number of features, feature selection by cross-validation frequently goes into random tie breaking because of the discrete recognition rate. This leads to inferior feature selection results. To solve this problem, we propose using a least squares support vector regressor (LS SVR), instead of an LS support vector machine (LS SVM). We consider the labels (1...

This paper considers the estimation methods for dynamic panel data (DPD) models with fixed effects, which suggested in econometric literature, such as least squares (LS) and generalized method of moments (GMM). These methods obtain biased estimators for DPD models. The LS estimator is inconsistent when the time dimension (T) is short regardless of the cross-sectional dimension (N). Although con...

A square-root-free matrix QR decomposition (QRD) scheme was rederived in [1] based on [2] to simplify computations when solving least-squares (LS) problems on embedded systems. The scheme of [1] aims at eliminating both the square-root and division operations in the QRD normalization and backward substitution steps in the LS computations. It is claimed in [1] that the LS solution only requires ...

Estimation theory has been developed over centuries. There are several approaches to utilizing this theory; in this chapter, these approaches are classified into three types. Type I includes the oldest two methods, the least squares (LS) and moment methods; both of these methods are non-optimal estimators. The least squares method was introduced by Carl Friedrich Gauss. Least squares problems f...

The impact of reliable estimation of stream flows at highly urbanized areas and the associated receiving waters is very important for water resources analysis and design. We used the least squares support vector machine (LS-SVM) based algorithm to forecast the future streamflow discharge. A Gaussian Radial Basis Function (RBF) kernel framework was built on the data set to optimize the tuning pa...