| The leave-one-out cross-validation scheme for generalization assessment of neu-ral network models is computationally expensive due to replicated training sessions. Linear unlearning of examples has recently been suggested as an approach to approximative cross-validation. Here we brieey review the linear unlearning scheme, dubbed LULOO, and we illustrate it on a system identiication example. F...

Cross-validation has long been used for choosing tuning parameters and other model selection tasks. It generally performs well provided the data are independent, or nearly so. Improvements have been suggested which address ordinary cross-validation’s (OCV) shortcomings in correlated data. Whereas these techniques have merit, they can still lead to poor model selection in correlated data or are ...

Integrated Mean Squared Error (IMSE) is a version of the usual mean squared error criterion, averaged over all possible training sets of a given size. If it could be observed, it could be used to determine optimal network complexity or optimal data subsets for efficient training. We show that two common methods of cross-validating average squared error deliver unbiased estimates of IMSE, conver...

This document is the final report describing the activities carried by the Cross Issue Validation (CIV) in the framework of the second phase of Wireless World Initiative (WWI) projects. The Cross Issue Validation is a cross-project working group, dedicated to demos and trials of WWI projects (Ambient Networks, WINNER, E2R, SPICE, MOBILIFE). The CIV main goal is to bring a general and synthetic ...

Cross-validation is a frequently used, intuitively pleasing technique for estimating the accuracy of theories learned by machine learning algorithms. During testing of a machine learning algorithm (foil) on new databases of prokaryotic RNA transcription promoters which we have developed, cross-validation displayed an interesting phenomenon. One theory is found repeatedly and is responsible for ...

In order to choose correctly the dimension of calibration model in chemistry, a new simple and effective method named Ž . Monte Carlo cross validation MCCV is introduced in the present work. Unlike leave-one-out procedure commonly used in Ž . chemometrics for cross validation CV , the Monte Carlo cross validation developed in this paper is an asymptotically consistent method in determining the ...

Cross-validation is an essential tool in machine learning and statistics. The typical procedure, in which data points are randomly assigned to one of the test sets, makes an implicit assumption that the data are exchangeable. A common case in which this does not hold is when the data come from multiple sources, in the sense used in transfer learning. In this case it is common to arrange the cro...

Partitioned cross-validation is proposed as a method for overcoming the large amounts of across sample variability to which ordinary cross-validation is subject. The price for cutting down on the sample noise is that a type of bias is introduced. A theory is presented for optimal trade-off of this variance and bias. Comparison with other bandwidth selection methods is given.

There are three major strategies to form neural network ensembles. The simplest one is the Cross Validation strategy in which all members are trained with the same training data. Bagging and boosting strategies pro-duce perturbed sample from training data. This paper provides an ideal model based on two important factors: activation function and number of neurons in the hidden layer and based u...

Cross-validation was originally invented to estimate the prediction error of a mathematical modelling procedure. It can be shown that cross-validation estimates the prediction error almost unbiasedly. Nonetheless, there are numerous reports in the chemoinformatic literature that cross-validated figures of merit cannot be trusted and that a so-called external test set has to be used to estimate ...