Requirements volatility is an important risk factor for software projects. Software measures can help in quantifying and predicting this risk. In this paper, we present the results of a correlational study with the goal of predicting requirements volatility for a medium size software project. Based on the data collected from two industrial software projects for four measures of size of requirem...

Learning of continuous valued functions using neural network ensembles (committees) can give improved accuracy, reliable estimation of the generalization error, and active learning. The ambiguity is defined as the variation of the output of ensemble members averaged over unlabeled data, so it quantifies the disagreement among the networks. It is discussed how to use the ambiguity in combination...

The aim of many microarray experiments is to build discriminatory diagnosis and prognosis models. Given the huge number of features and the small number of examples, model validity which refers to the precision of error estimation is a critical issue. Previous studies have addressed this issue via the deviation distribution (estimated error minus true error), in particular, the deterioration of...

For a small sample problem with a large number of features, feature selection by cross-validation frequently goes into random tie breaking because of the discrete recognition rate. This leads to inferior feature selection results. To solve this problem, we propose using a least squares support vector regressor (LS SVR), instead of an LS support vector machine (LS SVM). We consider the labels (1...

The study was conducted with the application of the model MoDrY (Model-Dry periods-Yield) for the estimation of the level of potato yields on the basis of dry periods occurring during the particular periods between the phenological phases of the crop plant. A characteristic feature of this model, unlike most existing weatheryield models, is that the principle of its operation is based only ...

We consider the mean prediction error of a classification or regression procedure as well as its cross-validation estimates, and investigate the variance of this estimate as a function of an arbitrary cross-validation design. We decompose this variance into a scalar product of coefficients and certain covariance expressions, such that the coefficients depend solely on the resampling design, and...

We give an analysis of the generalization error of cross validation in terms of two natural measures of the difficulty of the problem under consideration: the approximation rate (the accuracy to which the target function can be ideally approximated as a function of the number of hypothesis parameters), and the estimation rate (the deviation between the training and generalization errors as a fu...

A nonparametric kernel methods is proposed and evaluated performance for estimating annual maximum stream flow quantiles. The bandwidth of the estimator is estimated by using an optimal technique and a cross-validation technique. Results obtained from a limited amount of real data from Malaysia show that quantiles estimated by nonparametric method using these techniques have small root mean squ...

A method for tuning MLP learning parameters in an ensemble classifier framework is presented. No validation set or cross-validation technique is required to optimize parameters for generalisability. In this paper, the technique is applied to face recognition using Error-Correcting Output Coding strategy to solve multiclass problems.

A statistical theory for overtraining is proposed. The analysis treats general realizable stochastic neural networks, trained with Kullback-Leibler divergence in the asymptotic case of a large number of training examples. It is shown that the asymptotic gain in the generalization error is small if we perform early stopping, even if we have access to the optimal stopping time. Based on the cross...