We consider the mean prediction error of a classification or regression procedure as well as its cross-validation estimates, and investigate the variance of this estimate as a function of an arbitrary cross-validation design. We decompose this variance into a scalar product of coefficients and certain covariance expressions, such that the coefficients depend solely on the resampling design, and...

Argument mining systems for student essays need to be able to reliably identify argument components independently of particular essay topics. Thus in addition to features that model argumentation through topic-independent linguistic indicators such as discourse markers, features that can abstract over lexical signals of particular essay topics might also be helpful to improve performance. Prior...

In wavelet regression, choosing threshold value is a crucial issue. A too large value cuts too many coefficients resulting in over smoothing. Conversely, a too small threshold value allows many coefficients to be included in reconstruction, giving a wiggly estimate which result in under smoothing. However, the proper choice of threshold can be considered as a careful balance of these principles...

Data that have not been modeled cannot be correctly predicted. Under this assumption, research studies how k-fold cross-validation can introduce dataset shift in regression problems. This fact implies data distributions the training and test sets to different and, therefore, a deterioration of model performance estimation. Even though stratification output variable is widely used field classifi...

OBJECTIVE To compare the performance of two predictive radiologic models, logistic regression (LR) and neural network (NN), with five different resampling methods. METHODS One hundred sixty-seven patients with proven calvarial lesions as the only known disease were enrolled. Clinical and CT data were used for LR and NN models. Both models were developed with cross-validation, leave-one-out, a...

Breast cancer is the second most common after lung and one of main causes death worldwide. Women have a higher risk breast as compared to men. Thus, early diagnosis with an accurate reliable system critical in treatment. Machine learning techniques are well known popular among researchers, especially for classification prediction. An investigation was conducted evaluate performance malignant tu...

BACKGROUND We consider both univariate- and multivariate-based feature selection for the problem of binary classification with microarray data. The idea is to determine whether the more sophisticated multivariate approach leads to better misclassification error rates because of the potential to consider jointly significant subsets of genes (but without overfitting the data). METHODS We presen...

We used learning-to-rank methods for training ranking model. Due to the limited number of training queries, we split them and conducted 5-fold cross validation. We set the proportion of training and testing as 4 : 6. For the features used in learning the model, a total of 231 features that are from multiple information retrieval models with different parameter settings were adopted. For the bas...