Cross-validation is an intuitive and effective technique for model selection in data analysis. In this discussion, I try to present a few incarnations of the general technique in a few nonparametric function estimation settings. Justifications of the technique in Gaussian regression settings will be discussed, along with possible reasons for the lack of similar justification in other settings. ...

The leave-one-out cross-validation scheme for generalization assessment of neural network models is computationally expensive due to replicated training sessions. In this paper we suggest linear unlearning of examples as an approach to approximative cross-validation. Further, we discuss the possibility of exploiting the ensemble of networks o ered by leave-one-out for performing ensemble predic...

Modern steganalysis is a combination of a feature space design and a supervised binary classification. In this report, we assume that the feature space has been already constructed, i.e., the steganalyst has a set of training features and needs to train a binary classifier. Any machine learning tool can be used for this task and its parameters can be tuned through cross-validation, a standard a...

For nonparametric regression. in the case of dependent observations. cross-validation is known to be severely affected by dependence. This effect is precisely quantified through a limiting distribution for the cross-validated bandwidth. The performance of two methods. the "leave-(2e+1)-out" version of cross-validation and partitioned cross-validation. which adjust for the dependence effect on b...

quantitative relationships between molecular structure of forty eight aldehyde compounds with their known cathepsin k inhibitory effects were discovered by partial least squares (pls) method. evaluation of a test set of 10 compounds with the developed pls model revealed that this model is reliable with a good predictability. since the qsar study was performed on the basis of theoretical descrip...

Social media are becoming an increasingly important source of information about the public mood regarding issues such as elections, Brexit, stock market, etc. In this paper we focus on sentiment classification of Twitter data. Construction of sentiment classifiers is a standard text mining task, but here we address the question of how to properly evaluate them as there is no settled way to do s...

Physical inactivity is identified by the World Health Organization as the fourth leading risk factor for global mortality as it increases the risk of various adverse health conditions. App-based health interventions promise to help increase physical activity levels by enhancing the motivation of the users to exercise on a regular basis. One such app-based intervention which sends tailored coach...

Assessing the fit of a model is an important final step in any statistical analysis, but this is not straightforward when complex discrete response models are used. Cross validation and posterior predictions have been suggested as methods to aid model criticism. In this paper a comparison is made between four methods of model predictive assessment in the context of a three level logistic regres...

Modeling with flexible models, such as neural networks, requires careful control of the model complexity and generalization ability of the resulting model. Whereas general asymptotic estimators of generalization ability have been developed over recent years (e.g., [9]), it is widely acknowledged that in most modeling scenarios there isn't sufficient data available to reliably use these estimato...