Tests of Statistical Significance Made Sound

Tests of Statistical Significance Made Sound

This article considers the nature and place of tests of statistical significance (ToSS) in science, with particular reference to psychology. Despite the enormous amount of attention given to this topic, psychology’s understanding of ToSS remains deficient. The major problem stems from a widespread and uncritical acceptance of null hypothesis significance testing (NHST), which is an indefensible...

Statistical significance tests.

Statistical methods for the analysis of data can be classed as descriptive or as probabilistic. The former methods include tabulation and graphical display, of great importance both in detailed analysis and also in presentation of conclusions. Probability enters statistical analysis in various ways. The most obvious is that one recognizes explicitly that conclusions drawn from limited data are ...

Statistical Significance Tests for Machine Translation Evaluation

If two translation systems differ differ in performance on a test set, can we trust that this indicates a difference in true system quality? To answer this question, we describe bootstrap resampling methods to compute statistical significance of test results, and validate them on the concrete example of the BLEU score. Even for small test sizes of only 300 sentences, our methods may give us ass...

Statistical Tests, Tests of Significance, and Tests of a Hypothesis Using Excel

More accurate tests for the statistical significance of result differences

Appears in the 18th International Conference on Computational Linguistics (COLING 2000), pages 947-953, Saarbrücken, Germany, July, 2000. cs.CL/0008005 Statistical significance testing of differences in values of metrics like recall, precision and balanced F-score is a necessary part of empirical natural language processing. Unfortunately, we find in a set of experiments that many commonly used...