How People Estimate Effect Sizes: The Role of Means and Standard Deviations

Many studies of causal judgments have dealt with the relation between the presence and the absence of a cause and an effect. However, little is known about causal learning with a continuous outcome. The present study adopted Cohen’s d as an objective standard for effect size in situations where a binary cause influenced a continuous effect and investigated how people use means and standard deviations in the estimation of effect sizes. The experimental task was to read a scenario where the performance of two groups was compared and to infer the causal effect. Whereas means were manipulated while holding standard deviations constant in the mean difference group, standard deviations were varied with holding means constant in the standard deviation difference group. The results demonstrate that participants could respond appropriately to the difference in two means, and that they gave a higher estimate of effect size in large standard deviation situations than in small standard deviation situations. Judgments about standard deviations are in contrast to Cohen’s d, indicating disproportionate attention to different kinds of data samples.