Averages of proper scoring rules are often used to rank probabilistic forecasts. In many cases, the individual terms in these averages based on observations and forecasts from different distributions. We show that some most popular rules, such as continuous ranked probability score (CRPS), give more importance with large uncertainty, which can lead unintuitive rankings. To describe this issue, ...