Reading Survey Reports Critically
Analyse reports of surveys in digital media and elsewhere for information on how data was obtained to estimate population means and medians. Analyse how different sampling methods can affect the results of surveys and how choice of representation can be used to support a particular point of view.
A reported average is a claim about a whole population made from a sample. How that sample was chosen decides whether to believe it.
Method
Ask how the data was obtained
Method first, number second.
Sample statistics estimate population values
An estimate, never a fact.
Sampling method decides the bias
Convenience and self-selected samples skew predictably.
Representation can push a view
Truncated axes, chosen scales, cherry-picked time ranges.
Mean or median changes the story
Skewed data makes the mean far higher than the median.
Worked example
e.g.
- Ask which average was used.
- A mean is dragged upward by a few very high incomes.
- Ask how the sample was chosen.
- Ask how many people were in it.
- Ask who published it and why.
- Conclude: the median would describe a typical resident better.
Practice
Name the weakness, then say which direction it pushes the result.
1
Answer
2
Answer
3
Answer
4
Answer
5
Answer
6
Answer