← Statistics and Probability from Scratch
Lesson
Lesson 3: Capstone case — drawing a sound statistical conclusion
Draw a sound statistical conclusion about a real-world situation using concepts from the whole course, and check yourself against a checklist.
How to read study results critically
A checklist for critical statistical conclusions
Real studies are rarely perfect. To read a result correctly, check the following in order. First, study design: a randomized experiment or an observational study? Only an experiment lets you talk about causation. Second, the sample: is it large enough? Any bias (e.g., volunteers vs. a random sample)? Third, hypotheses and the p-value: were H0 and H1 stated before data collection? Is the p-value read correctly (as the probability of the data given a true H0, not of the hypothesis)?
Fourth, the confidence interval and effect size: how wide is the CI? Does it include practically insignificant values? A narrow CI + a large sample size often yield a “significant” result for a tiny effect. Fifth, practical significance: even with p < 0.05, the effect may be too small to matter in practice. Sixth, correlation vs. causation: if it's an observational study, you can't claim that A causes B.
Working through this checklist protects you from the most common traps: a false conclusion about causation, overrating the “significance” of a small effect, and misreading a p-value or a confidence interval.
Lesson notes
A checklist for critical statistical conclusions
Real studies are rarely perfect. To read a result correctly, check the following in order. First, study design: a randomized experiment or an observational study? Only an experiment lets you talk about causation. Second, the sample: is it large enough? Any bias (e.g., volunteers vs. a random sample)? Third, hypotheses and the p-value: were H0 and H1 stated before data collection? Is the p-value read correctly (as the probability of the data given a true H0, not of the hypothesis)?
Fourth, the confidence interval and effect size: how wide is the CI? Does it include practically insignificant values? A narrow CI + a large sample size often yield a “significant” result for a tiny effect. Fifth, practical significance: even with p < 0.05, the effect may be too small to matter in practice. Sixth, correlation vs. causation: if it's an observational study, you can't claim that A causes B.
Working through this checklist protects you from the most common traps: a false conclusion about causation, overrating the “significance” of a small effect, and misreading a p-value or a confidence interval.