Lesson 2: Type I and Type II errors, statistical vs. practical significance, correlation ≠ causation

There are two possible errors in hypothesis testing. A Type I error (false alarm): rejecting H0 when it is actually true. Its probability is α. For example, a drug doesn't work, but the test says “it works.” A Type II error (missed effect): failing to reject H0 when it is false. Its probability is denoted β. For example, a drug really helps, but the test misses the effect. Lowering α cuts the risk of a Type I error but raises the risk of a Type II error — a trade-off you can't avoid. Statistical significance (p < α) does not mean practical importance. With a very large sample (n = 100,000), a test can detect a difference of 0.001 units — statistically significant, but meaningless in practice. That's why it's important to look at the effect size (for example, how big the difference is in absolute terms), not just the p-value. Correlation measures the linear relationship between two variables: r ∈ [−1, 1]. But correlation does not prove causation. Three other explanations: (1) a common cause — both variables are driven by a third variable; (2) reverse causation — the relationship runs the other way; (3) coincidence. A classic example: ice cream sales and drownings are correlated — both rise in summer, but ice cream doesn't cause drowning. To establish causation, you need a randomized experiment, not observation.