Statistics (university) · Part 06
Hypothesis testing
Decide whether data gives real evidence or just chance, using p-values and standard tests.
Last updated
- Null and alternative hypothesesThe null hypothesis H0 is the default claim of no effect or no difference. The alternative H1 is what you suspect instead. A test asks whether the data is too surprising to fit H0.
- p-valueThe p-value is the probability of getting data at least as extreme as yours if the null hypothesis were true. A small p-value means the data would be surprising under H0.
- Significance level and decisionThe significance level alpha is the cut-off you set before the test, often 0.05. If the p-value is at most alpha, you reject H0. Otherwise you fail to reject it, which is not the same as proving H0 true.
- Type I and Type II errorsA Type I error is rejecting a true null (a false alarm). Its probability is alpha. A Type II error is failing to reject a false null (a missed effect). Its probability is beta.
- Power of a testPower is the probability of rejecting the null when a real effect exists: power = 1 - beta. Power rises with a larger sample, a bigger true effect and a larger alpha.
- One-sample t testA one-sample t test compares a sample mean with a claimed value when sigma is unknown. The statistic is t = (sample mean - claimed mean) / (s / square root of n), with n - 1 degrees of freedom.
- Two-sample t testA two-sample t test compares the means of two independent groups. The statistic is t = (mean1 - mean2) / square root of (s1 squared / n1 + s2 squared / n2). Welch's version does not assume equal variances.
- Paired t testA paired t test compares two measurements on the same subjects, such as before and after. Work out each person's difference, then run a one-sample t test on the differences against 0.
- Test for a proportionA one-proportion z test checks a claimed share p0. The statistic is z = (p-hat - p0) / square root of (p0(1 - p0) / n). Use the claimed p0, not p-hat, in the standard error.