# Significance level and decision

The significance level alpha is the cut-off you set before the test, often 0.05. If the p-value is at most alpha, you reject H0. Otherwise you fail to reject it, which is not the same as proving H0 true.

Statistics (university) · Part 06: Hypothesis testing · Card 3 of 9 · Last updated 2026-10-19

## Analogy

It is the bar of evidence you set in advance, like deciding how convincing a witness must be.

## Example

**Making the call**

p = 0.0077, alpha = 0.05
0.0077 < 0.05: reject H0
If p were 0.20:
fail to reject H0
(not enough evidence)

## In practice

Drug trials and quality checks fix alpha first to avoid choosing a bar after seeing the result.

## Practice

### Flashcards

- **When do you reject H0?** When the p-value is at most alpha.
- **Does a non-significant result prove H0 true?** No. It only means the evidence against H0 was not strong enough.

### Interview questions

**A machine should fill bottles with a mean of 500 ml. A sample of 16 bottles has a mean of 504 and s of 8. Test H0: mean = 500 at the 5% level.**

- SE = 8 / 4 = 2
- t = 4 / 2 = 2.0 with 15 df
- Critical value 2.131 (two-sided)
- Fail to reject H0
- State the conclusion in words

## Questions people ask

### What happens when I run many tests at once?

Each test at alpha 0.05 carries a 5 percent false alarm chance, and the chances pile up. Across 20 independent tests, the chance of at least one false alarm is 1 minus 0.95^20, about 64 percent. Corrections such as Bonferroni tighten the threshold.

### Why say fail to reject instead of accept the null?

A non-significant result may simply mean the sample was too small to detect a real effect. Absence of evidence is not evidence of absence. We say we lack enough evidence against H0, leaving open that a real effect exists but was too small to detect with this sample.

## Related concepts

- [One-sample t test](https://studyvisually.com/learn/science/statistics-university/statistics-university-part-6/statistics-university-part-6-concept-6)
- [Null and alternative hypotheses](https://studyvisually.com/learn/science/statistics-university/statistics-university-part-6/statistics-university-part-6-concept-1)
- [p-value](https://studyvisually.com/learn/science/statistics-university/statistics-university-part-6/statistics-university-part-6-concept-2)
- [Type I and Type II errors](https://studyvisually.com/learn/science/statistics-university/statistics-university-part-6/statistics-university-part-6-concept-4)

Web page: https://studyvisually.com/learn/science/statistics-university/statistics-university-part-6/statistics-university-part-6-concept-3
Study this card in the app: https://studyvisually.com/app/learn?category=statistics-university&card=statistics-university-part-6-concept-3