# Post-hoc comparisons After a significant ANOVA, a post-hoc test such as Tukey's finds which pairs of groups differ. It adjusts for the many comparisons so the overall false alarm rate stays near 5 percent. Statistics (university) · Part 07: Comparing categories and many groups · Card 6 of 8 · Last updated 2026-10-19 ## Analogy After the school says one class is different, you check each pair of classes, with a stricter bar each time. ## Example **Why adjust?** 3 groups = 3 pairs Chance of a false alarm in at least one: 1 - 0.95^3 = 0.143 10 groups = 45 pairs: 1 - 0.95^45 = 0.90 ## In practice Without correction, comparing many pairs almost guarantees a false positive somewhere. ## Practice ### Flashcards - **Why use a post-hoc correction?** To stop many comparisons from inflating the false alarm rate. ### Interview questions **Why use ANOVA instead of several t tests, and what does the F statistic compare?** - Many t tests inflate the false alarm rate - ANOVA tests all means at once - F compares between-group and within-group variation - A large F suggests real differences - A post-hoc test finds which groups differ ## Questions people ask ### What is the multiple comparisons problem? Each test has a chance of a false alarm, and running many tests multiplies the chances. With 45 pairwise tests, a false discovery is almost certain. Corrections such as Tukey or Bonferroni raise the bar for each comparison to compensate. ### What is the Bonferroni correction? Divide the significance level by the number of tests. For 5 comparisons at an overall alpha of 0.05, use 0.01 for each. It is simple and safe but conservative, so real effects may be missed. Tukey's method is usually tighter for pairwise comparisons. ## Related concepts - [One-way ANOVA](https://studyvisually.com/learn/science/statistics-university/statistics-university-part-7/statistics-university-part-7-concept-4) - [F statistic](https://studyvisually.com/learn/science/statistics-university/statistics-university-part-7/statistics-university-part-7-concept-5) Web page: https://studyvisually.com/learn/science/statistics-university/statistics-university-part-7/statistics-university-part-7-concept-6 Study this card in the app: https://studyvisually.com/app/learn?category=statistics-university&card=statistics-university-part-7-concept-6