In short#
- One-way ANOVA: F = MS between groups / MS within groups, df = (k − 1, N − k). It tests equality of means; valid inference assumes independent observations, approximately normal errors and equal variances.
- Welch ANOVA allows unequal variances but still assumes independent observations and approximately normal errors. Kruskal-Wallis compares ranks; a location interpretation is clearest when group distributions have similar shapes.
- For multiple comparisons, choose the family and adjustment in advance. The omnibus test assesses evidence against equal means but does not identify particular pairs.
- Repeated measurements on the same experimental unit require the Factor data table, not one-way ANOVA.
- For three or more matched conditions, use the Friedman test, which compares ranks within each experimental unit. It does not require normally distributed raw values, but it does require complete measurements and independent units.
In plain words#
With three or more groups, it is tempting to test every pair with a separate t-test. The more tests in a family, the greater the chance of a false positive. The omnibus ANOVA asks whether the data are compatible with all means being equal; it does not replace tests of specific planned comparisons.
ANOVA compares variation among group means with variation among observations within groups. The larger this ratio, the stronger the evidence against equal means under the model assumptions.
A significant overall test indicates evidence that not all means are equal, but it does not say which. Pairwise tests assess specific pairs; after adjustment, some comparisons may still be non-significant.
Within each block, values become ranks from 1 to the number of conditions; ties receive their average rank. The statistic summarizes how consistently condition ranks differ across blocks. A tie correction accounts for equal values, and Kendall's W from 0 to 1 describes the strength of rank agreement. With few blocks, the asymptotic p value may be inaccurate.
In Linza#
- For independent groups, create a Column table with three or more groups. In Analyses, choose ANOVA, Welch ANOVA or Kruskal-Wallis to match the design and assumptions.
- Choose a comparison family and adjustment. Tukey and Dunnett report adjusted p-values and 95% intervals. Kruskal-Wallis pair comparisons show the rank-biserial effect for each pair beside Dunn's adjusted p-value. This effect describes pairwise superiority, not a mean difference; its confidence interval is not calculated.
- The demo project contains three synthetic groups and is a good place to start with the analysis.
- For repeated measurements, choose Matched conditions and Rank comparison. Linza recommends the Friedman test, then shows pairwise Wilcoxon signed-rank tests with Holm correction. If measurements are missing, exclude the whole block; do not combine repeated measurements from different people into one block.
Common mistakes#
- Reporting unadjusted pairwise p values after ANOVA.
- Claiming that group B differs from all others without pairwise results.
Check yourself#
Sources#
- Fisher R.A. (1925) Statistical Methods for Research Workers. Oliver and Boyd. Search for publication Opens in a new tab
- Welch B.L. (1951) On the comparison of several mean values: an alternative approach. Biometrika, 38. Open publication Opens in a new tab
- Kruskal W.H., Wallis W.A. (1952) Use of ranks in one-criterion variance analysis. Journal of the American Statistical Association, 47(260). Search for publication Opens in a new tab
- Friedman M. (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association, 32(200). Search for publication Opens in a new tab
References are for further reading. A method’s suitability depends on the study design and its assumptions.