When should you use a Kruskal-Wallis test instead of ANOVA?

Direct answer: The Kruskal-Wallis test checks whether three or more independent groups come from the same distribution, using ranks instead of raw scores. Nahm (2016) describes it as a nonparametric way to analyse variance that asks whether the median values of the groups differ (given similar group shapes), which makes it the rank-based counterpart of a one-way ANOVA.

A typical dissertation case is a survey that compares satisfaction ratings from first-year, second-year and final-year students, where each student belongs to one group only. The ratings are ordered categories, not true measurements, so a mean of 3.4 on a five-point scale is hard to defend. Kruskal and Wallis (1952) built the test by pooling every observation, ranking them from 1 to the total sample size, giving tied values the mean of their ranks, and summing the ranks within each group. If one group consistently holds the higher ranks, its rank sum grows and the test statistic, H, grows with it. The R Core Team (n.d.) states the null hypothesis in general terms: the location parameters of the distribution are the same in each group, and the alternative is that at least one differs. With two groups only, the same logic gives the Mann-Whitney U test.

Direct answer: A one-way ANOVA compares group means on raw scores and assumes roughly normal residuals with similar variances. The Kruskal-Wallis test compares groups on ranks, so it drops the normality assumption and tolerates outliers (Nahm, 2016). The price is that it answers a looser question, about distributions or medians, rather than a precise question about means.

Choosing the rank test is not a free upgrade. When the ANOVA assumptions hold, ANOVA uses more of the information in your data, so most supervisors expect you to justify the switch in one sentence of your methods chapter, for example by pointing to a clearly skewed histogram, an outlier you cannot remove, or a Likert outcome. If your data are fine for ANOVA, run the one-way ANOVA instead.

Direct answer: The groups must be independent, each person must appear in one group only, and the outcome must be at least ordinal. To read a significant result as a difference in medians, the group distributions also need a similar shape. A common rule asks for at least five observations in each group.

Independence is the assumption students break most often without noticing. Kruskal and Wallis (1952) framed the test for separate samples, and the statstutor guide from the University of Sheffield lists independent observations first, followed by similar sample sizes and more than five data points per sample (Marshall & Patel, 2015), slightly stricter than the rule below. SciPy's documentation gives the same rule of thumb that "each sample must have at least 5 measurements" (SciPy developers, n.d.), because below that size the chi-square approximation for H becomes unreliable.

The shape condition matters for interpretation rather than for running the test. In their original paper, Kruskal and Wallis (1952) wrote that "one may often assume that the populations are of approximately the same form, in the sense that if they differ it is by a shift or translation" (p. 584). When the groups have that same shape, a significant H can fairly be reported as a difference in medians. When one group is skewed left and another skewed right, the test still detects that the distributions differ, but the word "median" in your results sentence overstates what you have shown. Plotting a boxplot or histogram for each group before you write is the quickest way to check.

Direct answer: Read the H statistic, its degrees of freedom (number of groups minus one) and the p-value. A p below your alpha, usually .05, means at least one group differs from another. The test does not say which groups differ, so a significant result needs pairwise follow-up tests before you can name them.

Software output often lists a mean rank for each group next to H. Mean ranks tell you the direction of the difference: the group with the highest mean rank tends to have the higher scores. They are not medians, so report the medians or interquartile ranges from your descriptive table as well. The SciPy developers (n.d.) put the limit plainly: "rejecting the null hypothesis does not indicate which of the groups differs."

The usual follow-up is Dunn's (1964) procedure, which compares pairs of groups using the same pooled ranks as the omnibus test. Because you run several comparisons, each p-value must be adjusted to keep the overall error rate at your chosen alpha. With three groups there are 3 pairwise comparisons, so a Bonferroni split of .05 tests each pair at about .0167; with four groups there are 6 pairs and each is tested at about .0083. Dunn (1964) noted that splitting alpha across the comparisons "is a conservative choice in order to obtain a 1 − α overall level" (p. 248). This Bonferroni-style correction is, according to the statstutor guide, "the standard adjustment made after the Kruskall-Wallis" (Marshall & Patel, 2015, p. 18), and in SPSS it appears as the "Adj. Sig." column of the pairwise table, which is already adjusted, so you compare it with .05 directly.

Direct answer: The test statistic is H = [12 / (N(N + 1))] × Σ(R_j² / n_j) − 3(N + 1), where N is the total sample, R_j the rank sum and n_j the size of group j. Effect size is often reported as epsilon-squared, H divided by (N² − 1)/(N + 1).

Take three groups of 10 students, so N = 30: the constant 12 / (30 × 31) is about 0.0129, the subtracted term 3 × 31 is 93, and the degrees of freedom are 3 − 1 = 2. At .05 with 2 degrees of freedom, the chi-square critical value is 5.99. Kruskal and Wallis (1952) showed that H is distributed approximately as chi-square with k − 1 degrees of freedom unless the samples are very small, which is why software reports a chi-square p-value.

A p-value says nothing about size, so pair it with an effect size. Tomczak and Tomczak (2014) give the epsilon-squared estimate as E²_R = H / [(n² − 1)/(n + 1)], with values running from 0, no relationship, to 1, a perfect relationship. The same paper gives an eta-squared version, η²_H = (H − k + 1)/(n − k), for k groups. The paper does not fix small, medium or large cut-offs for these measures, so describe the size against studies in your own field rather than borrowing labels. Our guide to reporting effect size and confidence intervals explains why the number matters more than the label.

Direct answer: A common convention is to give the H statistic with its degrees of freedom in brackets, the exact p-value and an effect size, then report the adjusted pairwise results with medians for each group. APA 7 does not prescribe a single template, so check your department handbook.

Here is a worked sentence with illustrative numbers, not data from a real study: "Satisfaction differed across the three year groups (N = 95), H(2) = 8.41, p = .015, ε² = .09. Dunn's pairwise tests with Bonferroni adjustment showed that final-year students (Mdn = 4) rated satisfaction higher than first-year students (Mdn = 3), p = .012, while no other pair differed." Write H and p in italics, give p to two or three decimals with no zero before the point, and write p < .001 rather than p = .000.

Direct answer: In SPSS use Analyze, Nonparametric Tests, Independent Samples, then request pairwise comparisons. In R call kruskal.test(score ~ group, data = df) (R Core Team, n.d.). In Python call scipy.stats.kruskal() with one array per group (SciPy developers, n.d.). All three return H and a p-value; post hoc tests need an extra step in R and Python.

The R Core Team (n.d.) notes that kruskal.test returns the statistic, the degrees of freedom of the approximate chi-squared distribution, and the p-value. SciPy's function returns the statistic "corrected for ties" (SciPy developers, n.d.), which matters for Likert data where many answers share a value. State the software and version in your methods section. If the analysis itself is the part you are unsure about, data analysis support from MAAS coaches you through choosing the test, running it and interpreting the output, while the analysis and the write-up stay yours.

Direct answer: Use Kruskal-Wallis when each participant belongs to one group only, and the Friedman test when the same participants are measured three or more times or under three or more conditions. Friedman is the rank-based counterpart of a repeated-measures ANOVA, just as Kruskal-Wallis is the counterpart of a one-way ANOVA.

The choice follows your design, not your data. If you surveyed different students in each year group, the groups are independent and Kruskal-Wallis fits. If you asked the same students to rate three teaching formats, every person contributes to every condition, the scores are linked, and running Kruskal-Wallis would break its independence assumption. For two related measurements, the paired version is the Wilcoxon signed-rank test.