Welch's t-test vs Student's t-test: which should you use?

Direct answer: Student's t-test pools the two sample variances into one estimate and assumes the populations share the same variance, with degrees of freedom of n1 + n2 − 2. Welch's t-test keeps the variances separate and adjusts the degrees of freedom, so it stays valid when the spreads differ (Delacre et al., 2017).

Welch (1947) proposed the separate-variances approach, and Delacre et al. (2017) describe it as dividing the mean difference by an unpooled error term. The t statistic is the mean difference divided by the square root of s1²/n1 + s2²/n2, which is the same formula NIST gives for the unequal-variance case (National Institute of Standards and Technology, n.d.). The only real change from Student's test is therefore the denominator and the degrees of freedom, not the question being asked.

The software defaults in the table come from the R and SciPy documentation, and Delacre et al. (2017) note that Welch's test is already the default in R and Minitab.

Direct answer: The Welch–Satterthwaite formula is ν = (s1²/N1 + s2²/N2)² divided by [(s1²/N1)²/(N1 − 1) + (s2²/N2)²/(N2 − 1)]. It lowers the degrees of freedom as the group variances or sizes drift apart, which makes the test slightly more cautious than Student's (National Institute of Standards and Technology, n.d.).

NIST states that "the Welch-Satterthwaite degrees of freedom are robust for unequal sample sizes and/or unequal variances" (National Institute of Standards and Technology, n.d.). Delacre et al. (2017) print the same expression as their equation 4, and it is the reason a Welch result so often shows a value such as 47.97 instead of a whole number.

Do not round the fraction when you report it. Report the degrees of freedom exactly as your software prints them, because the fraction is the visible sign that Welch's correction was applied. If your output shows a whole number such as 48 for a two-group comparison with 50 participants, you are most likely looking at the Student row.

Direct answer: Use Welch's t-test whenever you compare the means of two independent groups, which is what Delacre et al. (2017) recommend. Student's test only has an edge when the variances are truly equal, and then Welch loses little. Unequal group sizes with unequal variances are where Student's test misleads most.

Delacre et al. (2017) conclude that "Welch's t-test should always be used instead of Student's t-test" (p. 99). Their reasoning is practical: "because there is no correct test to perform that assures SDR = 1, and because variances are likely not to be equal in certain research areas, our recommendation is to always use Welch's t-test" (p. 97). SDR here is the ratio of the two standard deviations.

The cost of ignoring this depends on the pairing of variance and sample size. When the larger variance sits with the smaller sample, Student's Type 1 error rate is inflated, and when the larger variance sits with the larger sample, the rate drops below the nominal level (Delacre et al., 2017). In one simulation with groups of 40 and 60 and a standard deviation ratio of 0.5, p-values below .05 occurred in 8.3% of runs when the larger variance went with the smaller sample (Delacre et al., 2017, p. 98). Equal group sizes soften the problem, which is why Student's test survives in textbooks.

Two points keep this honest. First, the 2017 paper has a published correction that leaves the main message intact, namely that Welch's test should be privileged over Student's when comparing means (Delacre et al., 2022). Second, Welch's advantage is about error control with unequal variances, not about magic: it still assumes roughly normal data, and a very small group makes any t-test fragile.

Direct answer: Delacre et al. (2017) advise against the two-step routine of running Levene's test and then picking Student's or Welch's. The preliminary test has low power with small samples, and running two tests on the same data makes the alpha level and power of the t-test depend on Levene's outcome.

Many courses still teach the routine. A University of Southern Queensland open textbook, for example, tells students that the output shows two rows, one for equal variances assumed and one for equal variances not assumed, and links the choice to Levene's test (Statistics for Research Students, section 3.3). You may have to follow that routine for a unit that marks it, so read the unit guide first.

The statistical objection is documented. Delacre et al. (2017, p. 99) write that "the widely recommended two-step approach, where the assumption of equal variances is tested using Levene's test and, based on the outcome of this test, a choice of Student's t-test or Welch's t-test is made, should not be used". They also report that detecting a standard deviation ratio of 1.5 with Levene's test needs about 120 participants for power of 0.80 (Delacre et al., 2017, p. 94). Zimmerman (2004) reached a similar verdict from simulations, finding that the two-stage procedure fails to protect the significance level and usually makes the situation worse.

The practical advice is short. Report Welch's result as your primary test, and if your marker expects a Levene's check, report it as a descriptive note rather than as the gatekeeper.

Direct answer: In SPSS, read the "Equal variances not assumed" row, which is the Welch test. R's t.test already runs Welch by default (var.equal = FALSE), but SciPy's ttest_ind defaults to Student's test (equal_var = True), so you must set equal_var=False yourself.

Navarro's open SPSS tutorial explains that SPSS outputs both the Student t and Welch's t, and that the second row is the Welch test (Learning Statistics with SPSS, section 10.4). Take the t, df and Sig. (2-tailed) values from that row, plus the confidence interval of the difference in the same row.

For R, the documentation says that if var.equal is TRUE the pooled variance is used, and otherwise the Welch (or Satterthwaite) approximation to the degrees of freedom is used (R Core Team, n.d.). For Python, the SciPy documentation states that equal_var=True is the default and performs the standard test, while False performs Welch's t-test, which does not assume equal population variance (SciPy community, n.d.). A student who pastes a SciPy call without the argument and writes "Welch's t-test" in the methods section has reported the wrong test. This is a common slip when moving between R and Python.

# R: Welch by default
t.test(score ~ group, data = d)

# Python (SciPy): switch Welch on explicitly
from scipy import stats
stats.ttest_ind(a, b, equal_var=False)

If your software offers neither a clear label nor a documented default, look up the documentation page for your version before you name the test in writing.

Direct answer: Report the t statistic, the fractional degrees of freedom, the exact p value, the group descriptives and the confidence interval for the mean difference, and name the test as Welch's. The example below uses invented numbers for illustration only, so replace them with your own output.

Here is a hypothetical example. Suppose a group of 18 students on a revised tutorial scored M = 72.4 (SD = 6.1) and a comparison group of 32 scored M = 66.8 (SD = 11.3). The standard deviations differ by almost a factor of two, so Welch's test is the sensible choice. The calculation gives a standard error of 2.46, t(47.97) = 2.28, p = .027, with a 95% confidence interval for the mean difference of about 0.65 to 10.55. Student's test on the same summary statistics gives t(48) = 1.94, p = .058, which would flip the conclusion at the .05 level.

A sample sentence in APA 7 style reads: "Welch's t-test showed that the revised tutorial group (M = 72.4, SD = 6.1, n = 18) scored higher than the comparison group (M = 66.8, SD = 11.3, n = 32), t(47.97) = 2.28, p = .027, 95% CI [0.65, 10.55]." Add an effect size as well, and the guide on reporting effect sizes and confidence intervals shows how.

To interpret the result, read the sentence the way a marker would. A p value below your alpha (usually .05) means the data would be unlikely if the population means were equal, and the confidence interval tells the reader how large the difference might plausibly be.

Direct answer: Use Welch's t-test when you want to compare means and the data are approximately continuous and roughly normal within each group. Use Mann-Whitney when your scale is ordinal or the distributions are badly non-normal. Mann-Whitney is not a cure for unequal variances (Delacre et al., 2017).

Delacre et al. (2017) explicitly list the belief that the Mann-Whitney test copes with unequal variances as a mistaken one (note 6, p. 99). Ruxton (2006) goes further in the title of his short article, arguing that the unequal variance t-test is an underused alternative to both Student's test and the Mann-Whitney U test. The two tests also answer different questions: Welch compares means, while Mann-Whitney compares rank distributions, so switching between them changes your research claim and not only your method.

A sensible order for a student is to look at the scale of measurement and the distribution shape first, then choose between a mean-based test (Welch) and a rank-based test (Mann-Whitney). Justify the choice in one sentence in your methods section, and avoid switching tests after seeing which one gives the smaller p value.

Direct answer: Welch's test has no separate sample-size rule, and Delacre et al. (2017) say it has practically the same power as Student's test even when the standard deviations are equal. The exact p value comes from the t distribution at the Welch degrees of freedom, so it will differ slightly from Student's.

Student's test is described as robust to unequal variances when group sizes are equal, provided each group is large enough to give correct estimates of means and standard deviations, which Delacre et al. (2017, p. 92) put at n ≥ 5. That is a floor for estimation and not a power calculation, so this guide gives no target sample size. If your unit asks for one, take the method from your unit guide or your supervisor.

Because the degrees of freedom are fractional, tables of critical values are of little help. Read the exact p value from your software, and report it to three decimals (or as p < .001 for very small values), as in the example above.

Many students lose marks not on the arithmetic but on the choice and the write-up: naming Student's test while the output used Welch, rounding the degrees of freedom, or defending Levene's test as a gatekeeper. In data analysis support at MAAS, a mentor guides you through the logic of the comparison, how to read the output from SPSS, R or Python, and how to word the result. You run the analysis and write the results yourself, and your supervisor and institution decide the outcome. Bring your dataset description, your unit's reporting requirements and the exact output you have so far to the first session, so the discussion stays on your assignment and not on a generic example.