How do you choose the right statistical test for a Q1 publication?

Direct answer: Choosing the right statistical test means selecting the one test whose assumptions fit your data and whose logic answers your research question, defined before you collect or analyse data, not after. The choice flows from three things: the question you are asking, the type of your variables, and the design that produced the numbers.

Our separate guide to Levene's test goes into this in more detail.

Evidence: Guidance for biomedical authors stresses that the test must follow from the hypothesis and data structure, and that the statistical approach should be pre-specified rather than chosen to fit the result (Marusteri & Bacarea, 2010). The SAMPL guidelines for reporting statistics in biomedical journals, developed by Lang and Altman, ask authors to state the purpose of each test and the assumptions behind it (Lang & Altman, 2015).

Example: A Vietnamese first-time author came to a MAAS mentor with results already run, asking which test "looked best." Her mentor walked it back to the research question and the data design first, then matched a single appropriate test, turning a fishing exercise into a defensible analysis plan a reviewer could trust.

Direct answer: Before any test, answer four questions: what type are your variables (nominal, ordinal, or continuous), how many groups are you comparing, are the groups independent or paired, and are you comparing differences or measuring association? Those four answers point to a small set of candidate tests.

Evidence: University statistics services frame test selection around variable type, the number of groups or levels, and whether samples are paired or independent (University of York, n.d.; UCLA Statistical Consulting Group, n.d.-a). Applied-statistics textbooks aimed at postgraduate researchers list the same four questions as the standard starting point for choosing a test (Field, 2013). Choosing a paired test when samples are independent or the reverse, is a common and avoidable error (Marusteri & Bacarea, 2010).

Example: A MAAS-coached student studying a before-and-after intervention had planned an independent t-test. Her mentor showed that because the same subjects were measured twice, a paired test was correct, a single reclassification that changed the entire analysis and its reported result.

Direct answer: Use a parametric test when your continuous data reasonably meet its assumptions, because parametric tests are more powerful when those conditions hold. Switch to a non-parametric alternative when data are ordinal, badly skewed, or contain extreme outliers; some guides also advise it for very small samples, because normality is hard to assess in them (Marshall, n.d.).

If this is new to you, the Kolmogorov-Smirnov test and Shapiro-Wilk is covered step by step in another guide. For two independent groups with skewed data, our Mann Whitney U test guide walks through the steps. For paired scores that are not normally distributed, the Wilcoxon signed rank test is the usual choice and our guide explains it. For three or more independent groups on ranked data, see our guide to the Kruskal Wallis test.

Evidence: When assumptions are met, parametric tests are generally more powerful and more likely to detect a real effect. However, with larger samples the Central Limit Theorem allows parametric tests to perform well even when the underlying data are not perfectly normal, so sample size is part of the decision, not just the histogram (Marusteri & Bacarea, 2010). Our decision guide to parametric and non-parametric tests works through this choice test by test.

Example: A Vietnamese researcher with a 14-participant pilot insisted on a t-test because "everyone uses it." Her MAAS mentor explained that at that sample size, with visible skew, a Mann-Whitney U test was the safer, more honest choice and reviewers would respect the caution rather than penalise it.

Direct answer: Every parametric test carries assumptions you must check and report: independence of observations, approximate normality (often of residuals, not raw data), and homogeneity of variance between groups. Inspect distributions visually, use formal tests where appropriate, and decide in advance what you will do if an assumption fails.

Evidence: SAMPL implementation research found that analysis of the assumptions necessary for a test, and consideration of the impact of outliers, are among the most frequently missing elements in published statistics, assumption checks were absent in a large share of articles reviewed (Ordak, 2025). Reviewers increasingly expect these checks to be stated explicitly. In practice, that means choosing a check you can defend and reporting it next to the primary statistic rather than leaving the reader to assume it was done. Formal normality tests are unreliable at both ends: with n below 20 they are unlikely to detect non-normality, and above n of 50 they can be too sensitive, so a histogram or QQ plot is the safer companion (Marshall, n.d.).

Example: A MAAS Publishing Advisory client had run an ANOVA without mentioning any assumption check. During a Final-stage review, her mentor added a normality and variance check, documented one mild violation, and noted the robustness of the chosen test, a short paragraph that pre-empted an obvious reviewer question.

Direct answer: A p-value tells you only how compatible your data are with a null hypothesis; it does not measure the size or importance of an effect. Q1 reviewers expect effect sizes and confidence intervals alongside p-values, so readers can judge practical significance, not just statistical significance.

Before you read any p-value, make sure your null and alternative hypotheses are written as exact opposites; our guide to the null vs alternative hypothesis shows the paired form for each common test.

Evidence: The American Statistical Association's 2016 statement on p-values, built around 6 core principles, warns that statistical significance should not by itself drive scientific conclusions and that p-values are widely misused (Wasserstein & Lazar, 2016). One of those principles states it directly: "Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold" (Wasserstein & Lazar, 2016, p. 131). Reporting effect sizes with confidence intervals communicates both the strength of evidence and the precision of the estimate, which a bare p-value cannot (Lakens, 2013). Cohen's (1988) benchmarks treat a standardized mean difference of 0.2 to 0.8 as small to large, so "p < 0.05" alone cannot tell a reviewer which end of that range the true effect sits at.

Example: A Vietnamese author reported a string of "p < 0.05" results with no effect sizes. Her MAAS mentor helped her add effect sizes and 95% confidence intervals throughout, reframing the discussion around how large and how precise each effect was, exactly the lens a Q1 reviewer applies.

Direct answer: The recurring rejection triggers are: choosing the test after seeing the data, confusing paired and independent designs, ignoring assumption checks, running many comparisons without correction, reporting post-hoc power, and describing methods too vaguely to reproduce. Each signals to a reviewer that the analysis may not be reliable.

Evidence: Post-hoc power calculations have no value once a study has concluded; sample size justification should be done a priori (Hickey et al., 2018). A Bonferroni correction divides the significance threshold by the number of tests run, so with 12 comparisons a 5% threshold falls to about 0.4%, which is why uncorrected multiple testing inflates false positives so quickly. SAMPL guidelines call for a clear statement of each test's purpose, its assumptions, and effect sizes, deficiencies in exactly these areas are the most commonly flagged (Ordak, 2025).

Example: A MAAS client's manuscript ran twelve uncorrected comparisons and justified the sample with post-hoc power. Her mentor flagged both before submission, helped her define one primary outcome and an a priori rationale, and the revised methods read as far more disciplined.

Direct answer: Decide the analysis before you collect data, write a short statistical analysis plan, and use a reporting checklist such as SAMPL as you draft the methods. Then get developmental feedback from a mentor who has reviewed quantitative papers, so problems surface before submission rather than in a reviewer's report.

Evidence: Following established reporting guidance measurably improves how reviewers receive a paper (Lang & Altman, 2015).

Example: A MAAS mentor guided a Vietnamese postgraduate through three stages: at the first the analysis plan and chosen tests were fixed, at the second the assumptions and effect sizes were documented, and at the last the language and reporting were checked against SAMPL. The student stayed the author throughout, with the mentor advising at each stage rather than running the analysis for her.

If you want to know which parts of the statistics a helper can legitimately share with you, see what thesis statistics help a student is allowed to get.