You have your dataset open. You know you need to compare groups, or test a relationship, or predict an outcome. But which test? A t-test? ANOVA? Something with a name you half-remember from a lecture six months ago? The answer is usually simpler than it feels. Four questions will get you there.
Start With Your Variables
Before anything else, look at what you are measuring. Your dependent variable is the outcome, the thing you want to explain or compare. Your independent variable is what defines the groups or serves as a predictor. Get this distinction wrong and every decision that follows will be off.
The type of your dependent variable determines the entire branch of tests available to you. If your outcome is categorical (yes or no, pass or fail, three diagnostic categories), you are working with chi-square tests or logistic regression. If your outcome is continuous (a score, a measurement in centimeters, reaction time in milliseconds), you move into the territory of t-tests, ANOVA, correlation, and linear regression.
Here is a concrete example. Suppose you are measuring exam scores across three teaching methods for your education thesis. Your dependent variable is continuous (the exam score), and your independent variable is categorical (method A, method B, method C). That combination points you toward ANOVA. Now suppose instead you are asking whether students passed or failed depending on which method they used. The outcome is now categorical, so you would reach for a chi-square test of independence. Same study topic, different test, because the variable type changed.
Two Groups or More?
Once you know your outcome is continuous, the next question is simple: how many groups are you comparing?
Two groups means a t-test. Three or more groups means ANOVA. That is the fork most students miss, and the mistake that follows is predictable. A student with three groups (say, a control group and two intervention groups) runs three separate t-tests: control vs. intervention A, control vs. intervention B, and intervention A vs. intervention B. It seems logical. It is also wrong.
Each t-test carries a 5% chance of a false positive when you set alpha at 0.05. Run three of them on the same dataset and your actual false-positive rate climbs to roughly 14%. Run five comparisons and it exceeds 22%. The math is straightforward: the probability of at least one false positive is 1 minus 0.95 raised to the power of the number of tests. ANOVA solves this by testing all groups simultaneously in a single procedure, holding the Type I error rate at 5% where it belongs. If ANOVA finds a significant difference, you then use post hoc tests (Tukey HSD, Bonferroni, or Games-Howell depending on your situation) to identify which specific groups differ. We explain this comparison in more depth in our article on when to use a t-test versus ANOVA.
Independent or Paired Samples?
Not all group comparisons are structured the same way. The relationship between your participants and your groups matters because it determines which variant of the test you need.
Independent samples means different people in each group. A clinical trial where 30 patients receive a drug and a separate 30 patients receive a placebo is an independent design. Paired samples (also called repeated measures) means the same people measured more than once, or participants matched on relevant characteristics. You measure anxiety levels in 40 patients before an 8-week therapy program, then measure the same 40 patients again after the program ends. Same patients, two time points. That is a paired design.
Why does this matter statistically? Paired designs reduce variability because each participant serves as their own control. The tests account for this. An independent-samples t-test compares means between two separate groups and pools their variance. A paired-samples t-test looks at the difference within each participant and tests whether the average of those differences is zero. The same logic extends to ANOVA: one-way ANOVA handles independent groups, while repeated-measures ANOVA handles the same participants measured across three or more conditions or time points.
Choosing the wrong variant is not a minor error. Using an independent-samples test on paired data throws away the within-subject information and typically produces a less powerful test. Using a paired test on independent data violates assumptions and produces unreliable p-values.
Check Your Assumptions
Parametric tests (t-tests, ANOVA, Pearson correlation, linear regression) assume specific things about your data. Violate those assumptions badly enough and the results become untrustworthy. The two assumptions that matter most are normality and homogeneity of variance.
Normality means the dependent variable (or the residuals, in regression) should follow an approximately normal distribution within each group. Test it with Shapiro-Wilk for samples smaller than 50, or Kolmogorov-Smirnov for larger ones. A Q-Q plot gives you a visual check: if the points follow the diagonal line reasonably well, normality holds. If they curve away sharply at the tails, it does not.
Homogeneity of variance means the spread of scores should be roughly equal across groups. Levene's test checks this. A significant Levene's test (p < 0.05) tells you the variances are unequal, which means you either need a correction (Welch's t-test instead of Student's t-test, or the Brown-Forsythe adjustment for ANOVA) or you should switch to a nonparametric alternative.
When assumptions fail, you have two options. You can apply corrections, like the Welch adjustment that does not assume equal variances. Or you can switch to a nonparametric test, which makes no distributional assumptions and instead works with ranks rather than raw values. The tradeoff is a small loss of statistical power when the data actually are normal, which is usually acceptable in a thesis context where the priority is defensible results.
One practical note: with large samples (roughly 30 or more per group), the Central Limit Theorem makes parametric tests tolerably robust even when normality is mildly violated. This does not mean you can ignore assumptions. It means that with n = 200 per group and a slight skew, a t-test will still perform well. With n = 12 per group and a heavy skew, it will not.
The Decision Table
The table below maps common research questions to the right test. Find your situation in the left column, check whether your data meet parametric assumptions, and read across.
| Research Question | Parametric Test | Nonparametric Alternative |
|---|---|---|
| Compare means of 2 independent groups | Independent-samples t-test | Mann-Whitney U |
| Compare means of 2 paired groups | Paired-samples t-test | Wilcoxon signed-rank |
| Compare means of 3+ independent groups | One-way ANOVA | Kruskal-Wallis |
| Compare means of 3+ paired groups | Repeated-measures ANOVA | Friedman test |
| Test association between 2 continuous variables | Pearson correlation | Spearman correlation |
| Test association between 2 categorical variables | Chi-square test | Fisher's exact test (small samples) |
| Predict a continuous outcome from predictors | Linear regression | — |
| Predict a binary outcome from predictors | Logistic regression | — |
Save this table. Print it. Tape it to the wall next to your monitor. We have seen hundreds of students reach for the wrong test simply because they did not have a quick reference available at the moment they needed it.
What About Correlation and Regression?
Not every research question is about comparing groups. Sometimes you want to know whether two variables move together, or whether one predicts the other.
Correlation measures the strength and direction of a linear relationship between two continuous variables. Pearson's r is the standard choice when both variables are approximately normally distributed. If either variable is ordinal (like a Likert scale with 5 points) or if the data are not normally distributed, Spearman's rank correlation is the safer option. Both produce a coefficient between -1 and +1, where values near zero indicate no linear relationship and values near the extremes indicate a strong one.
Regression goes a step further. Instead of just measuring association, regression builds a predictive model. Linear regression predicts a continuous outcome from one or more predictors. A psychology student might use it to predict exam performance from hours of study, sleep quality, and anxiety score. Logistic regression does the same thing for a binary outcome: pass or fail, diagnosed or not diagnosed, employed or unemployed.
A common mistake we see in thesis drafts is using correlation when the research question is clearly predictive. If your hypothesis says "sleep quality predicts academic performance," you need regression, not correlation. Correlation tells you they are related. Regression tells you by how much the outcome changes for each unit change in the predictor, while controlling for other variables. The second mistake is interpreting either one as proof of causation. A strong correlation between ice cream sales and drowning rates does not mean ice cream causes drowning. Both increase in summer. Correlation and regression describe associations, not causes, unless your study design (a randomized controlled experiment) supports causal claims.
A Practical Shortcut
When you are stuck, run through these four questions in order. First: is your dependent variable continuous or categorical? That eliminates half the options immediately. Second: how many groups are you comparing? Two means t-test territory; three or more means ANOVA. Third: are your samples independent or paired? That picks the specific variant. Fourth: do your data meet normality and equal-variance assumptions? That decides between the parametric version and its nonparametric alternative.
Those four questions will narrow you down to one or two candidate tests in under a minute. From there, you run the test, check the output, and interpret the results. Our guide on how to interpret p-values correctly covers what to do once you have your output in front of you.
If you want the process automated entirely, Academic Stats Agent handles this decision for you. Upload your data, describe your research question, and it identifies the correct test, checks assumptions, runs the analysis, and produces a report you can paste into your thesis. It follows the same logic described in this article, just faster.
Key takeaway: The right statistical test depends on three things: what type of variables you have, how many groups you are comparing, and whether your data meet the assumptions for parametric testing. When in doubt, start with the decision table above.
Frequently Asked Questions
Which statistical test should I use for ordinal data?
Ordinal data such as single Likert items are usually analyzed with nonparametric tests: Mann-Whitney U for two independent groups, Kruskal-Wallis for three or more groups, Wilcoxon signed-rank for paired measurements, and Spearman correlation for relationships between variables.
When should I use a parametric vs a nonparametric test?
Use a parametric test when your dependent variable is continuous, approximately normally distributed, and group variances are roughly equal. If normality or equal variances are clearly violated, especially with small samples, switch to the nonparametric alternative, which uses ranks and makes no distributional assumptions.
How do I choose between a t-test, ANOVA, and chi-square?
Look at your variables: a continuous outcome compared across two groups calls for a t-test, across three or more groups for ANOVA, while a categorical outcome crossed with a categorical predictor calls for a chi-square test of independence.