Statistical Test for Your Research
— Decision Tree Guide —
A practical, no-jargon guide for researchers, students, and data analysts who want to stop guessing and start choosing with confidence
Picture this: you’ve spent weeks collecting data, your spreadsheet is immaculate, and your research question is crystal clear. Then your supervisor asks: ‘So what statistical test are you running?’ And suddenly your mind goes blank.
You’re not alone. Choosing the right statistical test is one of the most anxiety-inducing parts of research not because it’s impossibly complex, but because nobody ever taught it to you in a way that sticks. Most textbooks bury it under layers of formulas. Most online guides either oversimplify it to the point of uselessness or assume you already know what you’re looking for.
This guide is different. By the end of it, you’ll have a clear, repeatable decision-making framework a mental decision tree that tells you exactly which test to use, why, and when to be careful. No unnecessary jargon. No assumption that you remember everything from your stats class.
“The right statistical test is not the fanciest one. It’s the one that matches your data, your question, and your assumptions.”
Before You Touch a Single Test: Ask These Four Questions
Every statistical test selection flows from four fundamental questions about your data and your research goal. Answer these honestly and the test almost chooses itself.
Question 1: What is your research goal?
Are you trying to describe your data, compare groups, find relationships between variables, or predict an outcome?
This is the fork in the road. Descriptive statistics (means, medians, standard deviations) are not inferential tests they summarize. Inferential tests let you make claims about populations from samples. Know which world you’re in.
Question 2: What type of data do you have?
Data type determines more about your test choice than almost anything else. Here are the four levels:
Nominal (Categorical): Names or labels with no meaningful order. Blood type, country of birth, species. You can count these, but you can’t rank them.
Ordinal: Categories with a meaningful order, but unequal gaps. Pain rated 1–5, education level (high school / bachelor’s / master’s). You can rank these, but the difference between rank 1 and 2 isn’t necessarily equal to the difference between 4 and 5.
Interval: Ordered, equal gaps, but no true zero. Temperature in Celsius, calendar years. 0°C doesn’t mean ‘no temperature.’
Ratio: Ordered, equal gaps, and a meaningful zero. Height, weight, income, reaction time. This is the richest data type zero actually means the absence of the thing.
Pro Tip: Interval and ratio data are often grouped together as ‘continuous’ data. Most parametric tests assume your dependent variable is continuous.
Question 3: How many groups or variables are involved?
Are you comparing one group to a known standard? Two groups to each other? Three or more groups? Are you looking at one outcome variable or multiple? Each combination points to a different family of tests.
Question 4: Are your data normally distributed?
This separates the world into two great kingdoms: parametric and non-parametric tests. Parametric tests are more powerful but assume your data follows a roughly normal distribution (the classic bell curve). Non-parametric tests make no such assumption they’re more flexible but somewhat less statistically powerful.
You can check normality visually with a histogram or Q-Q plot, or formally with tests like Shapiro-Wilk (best for small samples) or Kolmogorov-Smirnov. For large samples (n > 30), the Central Limit Theorem usually saves you the sampling distribution of the mean will be approximately normal regardless.
The Decision Tree: Step-by-Step
Work through these decision points in order. Each answer narrows your options until the right test emerges.
| Q1 | Are you comparing groups or examining relationships/associations? | YES → Go to Q2 (Comparison) | NO → Go to Q5 (Relationship) |
| Q2 | How many groups are you comparing? | YES → 2 groups → Go to Q3 | NO → 3+ groups → Go to Q4 |
| Q3 | Are the two groups independent (different people) or paired (same people, two conditions)? | YES → Independent → t-test or Mann-Whitney U | NO → Paired → Paired t-test or Wilcoxon Signed-Rank |
| Q4 | Are the groups independent or are repeated measures involved? | YES → Independent → One-Way ANOVA or Kruskal-Wallis | NO → Repeated → Repeated Measures ANOVA or Friedman |
| Q5 | Are both variables continuous (interval or ratio)? | YES → Yes → Pearson or Spearman Correlation | NO → No → Chi-Square Test of Association |
| Q6 | Are your data normally distributed? (applies to Q3 and Q4) | YES → Yes → Use the parametric test | NO → No → Use the non-parametric equivalent |
The Full Statistical Test Reference Table
Use the decision tree above to find your category, then use this table to confirm the right test and understand its requirements.
COMPARING TWO INDEPENDENT GROUPS
| Test | Data Type | Assumption | Use When |
| Independent Samples t-test | Continuous (ratio/interval) | Normal distribution, equal variance | Comparing means of 2 separate groups (e.g., men vs. women on test scores) |
| Welch’s t-test | Continuous | Normal distribution, unequal variance OK | Like t-test but when group variances differ significantly |
| Mann-Whitney U | Ordinal or non-normal continuous | No normality required | Non-parametric alternative to independent t-test |
COMPARING TWO PAIRED/RELATED GROUPS
| Test | Data Type | Assumption | Use When |
| Paired Samples t-test | Continuous | Differences are normally distributed | Same subjects measured twice (e.g., before and after treatment) |
| Wilcoxon Signed-Rank | Ordinal or non-normal | No normality required | Non-parametric alternative to paired t-test |
| McNemar’s Test | Nominal (binary) | Large enough sample | Before/after comparison of categorical outcomes (e.g., pass/fail) |
COMPARING THREE OR MORE INDEPENDENT GROUPS
| Test | Data Type | Assumption | Use When |
| One-Way ANOVA | Continuous | Normal, homogeneous variance | Comparing means across 3+ independent groups |
| Welch’s ANOVA | Continuous | Normal, unequal variance OK | 3+ groups with unequal variances |
| Kruskal-Wallis | Ordinal or non-normal | No normality required | Non-parametric alternative to one-way ANOVA |
COMPARING THREE OR MORE GROUPS — REPEATED MEASURES
| Test | Data Type | Assumption | Use When |
| Repeated Measures ANOVA | Continuous | Normality, sphericity | Same subjects across 3+ conditions or time points |
| Friedman Test | Ordinal or non-normal | No normality required | Non-parametric alternative to repeated measures ANOVA |
EXAMINING RELATIONSHIPS BETWEEN VARIABLES
| Test | Data Type | Assumption | Use When |
| Pearson Correlation | Continuous (both variables) | Linear relationship, normality | Measuring strength and direction of linear relationship |
| Spearman Correlation | Ordinal or non-normal | Monotonic relationship | Non-parametric alternative to Pearson; ranks-based |
| Chi-Square Test | Categorical (both variables) | Expected cell freq. ≥ 5 | Testing association between two categorical variables |
| Point-Biserial Correlation | One continuous, one binary | Normality of continuous var. | Relationship between continuous and binary variable |
PREDICTING OUTCOMES (REGRESSION FAMILY)
| Test | Data Type | Assumption | Use When |
| Simple Linear Regression | Continuous outcome | Linearity, normality of residuals | Predicting one continuous outcome from one predictor |
| Multiple Linear Regression | Continuous outcome | Same as above, no multicollinearity | Predicting continuous outcome from 2+ predictors |
| Logistic Regression | Binary outcome (0/1) | Large sample, independence | Predicting probability of a categorical outcome |
| Ordinal Regression | Ordinal outcome | Proportional odds assumption | Predicting an ordered categorical outcome |
The Five Most Common Test-Selection Mistakes
Even experienced researchers get these wrong. Know them, and you’re already ahead of the curve.
Mistake 1: Ignoring the Normality Assumption
Using a t-test or ANOVA on data that is severely skewed or has significant outliers. The tests will run software never refuses but the results may be meaningless. Always check your distributions before testing. If normality is violated, switch to the non-parametric equivalent.
⚠ Watch Out: Normality tests (Shapiro-Wilk, K-S) have low power in small samples and too much power in large ones. Always combine formal tests with visual inspection (histogram, Q-Q plot).
Mistake 2: Multiple Comparisons Without Correction
Running ten separate t-tests across ten groups inflates your false positive rate dramatically. If you’re comparing multiple groups, use ANOVA first. If you then want to know which specific groups differ, use a post-hoc test (Tukey’s HSD, Bonferroni, Games-Howell) that corrects for multiple comparisons.
Mistake 3: Treating Ordinal Data as Continuous
Likert scale responses (Strongly Agree to Strongly Disagree) are ordinal, not continuous. You cannot assume equal intervals between ‘Agree’ and ‘Strongly Agree.’ Running a t-test on raw Likert scores is technically incorrect — though with enough items, many researchers’ average scales and treat them as approximately continuous. Know what you’re doing and why.
Mistake 4: Confusing Statistical and Practical Significance
A p-value tells you whether a result is unlikely to be due to chance. It says nothing about whether the effect is meaningful or large. Always pair your test results with effect size measures: Cohen’s d for t-tests, eta-squared for ANOVA, r for correlations. A study with 10,000 participants will find ‘significant’ differences that are utterly trivial in practice.
Mistake 5: Choosing a Test After Seeing the Data
This is called HARKing — Hypothesizing After Results are Known. If you run every possible test and report only the one that gave p < 0.05, you’re not doing science, you’re doing data fishing. Pre-register your analysis plan where possible, and be honest when analysis is exploratory.
Quick-Reference Cheat Sheet
When you’re in a hurry, use this summary to orient yourself fast.
| Situation | Parametric Test | Non-Parametric Alternative |
| 2 independent groups | Independent t-test | Mann-Whitney U |
| 2 related/paired groups | Paired t-test | Wilcoxon Signed-Rank |
| 3+ independent groups | One-Way ANOVA | Kruskal-Wallis |
| 3+ repeated/related groups | Repeated Measures ANOVA | Friedman Test |
| Relationship between 2 continuous vars | Pearson Correlation | Spearman Correlation |
| Association between 2 categorical vars | — (no parametric version) | Chi-Square Test |
| Predict continuous outcome | Linear Regression | — (use transformation or GLM) |
| Predict binary outcome | Logistic Regression | — |
| Before/after, binary outcome | McNemar’s Test | — |
A Practical Workflow: From Data to Test in 5 Steps
Here’s the exact process to follow every single time:
- Define your research question in one sentence. What are you comparing or measuring?
- Identify your variables. Which is the dependent variable (outcome)? Which are independent variables (predictors/groups)?
- Classify your data types. Use the nominal/ordinal/interval/ratio framework for each variable.
- Check your assumptions. Run a normality check on your dependent variable. Check for outliers. Assess group sizes and variance equality.
- Apply the decision tree. Let your answers to the four core questions guide you to the right test family, then confirm with the reference table.
Remember: There is rarely one single ‘correct’ test. Often two or three tests are defensible for the same dataset. What matters is that you can justify your choice based on your data’s properties and your research question — and that you check your assumptions.
Final Thought: Tests Serve Your Research — Not the Other Way Around
One of the strangest things that happens in academic training is that statistical tests start to feel like the point of research. They’re not. They’re tools. Hammers in service of building something.
The test you choose should be driven by what your data actually looks like, what your research question genuinely asks, and what assumptions you can honestly defend. Not by what will give you a p-value below 0.05. Not by what your supervisor used in their last paper. Not by what you’ve always done.
Understanding why you’re choosing a test not just which test to choose is what separates competent data analysts from excellent ones. Once you internalize the decision framework in this guide, you’ll find that choosing the right test stops being anxiety-inducing and starts being the natural, almost obvious outcome of understanding your own data.
“Statistics is not about finding significance. It is about finding truth — and being honest about what your data can and cannot tell you.”
The next time someone asks which test you’re running, you won’t hesitate. You’ll know and more importantly, you’ll know why.