Post 1 — ANOVA is not a test for “more than two groups”

Suppose I have four analytical methods and want to know whether they produce the same mean relative bias.

The first temptation is to say: “There are four groups, so I need ANOVA.”

That is technically correct, but it misses the important part.

The real reason to use a one-way ANOVA is that the scientific question is:

Do the four population means differ?

For the example dataset, the estimated mean relative biases are approximately:

Reference: −9.5% Method A: −17.7% Method B: +2.0% Method C: −2.6%

The global F-test is extremely strong: F(3,236) ≈ 320, p < 0.001.

So the four means are clearly not all equal.

But notice what the ANOVA has not told us.

It has not told us which methods differ.

It has not told us which method is closest to zero.

It has not told us whether a difference is chemically relevant.

And it certainly has not told us why the methods behave differently.

Those are different questions.

This is why I prefer to think of ANOVA as a linear model rather than simply as “the test for more than two groups”.

The code can be almost trivial:

lm(RelativeBias_pct ~ Method)

The difficult part happened before the function call: defining the response, the experimental unit, the hypothesis and what magnitude of difference would actually matter.

For this project, a difference of 5 percentage points is used as a predefined reference for chemical relevance.

That number does not come from ANOVA.

It comes from the scientific problem.

The statistical test answers a statistical question. It does not define the scientific question for us.

---

The implementation details and reproducible analysis for this step are available in the GitHub repository at https://github.com/andreabz/analytical-anova/ and on the project page at https://andreabz.github.io/analytical-anova/