ANOVA and Linear Models for Analytical Chemistry

About this project

Comparing analytical methods is rarely a matter of asking whether their mean results are different.

The statistical question depends on the experimental design, on the structure of the measurements, and on what we actually want to conclude.

This project follows a single analytical chemistry case study through a sequence of increasingly realistic statistical problems. The aim is not to build a catalogue of statistical tests, but to show how the choice of model follows from the scientific question and from the way the data were generated.

The central response variable is relative bias, expressed as a percentage. Four analytical methods are compared across three matrices.

As the analysis develops, the same dataset allows us to ask increasingly specific questions:

• Do the methods differ in their mean bias?
• Which comparisons between methods are scientifically relevant?
• Does the matrix affect the results?
• Does the effect of the method depend on the matrix?
• What changes when the design is unbalanced?
• Can a continuous covariate help explain the observed pattern?
• Are the assumptions of the fitted models defensible?
• How sensitive are the conclusions to assumption violations?
• How should repeated measurements from the same experimental unit be analysed?
• What happens when the observations have a hierarchical structure?

The statistical methods become more complex only when the structure of the problem requires them to.

The experimental setting

The main dataset contains 60 samples, each analysed using the same four analytical methods.

The samples belong to three matrices:

• Fill
• Soil
• Sediment

Thus, the complete balanced dataset contains:

\[ 60 \times 4 = 240 \]

observations.

This structure is important. The four measurements obtained from the same sample are related because they originate from the same experimental unit. This feature is initially easy to overlook, but becomes central in the later episodes on repeated measurements and mixed-effects models.

The data also contain information about analytical batches, days, analysts, and instruments. These variables allow us to consider additional sources of experimental structure without treating every available variable as automatically belonging in the statistical model.

From question to model

The episodes follow a deliberate progression.

01 — One-way ANOVA

We begin with the simplest question:

Do the four methods have the same mean bias?

A one-way ANOVA provides the initial global comparison.

02 — Contrasts

A significant global test does not tell us which differences matter.

We therefore move from the omnibus question to scientifically meaningful comparisons between methods.

03 — Two-way ANOVA

The matrix is introduced as a second factor.

The question becomes:

Does matrix explain systematic differences in relative bias in addition to method?

04 — Interaction

The analysis then asks whether the effect of the analytical method is consistent across matrices.

This is where the distinction between main effects and interactions becomes important.

05 — Unbalanced designs

Real experiments are not always perfectly balanced.

We examine how unequal numbers of observations affect the analysis and why the interpretation of effects becomes more dependent on the statistical model.

06 — ANCOVA

A continuous variable is introduced as a potential explanatory factor.

The question is no longer only whether groups differ, but whether part of the observed variation can be explained by a measured continuous covariate.

07 — Diagnostics

A fitted model is not automatically a trustworthy model.

We examine residuals, variance patterns, normality, influential observations, and the limitations of formal diagnostic tests.

08 — Robustness

Diagnostics tell us whether violations are apparent.

Robustness analysis asks a complementary question:

Would our conclusions change if the assumptions were not fully satisfied?

Bootstrap procedures, permutation-based approaches, Welch-type inference, and robust standard errors provide different ways of addressing this question.

09 — Repeated measures

The experimental structure now becomes explicit.

The four method measurements obtained from the same sample are not independent observations. We examine how repeated measurements can be analysed while accounting for the within-sample correlation.

10 — Mixed models

Finally, we consider the hierarchical structure of the data.

A mixed-effects model allows the sample-to-sample variability to be represented explicitly rather than treating the 240 measurements as independent.

The purpose is not to make the model more complicated for its own sake. It is to ask whether acknowledging the experimental structure changes the uncertainty associated with our conclusions.

A recurring principle

Throughout the project, statistical significance is treated as only one part of the analysis.

A small p-value can establish evidence against a null hypothesis, but it does not by itself tell us whether the difference is scientifically important, whether the model is appropriate, or whether the result is robust.

The analysis therefore repeatedly returns to three questions:

1. What scientific question are we asking?
2. Does the statistical model represent the way the data were generated?
3. Are the conclusions sensitive to assumptions or modelling choices?

These questions are more important than the choice of any individual test.

Reproducibility

All analyses are implemented in R and documented using Quarto.

The project uses renv to manage the R package environment. The required package versions are recorded in renv.lock, allowing the computational environment to be restored rather than relying on the packages currently installed on a particular machine.

The datasets used by the analyses are included in the repository, and the statistical results are generated directly from the analysis documents.

How to use the project

The individual episodes can be read independently, but they are designed to be followed in sequence.

The recommended order is:

01 → 02 → 03 → 04 → 05 → 06 → 07 → 08 → 09 → 10

Earlier episodes introduce concepts that are reused later, while the final episodes return to the experimental structure that was present in the data from the beginning.

The repository is therefore best viewed not as a collection of independent examples, but as one statistical analysis developed progressively as the scientific questions become more demanding.

Scope

This is a focused case study, not a general statistics textbook.

The emphasis is on ANOVA and related linear-model approaches commonly encountered when analysing experimental data from analytical chemistry. More advanced methods are introduced only when they help address a specific feature of the experimental design.

The underlying principle is simple:

The statistical model should follow the scientific question and the experimental design — not the other way around.