Paired versus independent observations in before-and-after analysis

When the same units are measured before and after, analyze their within-unit differences and preserve the pairing by identity. Two measurements from one person are related observations, not two independent people.

A paired analysis estimates average observed change under its assumptions. Before/after data alone do not establish that an intervention caused the change.

Build the difference for each person

The analyst statistics lab contains eight synthetic people with before scores ranging from 40 to 145. Their changes are 5, 7, 4, 6, 8, 5, 7 and 6 score units.

import numpy as np
from scipy import stats
from build_and_verify import BEFORE, AFTER

differences = AFTER - BEFORE
paired = stats.ttest_rel(AFTER, BEFORE, alternative='two-sided', nan_policy='raise')
on_differences = stats.ttest_1samp(differences, popmean=0, nan_policy='raise')
wrong_design = stats.ttest_ind(AFTER, BEFORE, equal_var=False, nan_policy='raise')
assert differences.tolist() == [5, 7, 4, 6, 8, 5, 7, 6]
assert differences.mean() == 6
assert np.allclose([paired.statistic, paired.pvalue],
                   [on_differences.statistic, on_differences.pvalue])
assert paired.pvalue < .001 and wrong_design.pvalue > .5
interval = paired.confidence_interval(confidence_level=.95)
assert np.allclose([interval.low, interval.high], [4.905391669732888, 7.094608330267112])
print({'mean_change': 6, 'paired_p': float(paired.pvalue),
       'independent_p_for_wrong_design': float(wrong_design.pvalue),
       'change_ci95': [float(interval.low), float(interval.high)]})

The paired result uses the relatively consistent within-person changes. The independent calculation treats the large between-person spread as if the two groups were unrelated, producing a p-value around 0.75 in this constructed example.

This contrast is not a reason to choose pairing whenever it yields significance. Pairing is justified by the study design. The SciPy paired-test reference describes the related-sample procedure.

Join on identity, not row position

Real before and after files may be sorted differently. Merge them by person_id with one-to-one validation, inspect unmatched people and only then calculate after minus before.

Two arrays can have equal length while pairing entirely different people. A numerically valid paired test on those arrays would answer an invented question.

If several measurements exist per person in each period, define the within-period summary or use a model appropriate to repeated observations. A many-to-many merge can create multiple artificial pairs and overstate the amount of independent evidence.

Handle incomplete pairs explicitly

If some people lack an after measurement, report how many pairs remain and compare the missingness pattern with the target population. Dropping incomplete pairs may change who is represented.

Do not replace a missing after score with the before score merely to create zero change. That inserts an assumption about an unobserved outcome.

The current fixture has complete pairs, so it does not evaluate an attrition strategy. A real analysis plan should specify that strategy before inspecting the observed changes.

Check assumptions on the relevant quantity

For a small-sample paired t analysis, the distributional assumption concerns the differences, alongside independence between pairs. It is not a requirement that before and after columns have identical distributions.

Inspect unusual differences and consider whether pairs are themselves clustered, such as several employees from one team. The person count may overstate independent information if shared conditions drive changes.

Separate change from causal effect

Practice, seasonality, regression to the mean and other concurrent changes can produce before/after differences. Without an appropriate comparison design, the estimated six-unit change cannot be attributed solely to a particular intervention.

The fixture is synthetic and makes no claim about a course's effectiveness. It demonstrates the arithmetic and uncertainty of a paired comparison under stated assumptions.

Exercise: shuffle the after file, merge by person_id and verify the unchanged result. Then deliberately pair by row position and show how the individual differences change even though the two column means remain the same.

NeuraPath's Data Analytics with Generative AI course connects statistical testing with identity and study design. A useful before/after report preserves the pairs and states what the design can—and cannot—attribute.

Continue learning

This article is part of the Statistics for analytical decisions sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Choose a t-test from the question and data design.
• Continue with Chi-square tests: check expected counts before interpreting results.