Mira scores 72% on one Mathematics set and 84% on another.

Has she improved?

Perhaps.

But only if the two performances are comparable enough for the difference to mean what we think it means.

Training comparability is the discipline of checking whether two performances are sufficiently alike in target capability, task demand and conditions before treating their difference as evidence of improvement or decline.

The idea is simple: compare like with like before telling a story about change.

---

Quick Read: What Must Be Comparable?

• the underlying capability;
• topic or objective coverage;
• task difficulty;
• representation and wording;
• timing;
• support and hints;
• familiarity;
• degree of independence;
• scoring criteria.

Performance A → Check Conditions → Performance B → Judge Equivalence → Interpret Change

Comparability Is Not Sameness

Two tasks do not need to be identical.

In fact, identical tasks can create familiarity and answer-memory effects.

The goal is comparable meaning.

If both tasks sample algebraic method selection at similar difficulty and under similar time and support conditions, they can be useful comparisons even when the numbers differ.

This is closely related to formal test-equating ideas. A 2025 Ofqual study in Frontiers in Education examined methods for identifying equivalent marks across different test forms and emphasised that changes in form difficulty can undermine straightforward score comparisons. Large-scale equating is far more technical than tuition monitoring, but the core warning travels well: different forms do not automatically support equivalent interpretations.

Comparable Capability

Before comparing scores, ask whether the two tasks sampled the same thing.

A 90% algebra worksheet and a 70% mixed examination paper are not direct measures of the same performance condition.

The first may test execution when the method is already known.

The second may test recognition, selection, timing, switching and checking in addition to algebra.

The lower score can coexist with stronger underlying algebra.

The tasks are asking different questions.

Comparable Difficulty

A learner can improve while a score falls if the later task is substantially harder.

Likewise, a score can rise because the later task is easier.

Difficulty should therefore be treated as part of the measurement context.

For informal training, exact statistical equating is usually unnecessary.

But basic discipline helps:

• use parallel item families;
• keep the target concept stable;
• avoid comparing a clean practice item with a full transfer task as though they were equivalent;
• record when the later task deliberately raises difficulty.

Comparable Support

Jonas answers five inference questions correctly during tuition.

His tutor asks, “Which phrase gives you evidence?” on four of them.

The following week he answers four of five correctly without prompts.

The raw score barely changes.

The capability does.

Support dependence fell.

This is why Training Cue Hierarchy should be recorded when support materially affects performance.

Comparable Timing

Untimed and timed tasks can measure overlapping knowledge but different performance demands.

If Mira solves accurately in five minutes during learning and later solves accurately in two minutes under examination timing, that change is meaningful.

If she scores lower under a strict time limit, do not conclude that conceptual knowledge declined without checking where time changed the route.

Timing is part of comparability whenever speed is part of the target.

Comparable Sampling

A Mathematics paper containing many of Mira’s strongest topics should not be compared naively with a paper containing a heavy concentration of her weakest topic.

Use Training Sampling.

Ask whether both samples represent the capability broadly enough.

This is one reason formal assessment systems use blueprints and statistical linking: content coverage and difficulty matter to score meaning.

Comparable Independence

Homework completed with parental checking is not directly comparable with a closed-book examination.

A model composition written after class discussion is not equivalent to a fresh timed composition.

A Science explanation produced after the tutor names the concept is not equivalent to one where the learner must first identify the concept from evidence.

Record independence because it changes the meaning of the score.

Mira: 68% Can Be Better Than 78%

Mira scores 78% on a blocked algebra worksheet immediately after teaching.

Two weeks later she scores 68% on a mixed set containing algebra, graphs and geometry.

The family sees decline.

The tutor compares conditions.

• first task: blocked;
• second task: mixed;
• first: immediate;
• second: delayed;
• first: method known;
• second: method had to be selected;
• first: untimed;
• second: timed.

The second task is harder across several dimensions.

Now inspect process signals.

Mira’s algebraic execution is actually cleaner.

The main losses occur in graph interpretation.

The 68% is not proof of decline.

It is a different performance sample.

Jonas: Same Score, Better Independence

Jonas scores 7/10 on two inference sets.

On the first, the tutor provides four evidence cues.

On the second, none.

The score is unchanged.

The conditions are not.

The second 7/10 represents stronger independent performance.

Nadia: Different Apparatus, Same Reasoning

Nadia handles a plant experiment accurately.

Later she handles an unfamiliar heat experiment with similar accuracy.

The content surface differs.

The underlying experimental reasoning is comparable.

This can provide stronger evidence of transfer than repeating another plant setup.

Comparability and Training Baselines

Training Baselines establishes the starting state.

Comparability protects the later comparison.

If the baseline is untimed and heavily supported while the retest is timed and independent, the apparent difference may be difficult to interpret.

Match the comparison deliberately or state clearly which dimension changed.

Comparability and Training Retests

A retest should preserve the target capability while changing enough surface detail to reduce answer memory.

This is the central bridge to Training Retests.

Fresh does not mean unrelated.

Comparable does not mean identical.

Comparability and Measurement Noise

If conditions differ substantially, the difference between two scores contains extra uncertainty.

Training Measurement Noise warns against overinterpreting that uncertainty.

Comparability reduces noise by making the comparison cleaner.

Do Not Force Comparability When the Target Has Progressed

Sometimes the later task should be harder.

The learner is progressing.

Do not keep every retest easy merely to preserve score comparability.

Instead, separate two questions:

• Did the old capability improve under comparable conditions?
• Can the learner now handle a harder condition?

Those are both useful, but they are different claims.

Do Not Compare Percentages Without Looking at the Paper

Parents often receive only percentages.

The paper tells the rest of the story.

Topic coverage.

Question type.

Difficulty.

Time distribution.

Error clustering.

The same percentage can emerge from completely different capability profiles.

Do Not Use Comparability to Explain Away Every Bad Result

Comparability is not an excuse.

If repeated comparable samples show deterioration, accept the signal.

Investigate.

Change the plan if needed.

The goal is calibrated interpretation, not protecting a preferred story.

A Parent Comparability Audit

• Are these two scores measuring roughly the same capability?
• Were the papers similar in difficulty and coverage?
• Was one timed and the other untimed?
• Was support different?
• Was one task familiar and one fresh?
• Are we comparing total scores when the important change is independence or speed?
• What claim can this comparison actually justify?

A Tutor Comparability Audit

• What construct am I comparing?
• Which task features changed?
• Which changes were intentional progression?
• Was support equivalent?
• Was timing equivalent?
• Was content sampling comparable?
• What should I avoid claiming from this pair of scores?

The Deeper Idea: Improvement Is Meaningful Only Relative to a Stable Question

“Better” always hides a comparison.

Better at what?

Under which conditions?

With how much support?

Against what level of difficulty?

Training comparability keeps improvement claims attached to the capability and conditions that gave the numbers meaning.

Research Foundations

Useful current anchors include the 2025 Frontiers in Education Ofqual study of comparative-judgment and statistical equating, the 2025 Cambridge University Press & Assessment report The Biggest Equating Study in the World… Ever, and the OECD discussion of assessment comparability and measurement equivalence. These sources concern formal assessment systems, not tuition worksheets, but they support the narrower principle used here: score interpretation becomes less defensible when forms, conditions and constructs differ in ways that are ignored.

Continue Through How Training Works

Read this with Training Baselines, Training Sampling, Training Retests and Training Measurement Noise.