Science Education Systems · Article 91. Maya, Jia Jun, Hana and Ethan are fictional learners used to make scientific reasoning visible. This article owns one distinct scientific job: counterfactual reasoning—comparing an observed outcome with the outcome that would have occurred for the same unit under a different causal condition. It does not replace general causality or comparison. Its job is the causal “what if” that experiments and quasi-experiments try to approximate.
The 50-second parent route
A causal question is rarely just “what happened?” It is usually:
What happened because X occurred, compared with what would have happened if X had not occurred?
The route is:
unit → actual condition → observed outcome → alternative condition → unobserved counterfactual outcome → comparison strategy → assumptions → estimated causal effect → sensitivity → decision
The fastest diagnostic is to ask: Compared with what believable alternative world? If a learner cannot define the alternative, the causal claim is incomplete.
This article extends How Scientific Causality Works, How Scientific Comparison Works, How Scientific Randomisation Works and How Scientific Confounding Works.
1. The counterfactual is the outcome we cannot observe directly
A plant receives extra light and grows 4 cm more. What would that exact plant have grown under the same time and conditions without extra light? We cannot observe both histories simultaneously. Causal inference exists because one of those worlds is missing.
2. This is the fundamental problem of causal inference
For one unit at one moment, only one potential outcome is observed. The alternative is counterfactual. Science therefore constructs credible comparison groups or models that approximate the missing world.
3. Maya’s first error is before-after certainty
Marks rise after a new study method. She says the method caused the improvement. Her repair is to ask what the marks would likely have done over the same period without the method.
4. Jia Jun’s first error is using any comparison
He compares treated students with an older class from another year. His repair is to ask whether the comparison group is credible enough to represent the untreated counterfactual.
5. Hana’s first error is impossible perfection
Because the true counterfactual can never be observed, she says causality can never be known. Her repair is to understand that scientific designs approximate counterfactuals with varying credibility.
6. Ethan’s first error is one counterfactual for everyone
He assumes all units would respond identically under the alternative condition. His repair is to allow treatment effects to vary across people, places and times.
7. Potential outcomes formalise the idea
For a unit, imagine Y(1) under treatment and Y(0) under control. The individual causal effect is Y(1) − Y(0). Only one is observed. The missing potential outcome must be estimated indirectly.
8. Randomisation creates exchangeable groups on average
If treatment is assigned by chance, the untreated group can approximate what would have happened to the treated group without treatment, subject to design quality and random variation.
9. Controls are counterfactual devices
A control group is not merely “the group that gets nothing.” It is an attempt to represent the relevant alternative condition.
10. The right control depends on the question
No treatment.
standard treatment.
placebo.
sham procedure.
alternative dose.
Different counterfactuals answer different causal questions.
11. Historical controls can be weak counterfactuals
Outcomes last year may differ because technology, population, environment or policy changed. Time introduces alternative explanations.
12. Self-controls can be useful
A unit is compared with itself before and after exposure. This controls stable individual characteristics but remains vulnerable to time trends, maturation, history and carryover.
13. Crossover designs approximate both worlds within one person
A participant receives Treatment A and Treatment B in different periods. Order is randomised, and washout may reduce carryover. This improves counterfactual comparison when the condition is reversible and stable enough.
14. Carryover can break the counterfactual
If Treatment A permanently changes the participant, the later Treatment B period no longer represents the world that would have existed without A.
15. Parallel-group designs avoid some carryover problems
Different units receive different conditions simultaneously. Randomisation then supports comparability across groups rather than within individuals.
16. Counterfactual reasoning is not fantasy
The alternative world must be constrained by scientific knowledge and design. “What if gravity stopped?” can be a useful thought experiment, but it is not the counterfactual for estimating the effect of a tutoring programme.
17. Good counterfactuals change one causal condition while preserving relevant background
The closer the alternative matches the actual world except for the exposure of interest, the stronger the causal interpretation.
18. This is why fair tests matter
Primary Science teaches “change one variable while keeping others the same.” That is early counterfactual logic: approximate what would have happened if only the chosen cause differed.
19. Primary 3 can learn “same object, different condition” thinking
We cannot place one ice cube both in shade and sun simultaneously, so we use matched ice cubes under different conditions. The match approximates the missing alternative.
20. Primary 4 can learn control logic
If one plant receives fertiliser, another similar plant without fertiliser helps estimate what growth would have looked like without the treatment.
21. Primary 5 can learn alternative-explanation logic
If the fertilised plant also receives more water, the control no longer isolates the counterfactual difference.
22. Primary 6 can learn counterfactual language
“If the independent variable had not changed, would the dependent variable still have changed?” This question links fair tests to causality.
23. Secondary Science can formalise potential outcomes
Students can distinguish observed and unobserved outcomes, treatment effects, exchangeability, confounding, selection and intervention.
24. Average treatment effect is a population counterfactual
It compares the average outcome if everyone received treatment with the average outcome if everyone received control.
25. Treatment effect on the treated asks a different question
What was the effect among those who actually received treatment? This can differ from the population average if treatment response varies.
26. Effect heterogeneity matters
Some learners benefit strongly.
others little.
some may be harmed.
A single average can hide different counterfactual effects across subgroups.
27. Counterfactuals require consistency
The treatment being imagined should correspond to a well-defined intervention. “More exercise” is vague if dose, duration and type differ dramatically.
28. Versions of treatment matter
Two interventions share the same label but differ in delivery. The counterfactual is ambiguous if treatment identity is not precise enough.
29. Interference complicates counterfactuals
One person’s treatment changes another person’s outcome. Vaccination, classroom teaching, social networks and environmental interventions can create spillovers.
30. Stable-unit assumptions can fail
If outcomes depend on others’ assignments, each unit has more than two potential outcomes. The counterfactual system becomes networked.
31. Spillover can be part of the effect
A school programme may change peer culture. Treating spillover as mere contamination can miss the real system mechanism.
32. Counterfactual reasoning can be individual or population-level
“Would this patient have recovered without treatment?” is individual. “Would recovery rates be higher if this population received treatment?” is population-level.
33. Population effects are easier to estimate than individual counterfactuals
Randomised groups can estimate average differences. Knowing exactly what would have happened to one specific individual is much harder.
34. Prediction is not automatically causal counterfactual reasoning
A model predicts who will score highly. That does not tell us whether changing study time will cause the score to rise.
35. Prediction asks what will happen
Counterfactual causality asks what would happen under intervention. The distinction matters whenever decisions aim to change outcomes.
36. Confounding breaks counterfactual comparability
Treated and untreated groups differ in another variable affecting the outcome. The control group no longer represents the treated group’s untreated world adequately.
37. Matching attempts to reconstruct the counterfactual
Find untreated units similar to treated units on relevant baseline variables. The quality of the approximation depends on what was measured and matched.
38. Propensity scores summarise treatment-assignment information
Under assumptions, units with similar probabilities of treatment based on observed covariates can be compared. Unmeasured confounding remains a threat.
39. Weighting creates a pseudo-population
Observations are weighted so treated and untreated groups become more comparable on measured covariates. The method reconstructs a target counterfactual population statistically.
40. Regression adjustment models the missing world
Estimate expected outcomes under treatment and control conditional on covariates. Model specification becomes part of causal credibility.
41. Doubly robust methods combine models
Some estimators combine treatment and outcome models so correct specification of one may be sufficient under certain assumptions. Robustness still depends on measured confounders.
42. Instrumental variables create indirect counterfactual variation
An instrument changes treatment exposure without affecting the outcome through other pathways. Valid instruments are powerful but difficult to justify.
43. Regression discontinuity uses thresholds
Units just above and below a treatment cutoff may be similar except for treatment eligibility. Near the threshold, the design approximates a local counterfactual comparison.
44. Difference-in-differences uses parallel trends
Compare outcome changes over time between treated and untreated groups. The crucial counterfactual assumption is that, without treatment, their trends would have remained parallel enough.
45. Interrupted time series builds a temporal counterfactual
Use pre-intervention trends to estimate what would likely have happened afterward without the intervention. Sudden external events can weaken that counterfactual.
46. Synthetic controls combine comparison units
A weighted combination of untreated regions or units is constructed to resemble the treated unit before intervention. The synthetic trajectory approximates the missing untreated world.
47. No observational method removes assumptions
Every counterfactual reconstruction relies on claims about exchangeability, trends, instruments, functional form or missing variables. Strong Science states these assumptions explicitly.
48. Counterfactual graphs make assumptions visible
Causal diagrams can show which variables create backdoor paths, which should be adjusted for, and which adjustments would create bias.
49. Mediators complicate counterfactual questions
If treatment changes a mediator which changes outcome, total effect and direct effect become different counterfactual quantities.
50. Natural direct and indirect effects require stronger assumptions
Mediation analysis imagines cross-world quantities that can be difficult to identify. Advanced causal questions demand careful definitions.
51. Counterfactual reasoning clarifies “because”
“The patient improved because of treatment” means, approximately, the outcome would have been worse under the relevant no-treatment alternative. Without that contrast, “because” is only narrative.
52. Worked case: tutoring programme
Students receiving extra tuition improve by 8 marks. Did tuition cause the gain? The counterfactual is what those same students would have achieved without tuition over the same period.
53. Why before-after comparison is weak
Students may improve through school teaching, maturation, exam familiarity or easier tests. The observed gain contains several possible causes.
54. A randomised control strengthens the tutoring counterfactual
Eligible students are assigned by chance to immediate tuition or delayed tuition. If conditions remain comparable, the control group better approximates the untreated path.
55. The causal estimand should be clear
Effect of being offered tuition?
effect of actually attending?
effect after ten sessions?
effect on exam marks or independent transfer?
Different counterfactuals produce different answers.
56. Worked case: fertiliser experiment
A plant receives fertiliser and grows taller. The counterfactual is the same plant under identical conditions without fertiliser. Since that cannot be observed, use comparable plants assigned to conditions.
57. Individual variability remains
Even perfectly matched plants differ. Replication estimates average treatment effects while natural variation remains.
58. Worked case: cooling insulation
An insulated container loses less heat. The relevant counterfactual is the same system without insulation under matched starting temperature, volume, environment and measurement.
59. Counterfactuals help choose controls
If the question is the effect of insulation material, the control should differ in insulation, not in container size or water volume.
60. Worked case: policy change
A city changes traffic rules and congestion falls. What would congestion have done without the policy? Nearby untreated cities, historical trends or synthetic controls may help construct the alternative.
61. The policy counterfactual is vulnerable to concurrent events
Fuel prices, school holidays, roadworks or economic changes may also affect congestion. Counterfactual credibility depends on addressing them.
62. Counterfactual reasoning helps failure analysis
Would the failure have occurred if one barrier had worked? If yes, that barrier was not sufficient. If no, it becomes a strong corrective target.
63. Counterfactual reasoning helps mechanism testing
If the proposed intermediate step were blocked, should the outcome change? This creates an intervention test for the mechanism.
64. Counterfactual reasoning helps decision-making
The decision-maker asks not which option has the best observed outcomes historically, but which action would produce the best outcome for the current target under plausible alternatives.
65. Counterfactual reasoning helps explain opportunity cost
Choosing one action means not choosing another. The relevant benefit is often incremental: outcome under chosen action minus outcome under the best feasible alternative.
66. Counterfactuals can be unrealistic
“What if this patient were ten years younger but everything else identical?” may be scientifically useful conceptually, but some attributes cannot be intervened upon cleanly.
67. Manipulability matters
Counterfactuals are clearest when they correspond to well-defined interventions. Some causal questions about intrinsic characteristics require more careful interpretation.
68. Counterfactual fairness is one AI-related application
A decision system can be examined by asking whether a person would receive the same outcome in a counterfactual world where a protected attribute changed while causally appropriate factors remained fixed. The formal details are complex, but the principle is a fairness-oriented what-if.
69. Counterfactual explanations can be useful for users
“If income were higher by X, the decision would change” describes a nearby alternative state. Such explanations should not be confused automatically with causal advice unless the variables are actionable and the model causal.
70. Prediction models can generate misleading counterfactual advice
A model associates one feature with success. Changing that feature may not cause success because the feature could be a proxy or consequence.
71. AI can help learners construct counterfactuals
Useful prompts include: “What is the unobserved alternative outcome in this experiment?”, “Which control group best approximates it?”, “List reasons the comparison group may fail as a counterfactual,” and “Turn this before-after story into a causal design.”
72. AI can invent counterfactual certainty
A language model may state what “would have happened” as if directly known. Counterfactual outcomes are inferred, not observed. Confidence should follow the design and assumptions.
73. Causal models can simulate interventions
Structural causal models encode relationships and can estimate outcomes under hypothetical interventions. The quality of the counterfactual depends on model correctness.
74. Digital twins are counterfactual tools
A validated digital model of a physical system can compare actual operation with hypothetical configurations. This is powerful in engineering, but model boundaries and calibration matter.
75. Counterfactuals can be prospective or retrospective
Prospective: what will happen if we choose Action A instead of B?
Retrospective: what would have happened if the past action had differed?
Both rely on causal models.
76. Retrospective counterfactuals invite hindsight bias
After a bad outcome, an alternative action can look obviously superior. But the decision-maker did not know the outcome then. Evaluate the counterfactual using information available at the time.
77. Near-miss analysis uses counterfactuals carefully
If a barrier had failed, would catastrophe have occurred? This can reveal system vulnerability even when the final outcome was safe.
78. Counterfactuals can be local
“What if the dose were slightly lower?”
Nearby alternatives are often easier to support because they remain close to observed conditions.
79. Counterfactuals can be distant
“What if the entire system operated under a completely different architecture?” Such comparisons can be useful for design but rely more heavily on modelling assumptions.
80. Distance from observed data matters
The farther the hypothetical world is from observed evidence, the more counterfactual prediction depends on extrapolation.
81. Boundary conditions constrain counterfactuals
A model valid at ordinary temperatures may not support counterfactuals at extreme temperatures. Article 90’s regime limits apply directly.
82. Spatial context constrains counterfactuals
Moving an intervention to another location changes neighbourhood, climate, infrastructure and population. The “same intervention” may not create the same outcome.
83. Time context constrains counterfactuals
A policy effective in one year may not produce the same effect later if background conditions changed.
84. Counterfactuals need consistency of units
When groups differ in hidden ways, comparing their outcomes can be equivalent to comparing different systems rather than alternative states of one target system.
85. Overlap supports credible comparison
If treated units are unlike every untreated unit, the counterfactual requires extrapolation. Good causal studies seek overlap in relevant baseline characteristics.
86. Positivity is a counterfactual support condition
For every relevant type of unit, there should be some possibility of receiving each treatment condition being compared. If one subgroup always receives treatment, its untreated outcome is difficult to infer from data.
87. Selection can destroy overlap
Only the most motivated students choose a programme. Untreated students with comparable motivation may be rare. The counterfactual becomes weak unless design addresses selection.
88. Missing data can distort counterfactual estimates
If outcomes are missing differently across treatment groups, the observed comparison may no longer represent the intended potential outcomes.
89. Sensitivity analysis asks how strong hidden bias must be
Suppose an unmeasured variable affects both treatment and outcome. How strong would it need to be to erase the estimated effect? This does not prove no hidden confounding exists, but it measures fragility.
90. Negative controls can test counterfactual assumptions
If an exposure appears to affect an outcome it could not plausibly cause, shared bias may be present. Negative controls can reveal hidden confounding or measurement pathways.
91. Placebo timing can test policy studies
Pretend the intervention occurred earlier. If the analysis finds a large “effect” before the real intervention, the design may be capturing background trends rather than causality.
92. Placebo locations can test spatial studies
Apply the same method to untreated locations. Spurious effects elsewhere suggest the causal comparison is not unique to the real treatment site.
93. Primary 3: ask “what if only this changed?”
Use paired objects or simple setups. The child learns to imagine the alternative condition while holding relevant background stable.
94. Primary 4: choose the control deliberately
For each experiment, ask which untreated or alternative condition best represents what would have happened otherwise.
95. Primary 5: identify bad counterfactuals
Compare a fertilised plant indoors with an unfertilised plant outdoors. Students should recognise that the alternative world changed more than one cause.
96. Primary 6: distinguish before-after from causal comparison
Ask which other changes over time could explain improvement. Add a control or repeated baseline to strengthen the counterfactual.
97. Secondary Science: formalise estimands and assumptions
Students can distinguish average effects, local effects, treatment-on-treated effects, confounding, overlap, interference and quasi-experimental designs.
98. Independent-attempt task 1: state the missing world
For a causal claim, write the actual outcome and the exact alternative outcome that cannot be observed directly.
99. Independent-attempt task 2: rank comparison groups
Create three possible controls. Rank them by how well they approximate the missing counterfactual and explain which differences threaten the comparison.
100. Independent-attempt task 3: repair a before-after story
Take a claim based only on improvement after an intervention. Add one design feature—randomisation, control group, interrupted time series or another valid strategy—to improve causal credibility.
101. Independent-attempt task 4: identify interference
Construct a classroom example where one student’s treatment changes another student’s outcome. Explain why ordinary individual-level counterfactuals become incomplete.
102. Diagnostic error: actual world compared with an impossible world
The alternative changes many conditions simultaneously. Repair by specifying a plausible intervention that changes the causal factor of interest while preserving relevant background.
103. Diagnostic error: counterfactual equals prediction
The learner predicts what happens next but does not specify an intervention. Repair by contrasting outcomes under two causal conditions.
104. Diagnostic error: matching on outcomes
Comparison units are selected using information affected by treatment. This can create bias. Counterfactual comparability should be based on pre-treatment characteristics.
105. Diagnostic error: controlling for a mediator
The learner adjusts away the process through which treatment acts, then claims to estimate the total effect. Repair by matching the adjustment set to the estimand.
106. Diagnostic error: no overlap
Treated and untreated populations occupy completely different ranges. Repair by narrowing the target population or acknowledging extrapolation.
107. Diagnostic error: certainty about one individual
Population evidence is used to declare exactly what would have happened to one person. Repair by distinguishing average causal effects from individual counterfactual certainty.
108. Parents can use counterfactual reasoning in learning decisions
A child improves after tuition. Ask what would likely have happened without it, using school trajectory, prior performance and independent work rather than assuming all improvement came from tuition.
109. Tutors can use counterfactual reasoning in intervention design
Change one support at a time where practical. If performance improves only while prompts remain, the causal effect may be assistance rather than independent learning.
110. The independence test is counterfactual
What would the learner do if the tutor were not present? That alternative condition reveals whether performance belongs to the learner or to the combined learner-plus-support system.
111. Examination transfer is a counterfactual test
If the surface context changed but the underlying concept stayed the same, would the learner still succeed? Transfer estimates performance under an alternative presentation.
112. The evidence boundary
Counterfactual claims are never observed directly. Their credibility comes from design, assumptions, overlap, mechanism, sensitivity analysis and replication. Strong causal language should be proportional to the strength of that reconstruction.
113. A compact counterfactual checklist
- What is the actual causal condition?
- What alternative condition is being imagined?
- What outcome is observed?
- What outcome is counterfactual?
- What comparison strategy approximates the missing world?
- Why is that comparison credible?
- What baseline differences threaten exchangeability?
- Could confounding remain?
- Is the intervention well defined?
- Could treatment versions differ?
- Could one unit’s treatment affect another?
- Is there adequate overlap?
- What estimand is being targeted?
- How sensitive is the result to hidden bias?
- What evidence would strengthen or weaken the counterfactual claim?
114. Frequently asked questions
What is scientific counterfactual reasoning?
It is reasoning about the outcome that would have occurred for the same target under an alternative causal condition, forming the conceptual basis of causal-effect estimation.
Why can’t we observe the counterfactual directly?
Because one unit at one time can experience only one of the competing causal conditions.
How do experiments solve this problem?
Randomisation creates groups that, on average, can stand in for one another’s unobserved alternative outcomes.
Can observational studies estimate counterfactuals?
Yes, using matching, adjustment, weighting, natural experiments and other designs, but the credibility depends on stronger assumptions.
How does counterfactual reasoning help PSLE Science?
It deepens fair-test logic by asking what would have happened if the independent variable had not changed while relevant background stayed the same.
How does it deepen in Secondary Science?
Students can connect controls, confounding, randomisation, quasi-experiments, treatment effects and causal assumptions more formally.
115. Continue the Science Education Systems series
- How Scientific Causality Works
- How Scientific Comparison Works
- How Scientific Randomisation Works
- How Scientific Confounding Works
Conclusion: Causality lives in the difference between what happened and what would otherwise have happened
Maya sees the outcome.
Jia Jun asks for the comparison.
Hana tests whether the alternative world is believable.
Ethan asks what assumptions hold the counterfactual together.
Science needs all four.
Name the actual condition.
name the alternative.
build the most credible comparison available.
protect it from confounding.
state the assumptions.
test sensitivity.
Then let causal claims grow only as strong as the counterfactual world supporting them.
