Science Education Systems · Article 80. Maya, Jia Jun, Hana and Ethan remain fictional Punggol learners. This article follows the Registered Reports layer: how Science can review the question, method and analysis plan before the results exist, so publication depends less on whether the outcome is exciting.
The 50-second parent route
Traditional publishing often evaluates a finished study after the results are known.
Registered Reports move a major part of peer review earlier.
The route is:
question → theory → protocol → analysis plan → Stage 1 peer review → in-principle acceptance → data collection → planned analysis → transparent deviations → Stage 2 review → publication
The key question is:
Can we judge the scientific value of the study before knowing whether the result is positive, negative or surprising?
This article completes Articles 77–80 after How Scientific Systematic Review Works, How Scientific Meta-Analysis Works and How Scientific Preregistration Works.
1. Registered Reports change the order of evaluation
Instead of waiting for results, reviewers first assess the research question, rationale, method and planned analysis.
This moves scientific attention upstream.
2. Stage 1 happens before results are known
The manuscript normally contains:
background.
research question.
hypotheses.
methods.
sampling plan.
analysis plan.
Reviewers judge whether the study can answer the question well.
3. In-principle acceptance changes incentives
If the Stage 1 plan is accepted, the journal commits in principle to publishing the study provided the approved protocol is followed sufficiently and the final report remains scientifically sound.
The direction of the result becomes less important to publication.
4. Maya’s Registered Reports error is thinking results no longer matter
Her repair:
results still matter scientifically.
What changes is whether surprising or statistically significant results are required to earn publication.
5. Jia Jun’s error is treating in-principle acceptance as guaranteed publication
His repair:
Stage 2 still checks adherence, transparency, interpretation and whether the work meets the agreed scientific standard.
6. Hana’s error is assuming no deviations are allowed
Unexpected problems arise in real research.
Her repair:
document deviations clearly, distinguish them from the approved plan and explain their consequences.
7. Ethan’s error is trying to hide exploratory findings
His repair:
exploratory analyses can still be reported when labelled honestly rather than presented as preregistered predictions.
8. Registered Reports extend preregistration
Preregistration timestamps the plan.
Registered Reports add external peer review before the results.
This gives the plan a stronger challenge before data collection begins.
9. Reviewers can improve the study before it becomes irreversible
Sample too small?
outcome poorly defined?
control condition weak?
analysis mismatched?
At Stage 1, these can still be repaired.
10. Traditional post-result review can be too late for design repair
Once data has been collected, reviewers can identify design problems but cannot always fix them.
Registered Reports move critique to the point where it can change the experiment.
11. Registered Reports can reduce publication bias
If publication is conditionally committed before results are known, null or negative findings are less likely to disappear simply because they are less exciting.
12. This can improve the visible scientific record
The literature becomes less filtered by outcome direction.
Readers see more of the full testing universe.
13. Registered Reports can reduce selective outcome reporting
Primary outcomes are defined before results.
Switching attention to a more favourable outcome later becomes visible.
14. They can reduce HARKing
Hypotheses exist before data collection.
Post-hoc explanations can still be discussed, but they cannot masquerade easily as original predictions.
15. They can reduce undisclosed p-hacking
The analysis path is reviewed beforehand.
Trying many alternate models later becomes a documented deviation or exploratory branch rather than invisible flexibility.
16. They can improve statistical power planning
Reviewers can challenge whether the planned sample is capable of resolving the effect that actually matters.
See How Scientific Statistical Power Works.
17. They can improve measurement quality
Are the instruments calibrated?
are outcomes reliable?
are operational definitions clear?
Review happens while those choices can still be improved.
18. They can improve internal validity
Randomisation.
controls.
blinding.
confounding.
attrition plans.
Reviewers can inspect threats before the experiment begins.
19. Registered Reports reward question quality
A scientifically important question can deserve publication even if the result later turns out to be null.
This shifts prestige away from outcome surprise toward methodological value.
20. Null results can become more informative
If the study was adequately powered, well measured and reviewed beforehand, a null result can rule out meaningful effects more convincingly than a weak post-hoc study.
21. Negative results can prevent repeated wasted effort
If a carefully designed test fails, future researchers learn that the proposed mechanism may be weak, context-specific or absent.
Scientific progress includes finding what does not work.
22. Surprising positive results also become more credible
Because the plan was fixed before outcome knowledge, an unexpected strong effect is less likely to be an artefact of selective analysis.
23. Stage 1 review should examine theory
Why should the effect exist?
What mechanism connects variables?
What alternative explanations matter?
A Registered Report is not merely a statistics form.
24. Stage 1 review should examine falsifiability
What result would challenge the hypothesis?
If every possible outcome can be explained afterward, the test is weak.
25. Stage 1 review should examine the primary outcome
Does the chosen measure actually represent the scientific construct?
Could ceiling, floor or measurement bias hide the effect?
26. Stage 1 review should examine the comparison
Is the control appropriate?
Is randomisation at the right level?
Could contamination occur?
The causal contrast should be credible before approval.
27. Stage 1 review should examine the analysis plan
Which model?
which exclusions?
which transformation?
which multiplicity correction?
which stopping rule?
The plan should be detailed enough to constrain hidden flexibility.
28. Stage 1 review should examine feasibility
Can enough participants be recruited?
can instruments measure the effect?
can the protocol be followed?
A beautiful impossible study is not useful.
29. Pilot data can inform Stage 1 design
Preliminary work may estimate variability, test procedures or reveal logistical problems.
Pilot results should not be used selectively to manufacture an overconfident target effect.
30. In-principle acceptance protects against outcome-based rejection
Once the design has been judged worthwhile, an unexciting result should not become a reason to discard the study solely because it lacks novelty.
31. Stage 2 asks whether the study followed the approved plan
Were the methods implemented?
were deviations disclosed?
were planned analyses reported?
was interpretation proportional to evidence?
32. Stage 2 should not reward statistical significance
The result may be positive.
null.
negative.
complex.
The publication decision should rest on adherence and scientific quality rather than desired direction.
33. Deviations can be legitimate
Equipment fails.
a pandemic interrupts recruitment.
software assumptions break.
unexpected missing data appears.
Science needs flexibility with provenance.
34. Deviations should be separated from the confirmatory track
Original plan.
necessary change.
exploratory addition.
Each should be labelled so readers understand which conclusions were prespecified.
35. Registered Reports do not eliminate bias
Poor theories can be registered.
weak measures can be approved.
implementation can fail.
reviewers can make mistakes.
The format reduces particular incentives; it is not a guarantee of truth.
36. Registered Reports do not eliminate publication bias completely
Only studies entering the Registered Reports pathway receive its protection.
Other research may still be selected by outcome.
Participation itself can be selective.
37. Registered Reports take more planning
The research question and method must mature before data collection.
This can slow impulsive experiments but improve design discipline.
38. They may be less suited to purely exploratory work
Some research begins without a precise hypothesis because the aim is discovery, mapping or method development.
Exploratory Science remains essential and should use transparent reporting suited to its purpose.
39. Registered Reports can still include exploratory analyses
The confirmatory core is protected.
Unexpected findings can be added as exploratory results.
The label tells future researchers what needs independent confirmation.
40. Primary Science can learn the same logic simply
Before the experiment:
show the teacher the question, prediction, variables and fair-test method.
Improve the design first.
Then run the test.
The answer does not determine whether the planning was good.
41. Primary 3 can separate planning from outcome
A prediction can be wrong while the experiment is excellent.
That is a powerful scientific lesson.
42. Primary 4 can receive design feedback before testing
Teacher asks:
What changes?
What stays the same?
How will you measure?
Students repair the method before collecting evidence.
43. Primary 5 can learn that null results are still results
If two conditions show no meaningful difference under a strong fair test, that outcome belongs in the scientific record.
Do not hide it because the prediction failed.
44. Primary 6 can label exploratory follow-ups
An unexpected pattern appears.
Students write a new hypothesis and plan a second experiment rather than pretending the first experiment predicted it.
45. Secondary Science can formalise the two-stage logic
Stage 1:
question, rationale, design and analysis.
Stage 2:
results, deviations, interpretation and reproducibility.
Students see publishing as part of experimental design.
46. Registered Reports and peer review are inseparable
Peer review moves from evaluating a finished story to helping shape the test itself.
See How Scientific Peer Review Works.
47. Registered Reports and preregistration are related but not identical
Preregistration records the plan publicly or privately before outcome knowledge.
Registered Reports add editorial and peer-review commitment around that plan.
48. Registered Reports and statistical power are connected
Reviewers can challenge whether the planned design has enough information to distinguish scientifically meaningful effects from noise.
49. Registered Reports and multiple comparisons are connected
Primary outcomes and analysis families are declared before results.
This exposes the testing denominator and supports appropriate correction.
50. Registered Reports and effect size are connected
Planning can be built around the smallest scientifically meaningful effect rather than whatever magnitude happens to become significant later.
51. Registered Reports and confidence intervals are connected
A study can be designed for precision:
What interval width would let us distinguish meaningful from trivial effects?
This can be more informative than binary significance planning.
52. Registered Reports and replication are natural partners
A surprising exploratory finding can motivate a preregistered replication submitted as a Registered Report.
The second study becomes a hard confirmatory test.
53. Registered Reports can strengthen systematic reviews later
Because null findings are less likely to disappear, the future evidence base may become less distorted by publication bias.
54. Registered Reports can improve meta-analysis
More complete outcome reporting and less selective publication reduce one source of pooled-effect distortion.
55. Registered Reports improve scientific provenance
Original plan.
peer-review revisions.
accepted protocol.
deviations.
final analysis.
The research history becomes easier to reconstruct.
56. Version control is essential
Which protocol version received in-principle acceptance?
Which changes came later?
Without version history, the protection against hindsight weakens.
57. AI can help draft Stage 1 protocols
Useful prompts:
“List threats to internal validity before data collection.”
“Identify ambiguous analysis choices.”
“Check whether the primary outcome matches the hypothesis.”
“Generate a deviation log structure.”
58. AI can help simulate the proposed analysis
Generate synthetic data under:
null effect.
small effect.
large effect.
missing data.
outliers.
Test whether the analysis pipeline behaves as expected before real outcomes are available.
59. AI can also create protocol bloat
A model can generate impressive-looking methods sections containing unnecessary analyses and vague contingencies.
Every planned step should serve the scientific question.
60. AI can silently rewrite the approved plan
Automated editing after data collection can blur what was prespecified.
Versioned documents and frozen protocols protect the confirmatory boundary.
61. AI-assisted peer review should preserve human accountability
AI can flag missing controls or statistical inconsistencies.
But responsibility for scientific judgement remains with the researchers, editors and reviewers using the tools.
62. Registered Reports matter for AI benchmarking
Before evaluating a model, define:
datasets.
prompts.
metrics.
tool access.
number of runs.
stopping rules.
Freeze the evaluation plan before inspecting comparative results.
63. This reduces leaderboard fishing
If model settings are changed repeatedly after seeing benchmark scores, the benchmark becomes part of training.
Pre-reviewed evaluation protocols can create cleaner tests.
64. Parents can teach the principle through learning experiments
Before trying a new revision method:
define the subject.
time spent.
comparison.
success measure.
Ask someone else whether the plan is fair.
Then run it.
65. Small-group tuition can use “review before results” tasks
Students receive an experimental plan without results.
They peer-review:
variables.
controls.
measurement.
sample size.
analysis.
Only after revisions are agreed are the results revealed.
66. This teaches a deep scientific value
A good experiment is good because of how it asks reality the question.
Not because reality happened to return an exciting answer.
67. A compact Registered Reports checklist
- Is the research question important and testable?
- Are hypotheses explicit?
- Is the primary outcome appropriate?
- Is the design internally valid?
- Are randomisation, controls and blinding adequate where relevant?
- Is the sample-size or precision plan justified?
- Are exclusions and stopping rules defined?
- Is multiplicity addressed?
- Can the analysis be reconstructed before results?
- Did Stage 1 review occur before outcome knowledge?
- Was in-principle acceptance granted?
- Were deviations documented transparently?
- Are confirmatory and exploratory analyses separated?
- Does Stage 2 interpretation match the evidence regardless of result direction?
68. Frequently asked questions
What is a Registered Report?
A Registered Report is a publishing format in which a study’s question, rationale, methods and analysis plan are peer reviewed before the results are known, with in-principle acceptance granted when the proposed study meets the required standard.
How is a Registered Report different from preregistration?
Preregistration records the plan before results. Registered Reports add formal pre-result peer review and a conditional publication commitment around the approved protocol.
Do Registered Reports guarantee publication?
No. Final publication still depends on following the approved protocol sufficiently, reporting deviations transparently and meeting Stage 2 scientific requirements.
Can Registered Reports publish null results?
Yes. Their design is intended to reduce dependence of publication on whether results are positive, negative or statistically significant.
Can researchers still explore unexpected findings?
Yes. Exploratory analyses can be reported when clearly distinguished from the prespecified confirmatory analyses.
How does this help students?
It teaches that experimental quality should be judged from the question, design and measurement before anyone knows whether the prediction succeeds.
69. Continue the Science Education Systems series
- How Scientific Systematic Review Works
- How Scientific Meta-Analysis Works
- How Scientific Preregistration Works
- How Scientific Peer Review Works
- How Scientific Replication Works
Conclusion: Review the question before the answer gets a chance to persuade you
Maya writes the hypothesis.
Jia Jun builds the analysis.
Hana challenges the method before data collection.
Ethan records every later deviation.
Science needs all four.
Judge the question.
repair the design.
freeze the confirmatory plan.
collect the evidence.
publish what reality returns.
That is the promise of Registered Reports:
methodological quality before outcome drama.
