Science Education Systems · Article 77. Maya, Jia Jun, Hana and Ethan remain fictional Punggol learners. This article follows the systematic-review layer: how Science searches for the whole relevant evidence base before deciding what the literature actually supports.
The 50-second parent route
One paper can be interesting.
A systematic review asks what all relevant studies say together.
The route is:
question → protocol → search strategy → eligibility rules → screening → data extraction → risk-of-bias assessment → synthesis → certainty → update
The key question is:
Did we examine the evidence systematically enough that the conclusion does not depend on which studies we happened to notice?
This article extends How Scientific Synthesis Works, How Scientific Provenance Works and How Scientific Bias Works.
1. A systematic review begins with a defined question
Population.
phenomenon or intervention.
comparison.
outcome.
study design.
The exact structure varies by field, but the question must be narrow enough to search and broad enough to matter.
2. The review question controls everything downstream
Which databases?
which keywords?
which dates?
which study types?
which outcomes?
Search strategy and eligibility rules should follow the question rather than be adjusted to favour a conclusion.
3. Maya’s review error is searching until she finds agreement
She stops after three papers support her preferred idea.
Her repair:
define the search before knowing which direction the evidence will point.
4. Jia Jun’s review error is keyword literalism
He searches only one phrase.
Relevant studies use synonyms and older terminology.
His repair:
build a vocabulary map before searching.
5. Hana’s review error is treating every paper equally
A tiny uncontrolled study and a large well-designed trial receive the same weight in her narrative.
Her repair:
assess design quality and risk of bias before synthesis.
6. Ethan’s review error is turning the search into a literature dump
He collects hundreds of citations without connecting them to the question.
His repair:
extract the specific evidence needed for a structured comparison.
7. A protocol protects the review from hindsight
Before screening results, record:
the question.
eligibility criteria.
search sources.
main outcomes.
planned synthesis.
Protocol-first review reduces opportunity for selective inclusion.
8. Search strategy should maximise relevant recall
Missing important studies creates evidence bias.
Strong searches use:
synonyms.
controlled vocabulary where available.
multiple databases.
citation tracing.
field-specific sources.
9. Search precision matters too
A search returning one million irrelevant papers creates a screening problem.
The aim is not maximum volume.
It is comprehensive retrieval of relevant evidence.
10. Search dates should be explicit
A review describes evidence available up to a particular date.
Science continues afterward.
A review can become outdated even if it was excellent when completed.
11. Language restrictions can create bias
If only one language is included for convenience, evidence from other regions may disappear.
Sometimes restrictions are necessary, but they should be disclosed and interpreted.
12. Publication status matters
Published papers are easier to find than unpublished or difficult-to-access results.
If positive findings are more likely to be published, the visible literature can overstate effects.
13. Grey literature can reduce publication bias
Theses.
conference reports.
registries.
technical reports.
preprints.
These sources may contain useful evidence, but quality assessment remains necessary.
14. Citation chaining finds studies around known studies
Backward:
Which sources did this paper cite?
Forward:
Which later papers cited this one?
Network searching can reveal evidence missed by keywords.
15. Eligibility criteria should be defined before screening
Who counts?
what intervention?
which comparator?
which outcomes?
which time period?
which study designs?
Criteria protect against choosing studies because of their results.
16. Screening usually happens in stages
Title and abstract screening narrows the pool.
Full-text screening determines final inclusion.
Exclusion reasons should be traceable.
17. Two reviewers can reduce idiosyncratic decisions
Independent screening followed by reconciliation makes inclusion less dependent on one person’s judgement.
Disagreements reveal ambiguous criteria.
18. Screening disagreement is useful information
If two reviewers repeatedly disagree about whether a study fits, the review question or eligibility criteria may be insufficiently clear.
The process exposes conceptual ambiguity.
19. Data extraction creates the evidence table
Study identity.
population.
sample size.
design.
exposure or treatment.
outcome.
effect estimate.
uncertainty.
bias notes.
Structured extraction makes studies comparable.
20. Extracting only headline conclusions is dangerous
“Study found benefit” hides:
effect size.
confidence interval.
population.
measurement method.
limitations.
Scientific synthesis needs the underlying quantities.
21. Risk of bias is not the same as journal prestige
A famous journal can publish a weak study.
A less famous source can contain strong methodology.
Assess the study itself.
22. Bias assessment should match the study design
Randomised experiments face one set of threats.
Observational studies face another.
Diagnostic studies have different problems again.
Evaluation should follow the causal and measurement structure.
23. Reviewers should distinguish risk of bias from imprecision
Bias shifts the estimate systematically.
Imprecision makes it uncertain.
A study can have low bias but wide uncertainty, or high bias with narrow uncertainty.
24. Heterogeneity asks why studies differ
Different populations.
different settings.
different doses.
different measurements.
different follow-up times.
Variation can be scientifically informative.
25. Narrative synthesis remains useful
Not every evidence base can or should be combined numerically.
When studies are too different, structured qualitative synthesis may be more honest than forcing a pooled number.
26. Meta-analysis is a separate quantitative layer
When effect estimates are sufficiently comparable, statistical pooling may summarise them.
The next article, How Scientific Meta-Analysis Works, follows that layer.
27. A systematic review does not require meta-analysis
Systematic review describes the process for finding and evaluating evidence.
Meta-analysis describes a statistical method for combining estimates.
One can exist without the other.
28. Meta-analysis without a systematic search can be misleading
Pooling only convenient studies produces a precise summary of an incomplete evidence base.
Search quality comes first.
29. Review-level publication bias can be investigated
Study registries.
small-study patterns.
funnel-plot asymmetry.
selective outcome reporting.
No single diagnostic proves publication bias, but several clues can strengthen concern.
30. Duplicate publications can double-count evidence
One trial may generate several papers.
If treated as separate studies, the same participants receive extra weight.
Provenance protects the synthesis.
31. Overlapping datasets create hidden dependence
Two large database studies may share many of the same records.
Counting them as fully independent can overstate evidence volume.
32. Retractions and corrections matter
A review should verify whether included studies have been corrected, withdrawn or materially updated.
Evidence status can change.
33. Systematic reviews need updating
New studies appear.
methods improve.
standards change.
A review is a timestamped synthesis, not permanent final truth.
34. Living reviews update continuously or periodically
For fast-moving topics, review teams can maintain search and synthesis workflows so new evidence enters as it appears.
This reduces the gap between literature change and synthesis update.
35. Primary Science can learn systematic-review thinking
Do not answer a question from the first website found.
Collect several sources.
check whether they are independent.
compare evidence quality.
The formal research method grows from this habit.
36. Primary 3 can compare multiple observations
One plant.
one day.
one observation.
Then compare several plants and several days.
Broader evidence changes confidence.
37. Primary 4 can practise source inclusion rules
Question:
Which materials are good conductors?
Collect only sources that actually measure or explain conductivity rather than any article mentioning the material.
Relevance becomes explicit.
38. Primary 5 can build an evidence table
Source.
claim.
method.
result.
limitation.
This teaches synthesis rather than copy-and-paste note taking.
39. Primary 6 can identify selective evidence
If a student uses only examples supporting the preferred explanation and ignores conflicting data, the review is biased.
Contradictory evidence belongs in the table too.
40. Secondary Science can formalise review methods
search terms.
screening criteria.
study designs.
risk of bias.
effect estimates.
heterogeneity.
Students can see Science as a cumulative literature rather than isolated textbook facts.
41. Systematic review and peer review are different
Peer review critiques one manuscript or research output before or after publication.
A systematic review synthesises a body of evidence using a defined protocol.
42. Systematic review and consensus are connected
Consensus should emerge from the pattern and quality of evidence, not from counting expert opinions alone.
Reviews provide one structured input into consensus formation.
43. Systematic review and triangulation are connected
A review can compare evidence from:
experiments.
observational studies.
field measurements.
mechanistic studies.
If independent methods converge, the synthesis strengthens.
44. Systematic review and provenance are inseparable
Every claim should be traceable to:
which study.
which dataset.
which outcome.
which analysis.
Evidence lineage prevents accidental duplication and distortion.
45. Search engines are useful but incomplete review tools
Ranking algorithms prioritise relevance, popularity and other signals.
They are not designed to guarantee exhaustive scientific retrieval.
Formal reviews use broader, documented search strategies.
46. AI can accelerate literature screening
Machine learning can help:
deduplicate records.
rank likely relevance.
extract structured fields.
flag missing information.
Human oversight remains important because exclusion mistakes can bias the evidence base.
47. AI can hallucinate studies
A fabricated citation has zero evidential value regardless of how plausible it looks.
Every included paper should be verified against a real source.
48. AI can hide dependence between sources
Ten summaries appear independent.
All may trace back to one trial.
Provenance checking is essential before counting evidence streams.
49. AI can help build search vocabularies
Useful prompts:
“List synonyms and historical terms for this concept.”
“Propose inclusion and exclusion criteria.”
“Convert these studies into a structured evidence table.”
“Identify which papers appear to share the same dataset.”
50. AI should not decide eligibility invisibly
If an automated system excludes records, the review should preserve enough information to audit how and why exclusions occurred.
Opacity can create silent selection bias.
51. Systematic reviews can still be wrong
Poor search.
ambiguous eligibility.
biased studies.
bad extraction.
inappropriate pooling.
selective interpretation.
The label “systematic” is not a guarantee of quality.
52. Review quality should be judged methodologically
Was the protocol clear?
search broad enough?
screening reproducible?
bias assessed?
heterogeneity respected?
conclusions proportional to evidence?
These questions matter more than authority alone.
53. Parents can teach systematic-review habits through ordinary claims
“This study method is the best.”
Ask:
According to which studies?
How were they selected?
Do other studies disagree?
Were the learners similar to you?
The child learns to look beyond one persuasive example.
54. Small-group tuition can run mini reviews
Give three students six short study summaries.
Ask them to:
define the question.
exclude irrelevant studies.
identify bias.
build an evidence table.
write a cautious synthesis.
The task trains scientific reading and reasoning simultaneously.
55. A compact systematic-review checklist
- What precise question is being reviewed?
- Was a protocol defined before screening?
- Which databases and sources were searched?
- Were synonyms and alternative terms included?
- What dates and languages were covered?
- Were grey literature and registries considered where relevant?
- What were the eligibility criteria?
- Were exclusions documented?
- Was study-level risk of bias assessed?
- Were duplicate or overlapping datasets identified?
- Was heterogeneity respected?
- Was quantitative pooling appropriate?
- How current is the review?
- Does the conclusion match the quality and breadth of evidence?
56. Frequently asked questions
What is a systematic review?
A systematic review is a structured synthesis that defines a question, searches for relevant studies using a documented strategy, applies prespecified eligibility criteria, assesses study quality and summarises the total evidence.
Is a systematic review the same as meta-analysis?
No. A systematic review describes how evidence is found and assessed. Meta-analysis is a statistical method for pooling sufficiently comparable effect estimates.
Can systematic reviews be biased?
Yes. Incomplete searches, selective eligibility rules, biased included studies, poor extraction and selective interpretation can all distort a review.
Why include unpublished evidence?
Because positive results may be more likely to appear in journals. Unpublished evidence can reduce publication bias when it is relevant and assessable.
How does systematic-review thinking help PSLE Science?
The formal method is advanced, but the habit teaches learners to compare several sources and pieces of evidence rather than trusting the first result they find.
How does it change in Secondary Science?
Students can evaluate search methods, study designs, bias, evidence tables, heterogeneity and the difference between narrative and quantitative synthesis.
57. Continue the Science Education Systems series
- How Scientific Meta-Analysis Works
- How Scientific Preregistration Works
- How Scientific Registered Reports Work
Conclusion: A review earns trust by making the search visible
Maya finds the first persuasive study.
Jia Jun expands the vocabulary.
Hana checks which studies deserve confidence.
Ethan traces whether apparently separate papers share one dataset.
Science needs all four.
Define the question.
search broadly.
screen transparently.
assess bias.
synthesise proportionally.
Then let the whole evidence base speak before one convenient paper becomes the story.
