eduKateSG · Why Science?
Benchmark the whole evidence pipeline—not only the final rank—so a fair comparison keeps the cohort, ontology, knowledge release, runner and metric visible
Choose the closest route, then return to the full index whenever you need the wider evidence chain.
Science learning becomes powerful when students can tell a measured pattern from a model, a statistical decision from a mechanism, and a useful lead from a finished conclusion. PhEval, the Phenotypic Inference Evaluation Framework, is a modular open-source framework for comparing phenotype-driven gene, disease and variant prioritisation methods. Its current documentation separates data preparation, tool execution and benchmark analysis; plugins adapt individual tools into standardised runners and outputs. PhEval can compare tool versions, ontology releases, knowledge updates and input cohorts using reproducible corpora and metrics. A benchmark result describes performance in that experimental setting; it does not prove universal superiority or clinical safety. This guide supports Primary Science, PSLE Science, Secondary Science, O-Level Science and STEM education as a longform evidence-reading route; it is not medical advice, a laboratory protocol, an admission promise or a career guarantee.
Related eduKate reading: Human Phenotype Ontology and structured clinical phenotypes; DECIPHER and phenotype-linked variant sharing; Matchmaker Exchange and federated rare-disease case matching; Science Learning Hub; Education Hub; STEM Education; Careers by Subject Capability.
Primary and official sources checked on 12 October 2026: Official PhEval documentation observed 12 October 2026; Official PhEval repository; PhEval primary study in BMC Bioinformatics; Phenopacket Store corpus paper; 2026 Singapore–Cambridge O-Level Biology syllabus; MOE G2/G3 Lower Secondary Science teaching and learning syllabus; 2026 MOE G2 Computing syllabus. Resource interfaces and recommendations can change, so preserve the release, access date and documentation used.
| Stage | Question | Evidence | Responsible output |
|---|---|---|---|
| Prepare | Which cases, phenotypes, variants and ground truth define the experiment? | Versioned Phenopackets, VCF references and corpus manifest | A governed benchmark corpus |
| Run | How is each tool configured and executed? | Plugin, runner, container or environment and parameters | Traceable standardised outputs |
| Measure | How often and how highly is the known entity recovered? | Rank, found or missed status and declared metrics | Comparable benchmark results |
| Interpret | Where do findings generalise and where do they not? | Subgroups, sensitivity analyses, knowledge dates and limits | A bounded performance claim |
Inside this guide
1–12 · Foundations and design
- 1. Start with the benchmarking question
- 2. Understand the moving parts
- 3. Separate preparation, execution and analysis
- 4. Use standardised case representations
- 5. Define the ground truth
- 6. Choose real or simulated corpora carefully
- 7. Keep the cohort manifest
- 8. Prepare phenotype noise deliberately
- 9. Prepare genomic inputs consistently
- 10. Understand plugins and runners
- 11. Pin tool configurations
- 12. Pin the environment
13–24 · Models and estimation
- 13. Standardise outputs
- 14. Separate gene, disease and variant tasks
- 15. Read top-k metrics
- 16. Read found and missed counts
- 17. Use rank distributions
- 18. Add uncertainty to comparisons
- 19. Stratify before claiming generality
- 20. Audit ontology versions
- 21. Audit knowledge releases
- 22. Avoid case leakage
- 23. Avoid parameter shopping
- 24. Report failures and timeouts
25–36 · Diagnostics and interpretation
- 25. Measure resources when relevant
- 26. Distinguish reproducibility from validity
- 27. Use bounded clinical language
- 28. Connect to Primary Science
- 29. Connect to Secondary Science
- 30. Connect to mathematics
- 31. Connect to computing
- 32. Connect to pathways
- 33. Did you know?
- 34. Write a reader-ready benchmark
- 35. Build a benchmark audit
- 36. Finish with a reproducible bundle
Section 1 of 36
1. Start with the benchmarking question
PhEval evaluates tools only after the task, corpus, knowledge date and performance measure have been defined.
Working checkpoint — benchmark corpus: Write the comparison claim before running software. Keep the corpus, truth labels, Phenopackets, configurations, plugins, environments, knowledge releases and metrics together.
For a classroom model, use fictional competitors and a fixed course to teach fair comparison.
Section 2 of 36
2. Understand the moving parts
Phenotype representation, ontology structure, disease mappings, algorithms and cohorts can all change measured performance.
Working checkpoint — Phenopacket: List every layer that differs between experiments. Read performance beside cohort composition, uncertainty, failures and subgroup behaviour.
For future study, connect benchmarking to biology, statistics and software engineering while checking official pathways separately.
Section 3 of 36
3. Separate preparation, execution and analysis
The framework deliberately divides data preparation, tool running and benchmarking so errors and changes can be located.
Working checkpoint — tool runner: Archive outputs at each boundary. Preserve missed cases, exclusions and timeouts; a success percentage is not the complete benchmark.
For scientific writing, report the corpus and versions whenever one tool is described as better.
Section 4 of 36
4. Use standardised case representations
Phenopackets can represent case-level phenotype and related genomic context across tools.
Working checkpoint — standardised output: Validate schema, identifiers and required fields. Separate what the benchmark measures from claims about clinical safety, generality or patient benefit.
For students, ask whether every tool ran on the same cases, inputs, rules and finish line.
Section 5 of 36
5. Define the ground truth
A benchmark needs a reference diagnosis, gene or variant, but published truth can itself be incomplete or revised.
Working checkpoint — rank metric: Record provenance and review date for every label. Treat identifiers, knowledge dates, runners and denominators as scientific inputs: a silent mismatch can reverse a comparison.
For parents, ask whether benchmark cases resemble the real problem and whether failures remained in the denominator.
Section 6 of 36
6. Choose real or simulated corpora carefully
Real cases carry authentic complexity and governance limits; simulations offer control but can simplify the problem.
Working checkpoint — reproducible evaluation: State which properties of reality the corpus preserves. Keep the corpus, truth labels, Phenopackets, configurations, plugins, environments, knowledge releases and metrics together.
For a classroom model, use fictional competitors and a fixed course to teach fair comparison.
Section 7 of 36
7. Keep the cohort manifest
Cases removed for format, missing truth or tool incompatibility change the evaluation population.
Working checkpoint — benchmark corpus: Publish counts and exclusion reasons. Read performance beside cohort composition, uncertainty, failures and subgroup behaviour.
For future study, connect benchmarking to biology, statistics and software engineering while checking official pathways separately.
Section 8 of 36
8. Prepare phenotype noise deliberately
Missing, imprecise or distracting terms can test robustness, but the noise model must be documented.
Working checkpoint — Phenopacket: Separate observed noise from synthetic perturbation. Preserve missed cases, exclusions and timeouts; a success percentage is not the complete benchmark.
For scientific writing, report the corpus and versions whenever one tool is described as better.
Section 9 of 36
9. Prepare genomic inputs consistently
Variant files, assemblies, pedigree structures and callable regions need aligned preprocessing across tools.
Working checkpoint — tool runner: Do not give one tool cleaner inputs than another. Separate what the benchmark measures from claims about clinical safety, generality or patient benefit.
For students, ask whether every tool ran on the same cases, inputs, rules and finish line.
Section 10 of 36
10. Understand plugins and runners
PhEval plugins translate standard benchmark inputs into tool-specific execution and convert outputs back to common results.
Working checkpoint — standardised output: Review each adapter for information loss. Treat identifiers, knowledge dates, runners and denominators as scientific inputs: a silent mismatch can reverse a comparison.
For parents, ask whether benchmark cases resemble the real problem and whether failures remained in the denominator.
Section 11 of 36
11. Pin tool configurations
Defaults, thresholds, databases and resources can change rankings more than the software name suggests.
Working checkpoint — rank metric: Version the full runner configuration. Keep the corpus, truth labels, Phenopackets, configurations, plugins, environments, knowledge releases and metrics together.
For a classroom model, use fictional competitors and a fixed course to teach fair comparison.
Section 12 of 36
12. Pin the environment
Libraries, containers, hardware and random seeds affect whether a run can be repeated.
Working checkpoint — reproducible evaluation: Record the executable environment and checksums. Read performance beside cohort composition, uncertainty, failures and subgroup behaviour.
For future study, connect benchmarking to biology, statistics and software engineering while checking official pathways separately.
Section 13 of 36
13. Standardise outputs
A common result schema lets rank and recovery metrics be computed consistently across different tools.
Working checkpoint — benchmark corpus: Validate gene, disease and variant identifiers after conversion. Preserve missed cases, exclusions and timeouts; a success percentage is not the complete benchmark.
For scientific writing, report the corpus and versions whenever one tool is described as better.
Section 14 of 36
14. Separate gene, disease and variant tasks
A tool that ranks diseases answers a different question from one that ranks genes or variants.
Working checkpoint — Phenopacket: Compare like target entities. Separate what the benchmark measures from claims about clinical safety, generality or patient benefit.
For students, ask whether every tool ran on the same cases, inputs, rules and finish line.
Section 15 of 36
15. Read top-k metrics
Top 1, 3, 5 or 10 recovery describes whether the reference entity appears within a cutoff, not how calibrated a score is.
Working checkpoint — tool runner: Report all planned cutoffs and denominators. Treat identifiers, knowledge dates, runners and denominators as scientific inputs: a silent mismatch can reverse a comparison.
For parents, ask whether benchmark cases resemble the real problem and whether failures remained in the denominator.
Section 16 of 36
16. Read found and missed counts
A missed entity can reflect filtering, mapping, absent knowledge or algorithmic ranking.
Working checkpoint — standardised output: Classify the failure mechanism. Keep the corpus, truth labels, Phenopackets, configurations, plugins, environments, knowledge releases and metrics together.
For a classroom model, use fictional competitors and a fixed course to teach fair comparison.
Section 17 of 36
17. Use rank distributions
Median or full rank distributions reveal performance beyond a single success threshold.
Working checkpoint — rank metric: Show distributions and not only one average. Read performance beside cohort composition, uncertainty, failures and subgroup behaviour.
For future study, connect benchmarking to biology, statistics and software engineering while checking official pathways separately.
Section 18 of 36
18. Add uncertainty to comparisons
Finite cohorts create sampling uncertainty, especially for rare subgroups.
Working checkpoint — reproducible evaluation: Use confidence intervals or resampling where justified. Preserve missed cases, exclusions and timeouts; a success percentage is not the complete benchmark.
For scientific writing, report the corpus and versions whenever one tool is described as better.
Section 19 of 36
19. Stratify before claiming generality
Performance may differ by inheritance, disease class, phenotype completeness, variant type or known-versus-novel gene status.
Working checkpoint — benchmark corpus: Predefine subgroup analyses. Separate what the benchmark measures from claims about clinical safety, generality or patient benefit.
For students, ask whether every tool ran on the same cases, inputs, rules and finish line.
Section 20 of 36
20. Audit ontology versions
Changing HPO structure or annotations can improve or worsen a tool without any code change.
Working checkpoint — Phenopacket: Hold one layer fixed while testing another. Treat identifiers, knowledge dates, runners and denominators as scientific inputs: a silent mismatch can reverse a comparison.
For parents, ask whether benchmark cases resemble the real problem and whether failures remained in the denominator.
Section 21 of 36
21. Audit knowledge releases
A tool evaluated with knowledge published after the cases gains an advantage unavailable at the historical decision point.
Working checkpoint — tool runner: Use dated knowledge snapshots for temporal claims. Keep the corpus, truth labels, Phenopackets, configurations, plugins, environments, knowledge releases and metrics together.
For a classroom model, use fictional competitors and a fixed course to teach fair comparison.
Section 22 of 36
22. Avoid case leakage
Duplicate or republished cases can appear across training, tuning and testing resources.
Working checkpoint — standardised output: Deduplicate people and publications. Read performance beside cohort composition, uncertainty, failures and subgroup behaviour.
For future study, connect benchmarking to biology, statistics and software engineering while checking official pathways separately.
Section 23 of 36
23. Avoid parameter shopping
Choosing the best configuration after inspecting the test set overfits the benchmark.
Working checkpoint — rank metric: Separate tuning and final evaluation. Preserve missed cases, exclusions and timeouts; a success percentage is not the complete benchmark.
For scientific writing, report the corpus and versions whenever one tool is described as better.
Section 24 of 36
24. Report failures and timeouts
Crashes, malformed outputs and timeouts are part of operational performance, not rows to quietly delete.
Working checkpoint — reproducible evaluation: Keep a complete run-status table. Separate what the benchmark measures from claims about clinical safety, generality or patient benefit.
For students, ask whether every tool ran on the same cases, inputs, rules and finish line.
Section 25 of 36
25. Measure resources when relevant
Runtime and memory can affect whether a method is usable at scale, but hardware and caching must be comparable.
Working checkpoint — benchmark corpus: Record machine and warm-cache conditions. Treat identifiers, knowledge dates, runners and denominators as scientific inputs: a silent mismatch can reverse a comparison.
For parents, ask whether benchmark cases resemble the real problem and whether failures remained in the denominator.
Section 26 of 36
26. Distinguish reproducibility from validity
A perfectly repeatable benchmark can still use unrealistic cases, biased truth or an irrelevant metric.
Working checkpoint — Phenopacket: Challenge the study design as well as the code. Keep the corpus, truth labels, Phenopackets, configurations, plugins, environments, knowledge releases and metrics together.
For a classroom model, use fictional competitors and a fixed course to teach fair comparison.
Section 27 of 36
27. Use bounded clinical language
PhEval measures configured tools on defined corpora; it does not certify diagnostic use or patient benefit.
Working checkpoint — tool runner: Write “performed better in this benchmark”. Read performance beside cohort composition, uncertainty, failures and subgroup behaviour.
For future study, connect benchmarking to biology, statistics and software engineering while checking official pathways separately.
Section 28 of 36
28. Connect to Primary Science
A fair fictional race keeps the course, starting point and finish rule the same for every competitor.
Working checkpoint — standardised output: Use the analogy to teach controlled comparison. Preserve missed cases, exclusions and timeouts; a success percentage is not the complete benchmark.
For scientific writing, report the corpus and versions whenever one tool is described as better.
Section 29 of 36
29. Connect to Secondary Science
Variables, controls, repeated trials and measurement validity map directly onto computational benchmarking.
Working checkpoint — rank metric: Identify independent, dependent and controlled variables. Separate what the benchmark measures from claims about clinical safety, generality or patient benefit.
For students, ask whether every tool ran on the same cases, inputs, rules and finish line.
Section 30 of 36
30. Connect to mathematics
Ranks, proportions, uncertainty intervals and subgroup denominators turn tool comparison into statistical reasoning.
Working checkpoint — reproducible evaluation: Recalculate a rate after restoring excluded failures. Treat identifiers, knowledge dates, runners and denominators as scientific inputs: a silent mismatch can reverse a comparison.
For parents, ask whether benchmark cases resemble the real problem and whether failures remained in the denominator.
Section 31 of 36
31. Connect to computing
Schemas, plugins, containers, logs, checksums and automation are part of reproducible experimental method.
Working checkpoint — benchmark corpus: Treat adapters as tested scientific code. Keep the corpus, truth labels, Phenopackets, configurations, plugins, environments, knowledge releases and metrics together.
For a classroom model, use fictional competitors and a fixed course to teach fair comparison.
Section 32 of 36
32. Connect to pathways
Benchmarking links biology, statistics, software engineering, ontology work and data governance and can illuminate STEM pathways.
Working checkpoint — Phenopacket: Check official education and professional requirements separately. Read performance beside cohort composition, uncertainty, failures and subgroup behaviour.
For future study, connect benchmarking to biology, statistics and software engineering while checking official pathways separately.
Section 33 of 36
33. Did you know?
PhEval is tool-agnostic: plugins prepare individual tools to consume standard inputs and return standardised outputs for comparison.
Working checkpoint — tool runner: Inspect the plugin, not only the core framework. Preserve missed cases, exclusions and timeouts; a success percentage is not the complete benchmark.
For scientific writing, report the corpus and versions whenever one tool is described as better.
Section 34 of 36
34. Write a reader-ready benchmark
Name the task, corpus, truth source, dates, tool and knowledge versions, configurations, metrics, uncertainty, failures and scope.
Working checkpoint — standardised output: Make every comparison auditable. Separate what the benchmark measures from claims about clinical safety, generality or patient benefit.
For students, ask whether every tool ran on the same cases, inputs, rules and finish line.
Section 35 of 36
35. Build a benchmark audit
Trace one case from Phenopacket through runner input, raw tool output, standardised result and metric calculation.
Working checkpoint — rank metric: Check that the reference entity survives every conversion. Treat identifiers, knowledge dates, runners and denominators as scientific inputs: a silent mismatch can reverse a comparison.
For parents, ask whether benchmark cases resemble the real problem and whether failures remained in the denominator.
Section 36 of 36
36. Finish with a reproducible bundle
Deliver the governed corpus manifest, configurations, environments, plugins, logs, raw and standard outputs, metrics, plots and limitations.
Working checkpoint — reproducible evaluation: Make rerunning and challenging the benchmark possible. Keep the corpus, truth labels, Phenopackets, configurations, plugins, environments, knowledge releases and metrics together.
For a classroom model, use fictional competitors and a fixed course to teach fair comparison.
Contents · Previous section · Continue to the Science Learning Hub