Chapter map

1. 1. Why annotations disagree
2. 2. Initial classification tree
3. 3. Cross-prediction
4. 4. Confusion matrix as evidence
5. 5. Label matching
6. 6. Adding a child label
7. 7. Adding a new branch
8. 8. Preserving prior names
9. 9. Retraining on accumulated data
10. 10. Linear SVM nodes
11. 11. One-class SVM option
12. 12. PCA-domain reconstruction
13. 13. Neighbour-distance rejection
14. 14. Hierarchical F1
15. 15. A student progressive project
16. 16. Governance and careers

Visit the How Mathematics Works hub for the wider series and its connections to education, reasoning, science, computing, design and careers.

---

1. Why annotations disagree

The central mathematical idea is concrete. Datasets may use broad labels, fine labels or different names for overlapping biology. A progressive hierarchy must reconcile without flattening.

A concrete situation

One study says T cell; another says CD4 and CD8.

What the mathematics does not license

Naming differences are not always biological differences. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Preserve provenance beside every node. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

2. Initial classification tree

Here the geometry becomes operational. The first dataset establishes a root, internal categories and leaves with classifiers at relevant nodes.

A concrete situation

Immune splits into lymphoid and myeloid.

What the mathematics does not license

The starting dataset influences all later placements. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Repeat with alternative starting orders. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

3. Cross-prediction

This step deserves a careful pause. The current hierarchy predicts every cell in the incoming labelled dataset.

A concrete situation

New CD4 cells map mostly to the existing T-cell node.

What the mathematics does not license

Batch shift can look like label mismatch. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Integrate and audit before matching. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

4. Confusion matrix as evidence

The useful question is not only how to calculate, but what the calculation assumes. Rows of incoming labels and columns of predicted nodes quantify overlap between naming systems.

A concrete situation

Eighty percent of label A lands in node B.

What the mathematics does not license

Counts depend on class size and threshold. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Inspect row- and column-normalised matrices. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

5. Label matching

A small example makes the mechanism visible. Rules compare overlap with match thresholds to decide equivalence, refinement or novelty.

A concrete situation

A broad label matches an internal parent.

What the mathematics does not license

A threshold selected after seeing results overfits. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Tune on separate simulated or reference data. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

6. Adding a child label

This is where modelling choices begin to affect evidence. When an incoming label refines an existing node, it becomes a child while the parent and previous labels remain.

A concrete situation

CD4 T cell is inserted below T cell.

What the mathematics does not license

False refinement fragments one population. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Check markers and cross-dataset separability. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

7. Adding a new branch

The central mathematical idea is concrete. An unmatched label can be attached where prediction patterns and distances support a new branch.

A concrete situation

A new stromal type joins below stromal parent.

What the mathematics does not license

Out-of-domain technical clusters can become fake types. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Require multi-donor evidence. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

8. Preserving prior names

Here the geometry becomes operational. Earlier annotations retain their identifiers when the tree is updated, supporting reproducible downstream work.

A concrete situation

Old T-cell results remain addressable.

What the mathematics does not license

Stable names can preserve an earlier mistake. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Version corrections rather than overwrite history. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

9. Retraining on accumulated data

This step deserves a careful pause. After tree update, node classifiers learn from all incorporated datasets and resolutions.

A concrete situation

A lineage boundary gains examples from three studies.

What the mathematics does not license

Unbalanced studies can dominate. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Weight or stratify by dataset and donor. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

10. Linear SVM nodes

The useful question is not only how to calculate, but what the calculation assumes. A linear support-vector machine separates child categories with a maximum-margin hyperplane using positive and negative samples.

A concrete situation

One node divides lymphoid and myeloid.

What the mathematics does not license

A linear boundary may miss curved support. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Inspect margins and errors per node. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

11. One-class SVM option

A small example makes the mechanism visible. A one-class model learns the support region of a child and can reject cells outside it without needing a complete negative class.

A concrete situation

A rare known type forms a compact region.

What the mathematics does not license

Kernel and nu settings control rejection. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Tune with known outliers and held-out types. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

12. PCA-domain reconstruction

This is where modelling choices begin to affect evidence. A query projected into a learned PCA space is reconstructed; large error signals that it lies outside the reference domain.

A concrete situation

An unseen lineage reconstructs poorly.

What the mathematics does not license

A rare known type may also have high error. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Calibrate errors by class and study. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

13. Neighbour-distance rejection

The central mathematical idea is concrete. A query far from its nearest supported reference neighbours can be rejected even if a classifier returns a label.

A concrete situation

A novel cell has a large local distance.

What the mathematics does not license

Distance depends on scaling and density. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Report local distributions and thresholds. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

14. Hierarchical F1

Here the geometry becomes operational. The metric compares predicted and true ancestor paths, giving different credit to a correct parent, wrong branch and exact leaf.

A concrete situation

Parent-only prediction receives partial credit.

What the mathematics does not license

Weights encode the value of resolution. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Publish the metric definition and flat scores. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

15. A student progressive project

This step deserves a careful pause. Start with broad labels, add a finer dataset from a shifted domain, update a tree and reject a withheld class.

A concrete situation

Known truth reveals order and threshold effects.

What the mathematics does not license

Real data add inconsistent names and batch. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Compare two ingestion orders. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

16. Governance and careers

The useful question is not only how to calculate, but what the calculation assumes. Version dataset order, cross-predictions, matches, tree edits, classifiers, PCA and rejection thresholds. This joins graphs and continual learning.

A concrete situation

Students can begin with confusion matrices and trees.

What the mathematics does not license

A stable hierarchy proves no mechanism or diagnosis. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Write a progressive atlas log. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

A seven-day practice lab

1. Day 1: define the decision, observation unit, X variables and Y or reference condition.
2. Day 2: reproduce the smallest worked example and check every dimension, unit and sign.
3. Day 3: build a transparent baseline and write what it assumes.
4. Day 4: fit the main method inside a training-only pipeline; record every parameter.
5. Day 5: challenge one assumption with a deliberately difficult dataset or operating regime.
6. Day 6: compare validation metrics with the practical workload created by errors.
7. Day 7: present one supported conclusion, one unresolved uncertainty and one safe next experiment.

Questions for a student, parent or mentor

• Which part of the model is a mathematical definition, and which part is a choice?
• What small calculation would catch a sign, scale or dimension error?
• What evidence comes from training, validation and the untouched test?
• What benign condition could look like a fault or prediction failure?
• How would the answer change for another response, asset, school project or time period?
• What should a human review before acting on the score?

A method-specific audit before deployment

A scHPL audit begins with one annotated dataset and a classification tree. Additional datasets are cross-predicted against the current hierarchy, their labels are matched to existing nodes from confusion patterns, and unmatched labels are inserted without erasing earlier names. The updated tree is retrained progressively on all incorporated data.

Record dataset order, preprocessing and integration, PCA domain, cross-prediction matrices, match thresholds, label attachments, internal nodes, classifier choice, rejection rules, reconstruction error and every tree version. Progressive learning is valuable because it preserves previous annotations while refining the hierarchy, but the order and quality of incoming datasets can change the learned structure.

Validate with reversed dataset orders, study-held-out data, hierarchical precision and recall, tree-edit distance, internal-node predictions, unseen-cell rejection and stability of original labels. A growing hierarchy should accumulate evidence without pretending that every naming system is identical.

---

Frequently asked questions

What is scHPL?

scHPL is a hierarchical progressive-learning framework that reconciles labels from multiple annotated datasets and continuously updates a cell-type classification tree.

What is cross-prediction?

The current classifier predicts cells in the incoming dataset while incoming labels are compared with predicted existing labels through a confusion or overlap matrix.

How are labels matched?

Thresholded overlap patterns decide whether an incoming label corresponds to an existing node, a more specific child or a genuinely new branch.

Why preserve old annotations?

Stable previous names protect downstream analyses and allow the hierarchy to grow without silently rewriting earlier results.

How can scHPL reject unseen cells?

Its rejection logic can use ambiguous classifier output, large neighbour distance and high PCA reconstruction error; one-class SVMs can add class-specific support regions.

How should scHPL be validated?

Reverse dataset order, hold out whole studies and types, report hierarchical F1, tree stability, internal-node outcomes and rejection performance.

---

Primary source and fact-check note

Lieke Michielsen, Marcel J. T. Reinders and Ahmed Mahfouz, “Hierarchical progressive learning of cell identities in single-cell data,” Nature Communications 12, 2799 (2021), DOI 10.1038/s41467-021-23196-8. Definitions and core claims in this article were checked against this primary publication. Calculations shown here were independently recomputed. Examples are educational illustrations; they are not reported industrial performance results.

Useful next reading

• CHETAH selective hierarchy
• CellO ontology constraints
• scPred SVM classification

Keep the options open. Mathematics education supports clearer questions, transferable problem-solving skills and informed choices across science, engineering, computing, AI, statistics, design and many other careers. It does not guarantee intelligence, income, admission or employment; those outcomes depend on far more than one subject or model.

Extended laboratory: topic-specific validation and responsible use

Dataset-order challenge

Build the hierarchy under several ingestion orders. Compare node placement, original-label preservation and hierarchical F1.

Cross-prediction ledger

Archive every incoming-label by predicted-node matrix, normalisation and matching threshold. Tree edits should be traceable to counts.

Integration audit

Measure whether batch correction improves cross-study agreement without collapsing known cell distinctions. Keep raw data and before–after metrics.

Tree-edit log

Record each matched, renamed, attached, split and rejected label with evidence and version. Never overwrite the previous tree.

Classifier comparison

Compare kNN, linear SVM and one-class SVM under identical hierarchy and data. Report internal-node and rejected outcomes.

Rejection decomposition

Separate ambiguity, neighbour-distance and reconstruction-error rejection. Different causes require different biological follow-up.

Withheld-population test

Remove a known leaf and measure whether it becomes a parent prediction, a rejection or a wrong sibling. Report hierarchical severity.

Final claim

Name data order, match rules, tree version, node classifiers and rejection thresholds. Progressive learning preserves history; it does not erase uncertainty.

Mechanism workshop: calculate, challenge and transfer

Workshop 1: Why annotations disagree

Start with the mechanism: Datasets may use broad labels, fine labels or different names for overlapping biology. A progressive hierarchy must reconcile without flattening. The worked case is concrete: One study says T cell; another says CD4 and CD8. Expose the boundary: Naming differences are not always biological differences. Complete this evidence step: Preserve provenance beside every node. Write inputs, the mathematical operation, expected output, uncertainty and one observation that would contradict the claim. Repeat under a nearby defensible setting and compare features, bins, trees, activations, probabilities, hierarchy edits or labels. Use untouched donors, studies or simulated truth because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so the conclusion remains auditable.

Workshop 2: Initial classification tree

Start with the mechanism: The first dataset establishes a root, internal categories and leaves with classifiers at relevant nodes. The worked case is concrete: Immune splits into lymphoid and myeloid. Expose the boundary: The starting dataset influences all later placements. Complete this evidence step: Repeat with alternative starting orders. Write inputs, the mathematical operation, expected output, uncertainty and one observation that would contradict the claim. Repeat under a nearby defensible setting and compare features, bins, trees, activations, probabilities, hierarchy edits or labels. Use untouched donors, studies or simulated truth because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so the conclusion remains auditable.

Workshop 3: Cross-prediction

Start with the mechanism: The current hierarchy predicts every cell in the incoming labelled dataset. The worked case is concrete: New CD4 cells map mostly to the existing T-cell node. Expose the boundary: Batch shift can look like label mismatch. Complete this evidence step: Integrate and audit before matching. Write inputs, the mathematical operation, expected output, uncertainty and one observation that would contradict the claim. Repeat under a nearby defensible setting and compare features, bins, trees, activations, probabilities, hierarchy edits or labels. Use untouched donors, studies or simulated truth because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so the conclusion remains auditable.

Workshop 4: Confusion matrix as evidence

Start with the mechanism: Rows of incoming labels and columns of predicted nodes quantify overlap between naming systems. The worked case is concrete: Eighty percent of label A lands in node B. Expose the boundary: Counts depend on class size and threshold. Complete this evidence step: Inspect row- and column-normalised matrices. Write inputs, the mathematical operation, expected output, uncertainty and one observation that would contradict the claim. Repeat under a nearby defensible setting and compare features, bins, trees, activations, probabilities, hierarchy edits or labels. Use untouched donors, studies or simulated truth because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so the conclusion remains auditable.

Workshop 5: Label matching

Start with the mechanism: Rules compare overlap with match thresholds to decide equivalence, refinement or novelty. The worked case is concrete: A broad label matches an internal parent. Expose the boundary: A threshold selected after seeing results overfits. Complete this evidence step: Tune on separate simulated or reference data. Write inputs, the mathematical operation, expected output, uncertainty and one observation that would contradict the claim. Repeat under a nearby defensible setting and compare features, bins, trees, activations, probabilities, hierarchy edits or labels. Use untouched donors, studies or simulated truth because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so the conclusion remains auditable.

Workshop 6: Adding a child label

Start with the mechanism: When an incoming label refines an existing node, it becomes a child while the parent and previous labels remain. The worked case is concrete: CD4 T cell is inserted below T cell. Expose the boundary: False refinement fragments one population. Complete this evidence step: Check markers and cross-dataset separability. Write inputs, the mathematical operation, expected output, uncertainty and one observation that would contradict the claim. Repeat under a nearby defensible setting and compare features, bins, trees, activations, probabilities, hierarchy edits or labels. Use untouched donors, studies or simulated truth because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so the conclusion remains auditable.

Workshop 7: Adding a new branch

Start with the mechanism: An unmatched label can be attached where prediction patterns and distances support a new branch. The worked case is concrete: A new stromal type joins below stromal parent. Expose the boundary: Out-of-domain technical clusters can become fake types. Complete this evidence step: Require multi-donor evidence. Write inputs, the mathematical operation, expected output, uncertainty and one observation that would contradict the claim. Repeat under a nearby defensible setting and compare features, bins, trees, activations, probabilities, hierarchy edits or labels. Use untouched donors, studies or simulated truth because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so the conclusion remains auditable.

Workshop 8: Preserving prior names

Start with the mechanism: Earlier annotations retain their identifiers when the tree is updated, supporting reproducible downstream work. The worked case is concrete: Old T-cell results remain addressable. Expose the boundary: Stable names can preserve an earlier mistake. Complete this evidence step: Version corrections rather than overwrite history. Write inputs, the mathematical operation, expected output, uncertainty and one observation that would contradict the claim. Repeat under a nearby defensible setting and compare features, bins, trees, activations, probabilities, hierarchy edits or labels. Use untouched donors, studies or simulated truth because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so the conclusion remains auditable.

Workshop 9: Retraining on accumulated data

Start with the mechanism: After tree update, node classifiers learn from all incorporated datasets and resolutions. The worked case is concrete: A lineage boundary gains examples from three studies. Expose the boundary: Unbalanced studies can dominate. Complete this evidence step: Weight or stratify by dataset and donor. Write inputs, the mathematical operation, expected output, uncertainty and one observation that would contradict the claim. Repeat under a nearby defensible setting and compare features, bins, trees, activations, probabilities, hierarchy edits or labels. Use untouched donors, studies or simulated truth because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so the conclusion remains auditable.

Workshop 10: Linear SVM nodes

Start with the mechanism: A linear support-vector machine separates child categories with a maximum-margin hyperplane using positive and negative samples. The worked case is concrete: One node divides lymphoid and myeloid. Expose the boundary: A linear boundary may miss curved support. Complete this evidence step: Inspect margins and errors per node. Write inputs, the mathematical operation, expected output, uncertainty and one observation that would contradict the claim. Repeat under a nearby defensible setting and compare features, bins, trees, activations, probabilities, hierarchy edits or labels. Use untouched donors, studies or simulated truth because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so the conclusion remains auditable.

Workshop 11: One-class SVM option

Start with the mechanism: A one-class model learns the support region of a child and can reject cells outside it without needing a complete negative class. The worked case is concrete: A rare known type forms a compact region. Expose the boundary: Kernel and nu settings control rejection. Complete this evidence step: Tune with known outliers and held-out types. Write inputs, the mathematical operation, expected output, uncertainty and one observation that would contradict the claim. Repeat under a nearby defensible setting and compare features, bins, trees, activations, probabilities, hierarchy edits or labels. Use untouched donors, studies or simulated truth because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so the conclusion remains auditable.

Workshop 12: PCA-domain reconstruction

Start with the mechanism: A query projected into a learned PCA space is reconstructed; large error signals that it lies outside the reference domain. The worked case is concrete: An unseen lineage reconstructs poorly. Expose the boundary: A rare known type may also have high error. Complete this evidence step: Calibrate errors by class and study. Write inputs, the mathematical operation, expected output, uncertainty and one observation that would contradict the claim. Repeat under a nearby defensible setting and compare features, bins, trees, activations, probabilities, hierarchy edits or labels. Use untouched donors, studies or simulated truth because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so the conclusion remains auditable.

Workshop 13: Neighbour-distance rejection

Start with the mechanism: A query far from its nearest supported reference neighbours can be rejected even if a classifier returns a label. The worked case is concrete: A novel cell has a large local distance. Expose the boundary: Distance depends on scaling and density. Complete this evidence step: Report local distributions and thresholds. Write inputs, the mathematical operation, expected output, uncertainty and one observation that would contradict the claim. Repeat under a nearby defensible setting and compare features, bins, trees, activations, probabilities, hierarchy edits or labels. Use untouched donors, studies or simulated truth because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so the conclusion remains auditable.

Workshop 14: Hierarchical F1

Start with the mechanism: The metric compares predicted and true ancestor paths, giving different credit to a correct parent, wrong branch and exact leaf. The worked case is concrete: Parent-only prediction receives partial credit. Expose the boundary: Weights encode the value of resolution. Complete this evidence step: Publish the metric definition and flat scores. Write inputs, the mathematical operation, expected output, uncertainty and one observation that would contradict the claim. Repeat under a nearby defensible setting and compare features, bins, trees, activations, probabilities, hierarchy edits or labels. Use untouched donors, studies or simulated truth because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so the conclusion remains auditable.

Workshop 15: A student progressive project

Start with the mechanism: Start with broad labels, add a finer dataset from a shifted domain, update a tree and reject a withheld class. The worked case is concrete: Known truth reveals order and threshold effects. Expose the boundary: Real data add inconsistent names and batch. Complete this evidence step: Compare two ingestion orders. Write inputs, the mathematical operation, expected output, uncertainty and one observation that would contradict the claim. Repeat under a nearby defensible setting and compare features, bins, trees, activations, probabilities, hierarchy edits or labels. Use untouched donors, studies or simulated truth because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so the conclusion remains auditable.

Workshop 16: Governance and careers

Start with the mechanism: Version dataset order, cross-predictions, matches, tree edits, classifiers, PCA and rejection thresholds. This joins graphs and continual learning. The worked case is concrete: Students can begin with confusion matrices and trees. Expose the boundary: A stable hierarchy proves no mechanism or diagnosis. Complete this evidence step: Write a progressive atlas log. Write inputs, the mathematical operation, expected output, uncertainty and one observation that would contradict the claim. Repeat under a nearby defensible setting and compare features, bins, trees, activations, probabilities, hierarchy edits or labels. Use untouched donors, studies or simulated truth because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so the conclusion remains auditable.