Chapter map

1. 1. Why projection is a comparison problem
2. 2. The shared-feature contract
3. 3. Feature selection by residuals
4. 4. Reference cluster centroids
5. 5. Cosine similarity
6. 6. Pearson correlation
7. 7. Spearman correlation
8. 8. Similarity consensus
9. 9. Thresholds and unassigned cells
10. 10. Cell-level neighbours
11. 11. Neighbour consensus
12. 12. Unknown states and open sets
13. 13. Technology and species shifts
14. 14. Held-out study validation
15. 15. A student similarity project
16. 16. Governance and careers

Visit the How Mathematics Works hub for the wider series and its connections to education, reasoning, science, computing, design and careers.

---

1. Why projection is a comparison problem

The central mathematical idea is concrete. A query cell must be compared with reference examples measured elsewhere. The mathematics turns expression profiles into defensible similarity evidence.

A concrete situation

A new immune study is projected onto an annotated atlas.

What the mathematics does not license

Different technologies and missing states weaken comparability. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Define the intended reference coverage. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

2. The shared-feature contract

Here the geometry becomes operational. Reference and query must use compatible gene identifiers and transformations. Only shared, valid features can enter similarity calculations.

A concrete situation

Gene symbols are intersected and ordered identically.

What the mathematics does not license

Silent duplicates or mismatches corrupt vectors. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Hash ordered genes and transformations. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

3. Feature selection by residuals

This step deserves a careful pause. scmap can select genes whose dropout behaviour departs from the average mean–dropout relationship, seeking informative features.

A concrete situation

Genes with unexpectedly frequent or rare detection may separate states.

What the mathematics does not license

Technical artifacts can look informative. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Plot selection and test alternatives. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

4. Reference cluster centroids

The useful question is not only how to calculate, but what the calculation assumes. For each labelled cluster, a centroid summarises expression across reference cells.

A concrete situation

A T-cell centroid is one vector over selected genes.

What the mathematics does not license

One centroid can hide multimodal substructure. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Compare mean and median summaries. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

5. Cosine similarity

A small example makes the mechanism visible. Cosine similarity measures the angle between two vectors after accounting for magnitude.

A concrete situation

Parallel expression patterns score near one.

What the mathematics does not license

Non-negative sparse vectors can still match for broad programmes. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Calculate one toy cosine score. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

6. Pearson correlation

This is where modelling choices begin to affect evidence. Pearson correlation compares centred linear patterns, ignoring global offsets.

A concrete situation

A query and centroid can rise and fall together across genes.

What the mathematics does not license

Outliers and nonlinear relations affect the result. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Compute covariance divided by standard deviations. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

7. Spearman correlation

The central mathematical idea is concrete. Spearman correlation applies Pearson correlation to ranks, emphasising monotonic ordering.

A concrete situation

Two profiles with similar gene ranks can match despite scale changes.

What the mathematics does not license

Ties and sparse detection reduce resolution. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Rank five genes and calculate correlation. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

8. Similarity consensus

Here the geometry becomes operational. Cluster projection combines evidence across similarity measures rather than trusting one number.

A concrete situation

Two or three measures may support the same cluster.

What the mathematics does not license

Agreement can still be jointly wrong when the reference is incomplete. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Publish every component score. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

9. Thresholds and unassigned cells

This step deserves a careful pause. A minimum similarity threshold prevents every query from receiving a forced label.

A concrete situation

A novel state stays unassigned below support.

What the mathematics does not license

A high threshold rejects valid shifted cells. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Plot coverage against error. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

10. Cell-level neighbours

The useful question is not only how to calculate, but what the calculation assumes. scmap-cell compares a query with individual reference cells through an approximate neighbour search.

A concrete situation

A query finds its closest reference examples.

What the mathematics does not license

Large references and density imbalance affect neighbours. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Audit exact-search recall on a sample. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

11. Neighbour consensus

A small example makes the mechanism visible. A cell assignment requires close neighbours to support the same reference label under the declared rule.

A concrete situation

Three top neighbours agree on one cluster.

What the mathematics does not license

A broad abundant class can dominate the index. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Report neighbour labels and margins. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

12. Unknown states and open sets

This is where modelling choices begin to affect evidence. A query can contain cell types absent from the reference. This is an open-set recognition problem.

A concrete situation

A disease-specific state has no valid owner.

What the mathematics does not license

The highest similarity always exists even when all matches are poor. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Include withheld-class challenges. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

13. Technology and species shifts

The central mathematical idea is concrete. Cross-dataset projection depends on shared genes and conserved expression structure.

A concrete situation

A plate-based reference can annotate droplet data.

What the mathematics does not license

Protocol, tissue and orthology shifts can lower support. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Stratify performance by domain. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

14. Held-out study validation

Here the geometry becomes operational. Entire studies should be excluded while thresholds and features are chosen, then used once for final evaluation.

A concrete situation

A new laboratory tests transfer.

What the mathematics does not license

Random cells share donor and batch signatures. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Lock the pipeline before the test. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

15. A student similarity project

This step deserves a careful pause. Create three reference centroids and query points, then compare cosine, Pearson and Spearman assignments.

A concrete situation

Known geometry reveals agreement and conflict.

What the mathematics does not license

Real data add sparsity and thousands of genes. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Add a novel query and tune abstention. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

16. Governance and careers

The useful question is not only how to calculate, but what the calculation assumes. Version references, features, scores, neighbours, thresholds and unassigned outcomes. Projection joins linear algebra, search and biological review.

A concrete situation

Students can begin with small vectors.

What the mathematics does not license

A projected label guarantees no diagnosis or mechanism. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Write a reference-coverage card. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

A seven-day practice lab

1. Day 1: define the decision, observation unit, X variables and Y or reference condition.
2. Day 2: reproduce the smallest worked example and check every dimension, unit and sign.
3. Day 3: build a transparent baseline and write what it assumes.
4. Day 4: fit the main method inside a training-only pipeline; record every parameter.
5. Day 5: challenge one assumption with a deliberately difficult dataset or operating regime.
6. Day 6: compare validation metrics with the practical workload created by errors.
7. Day 7: present one supported conclusion, one unresolved uncertainty and one safe next experiment.

Questions for a student, parent or mentor

• Which part of the model is a mathematical definition, and which part is a choice?
• What small calculation would catch a sign, scale or dimension error?
• What evidence comes from training, validation and the untouched test?
• What benign condition could look like a fault or prediction failure?
• How would the answer change for another response, asset, school project or time period?
• What should a human review before acting on the score?

A method-specific audit before deployment

A scmap audit begins with a labelled reference, a query dataset, a shared feature set and a declared projection task. The cluster route compares each query cell with reference cluster centroids using several similarity measures; the cell route searches for nearby reference cells and applies a consensus rule.

Record feature selection, reference labels, centroids, similarity metrics, thresholds, nearest-neighbour settings, consensus rules, shared genes and every unassigned result. A numeric best match is not automatically adequate support, especially when the query contains biology absent from the reference.

Validate on untouched studies, withheld cell types, technology shifts and deliberately novel populations. Report coverage, class-specific error, unassigned rates, threshold sensitivity and marker contradictions. Reference projection can organise evidence quickly while leaving unsupported queries visible.

---

Frequently asked questions

What does scmap do?

It projects query single-cell profiles onto reference clusters or reference cells from another dataset.

What is scmap-cluster?

It represents each labelled reference cluster with a centroid and compares each query cell with those centroids.

What is scmap-cell?

It searches for neighbouring reference cells and uses neighbour agreement and a similarity threshold for assignment.

Why use several similarity measures?

Cosine, Pearson and Spearman similarities capture related but different aspects of expression agreement; consensus reduces reliance on one score.

Can scmap return no label?

Yes. Threshold and consensus rules can leave a query unassigned when reference support is insufficient.

How should scmap be validated?

Use study-held-out data, withheld cell types, threshold curves, class-specific errors, unassigned rates, marker checks and alternative references.

---

Primary source and fact-check note

Vladimir Yu Kiselev, Andrew Yiu and Martin Hemberg, “scmap: projection of single-cell RNA-seq data across data sets,” Nature Methods 15, 359–362 (2018), DOI 10.1038/nmeth.4644. Definitions and core claims in this article were checked against this primary publication. Calculations shown here were independently recomputed. Examples are educational illustrations; they are not reported industrial performance results.

Useful next reading

• scArches reference mapping
• scANVI annotation uncertainty
• Conos label propagation

Keep the options open. Mathematics education supports clearer questions, transferable problem-solving skills and informed choices across science, engineering, computing, AI, statistics, design and many other careers. It does not guarantee intelligence, income, admission or employment; those outcomes depend on far more than one subject or model.

Extended laboratory: topic-specific validation and responsible use

Feature-selection audit

Compare dropout-residual features with highly variable genes, known markers and a locked common-gene set. Evaluate assignments only on untouched studies. If one feature rule improves common classes while erasing rare states, report that trade-off rather than selecting a universal winner.

Three-score notebook

For every reviewed cell, store cosine, Pearson and Spearman similarities for the best and runner-up clusters. Agreement should be visible, and disagreement should trigger inspection of sparse ties, outliers or scale shifts. A consensus label is meaningful only when the component evidence is preserved.

Threshold curve

Sweep the similarity threshold inside validation data and plot coverage, total error, class-specific error and unassigned rate. Lock a policy before the final study. The threshold is a decision rule shaped by the cost of wrong labels versus review, not a biological constant.

Withheld-class challenge

Remove one known cell type from the reference, project its cells and measure how often they become unassigned versus confidently mislabelled. Repeat for several classes. This open-set test exposes whether the system recognises missing coverage or merely chooses the least-bad reference label.

Density challenge

Downsample common reference classes and oversample rare ones without changing biology. Rebuild the cell index and track neighbour labels. Cell-level methods can inherit reference density; cluster centroids can hide substructure. The two modes provide complementary sensitivity evidence.

Exact-search audit

For a stratified set of common, rare and boundary queries, compare approximate scmap-cell neighbours with exact search. Report recall and every assignment changed by approximation. Speed is valuable only when influential search errors are measured.

Cross-technology boundary

Evaluate each technology pair separately and record shared genes, detection rates and score shifts. A global accuracy can hide one poor domain. If one protocol produces systematically lower support, recalibrate or leave more cells unassigned rather than lowering standards for every dataset.

Final claim

A defensible statement names the reference, feature rule, similarity measures, threshold, coverage, held-out error and classes left unassigned. It does not say that a projected label proves mechanism, diagnosis or completeness. The unknown route is part of the scientific result.

Mechanism workshop: calculate, challenge and transfer

Workshop 1: Why projection is a comparison problem

Start with the mechanism: A query cell must be compared with reference examples measured elsewhere. The mathematics turns expression profiles into defensible similarity evidence. The worked case is concrete: A new immune study is projected onto an annotated atlas. Expose the boundary immediately: Different technologies and missing states weaken comparability. Complete this evidence step: Define the intended reference coverage. Write the input vectors or matrices, the operation, the expected output, uncertainty and one observation that would contradict the claim. Repeat the calculation under a nearby defensible setting and compare assignments, neighbours or corrections. Use untouched donors, studies or simulated truth where possible because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so every conclusion remains auditable.

Workshop 2: The shared-feature contract

Start with the mechanism: Reference and query must use compatible gene identifiers and transformations. Only shared, valid features can enter similarity calculations. The worked case is concrete: Gene symbols are intersected and ordered identically. Expose the boundary immediately: Silent duplicates or mismatches corrupt vectors. Complete this evidence step: Hash ordered genes and transformations. Write the input vectors or matrices, the operation, the expected output, uncertainty and one observation that would contradict the claim. Repeat the calculation under a nearby defensible setting and compare assignments, neighbours or corrections. Use untouched donors, studies or simulated truth where possible because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so every conclusion remains auditable.

Workshop 3: Feature selection by residuals

Start with the mechanism: scmap can select genes whose dropout behaviour departs from the average mean–dropout relationship, seeking informative features. The worked case is concrete: Genes with unexpectedly frequent or rare detection may separate states. Expose the boundary immediately: Technical artifacts can look informative. Complete this evidence step: Plot selection and test alternatives. Write the input vectors or matrices, the operation, the expected output, uncertainty and one observation that would contradict the claim. Repeat the calculation under a nearby defensible setting and compare assignments, neighbours or corrections. Use untouched donors, studies or simulated truth where possible because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so every conclusion remains auditable.

Workshop 4: Reference cluster centroids

Start with the mechanism: For each labelled cluster, a centroid summarises expression across reference cells. The worked case is concrete: A T-cell centroid is one vector over selected genes. Expose the boundary immediately: One centroid can hide multimodal substructure. Complete this evidence step: Compare mean and median summaries. Write the input vectors or matrices, the operation, the expected output, uncertainty and one observation that would contradict the claim. Repeat the calculation under a nearby defensible setting and compare assignments, neighbours or corrections. Use untouched donors, studies or simulated truth where possible because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so every conclusion remains auditable.

Workshop 5: Cosine similarity

Start with the mechanism: Cosine similarity measures the angle between two vectors after accounting for magnitude. The worked case is concrete: Parallel expression patterns score near one. Expose the boundary immediately: Non-negative sparse vectors can still match for broad programmes. Complete this evidence step: Calculate one toy cosine score. Write the input vectors or matrices, the operation, the expected output, uncertainty and one observation that would contradict the claim. Repeat the calculation under a nearby defensible setting and compare assignments, neighbours or corrections. Use untouched donors, studies or simulated truth where possible because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so every conclusion remains auditable.

Workshop 6: Pearson correlation

Start with the mechanism: Pearson correlation compares centred linear patterns, ignoring global offsets. The worked case is concrete: A query and centroid can rise and fall together across genes. Expose the boundary immediately: Outliers and nonlinear relations affect the result. Complete this evidence step: Compute covariance divided by standard deviations. Write the input vectors or matrices, the operation, the expected output, uncertainty and one observation that would contradict the claim. Repeat the calculation under a nearby defensible setting and compare assignments, neighbours or corrections. Use untouched donors, studies or simulated truth where possible because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so every conclusion remains auditable.

Workshop 7: Spearman correlation

Start with the mechanism: Spearman correlation applies Pearson correlation to ranks, emphasising monotonic ordering. The worked case is concrete: Two profiles with similar gene ranks can match despite scale changes. Expose the boundary immediately: Ties and sparse detection reduce resolution. Complete this evidence step: Rank five genes and calculate correlation. Write the input vectors or matrices, the operation, the expected output, uncertainty and one observation that would contradict the claim. Repeat the calculation under a nearby defensible setting and compare assignments, neighbours or corrections. Use untouched donors, studies or simulated truth where possible because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so every conclusion remains auditable.

Workshop 8: Similarity consensus

Start with the mechanism: Cluster projection combines evidence across similarity measures rather than trusting one number. The worked case is concrete: Two or three measures may support the same cluster. Expose the boundary immediately: Agreement can still be jointly wrong when the reference is incomplete. Complete this evidence step: Publish every component score. Write the input vectors or matrices, the operation, the expected output, uncertainty and one observation that would contradict the claim. Repeat the calculation under a nearby defensible setting and compare assignments, neighbours or corrections. Use untouched donors, studies or simulated truth where possible because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so every conclusion remains auditable.

Workshop 9: Thresholds and unassigned cells

Start with the mechanism: A minimum similarity threshold prevents every query from receiving a forced label. The worked case is concrete: A novel state stays unassigned below support. Expose the boundary immediately: A high threshold rejects valid shifted cells. Complete this evidence step: Plot coverage against error. Write the input vectors or matrices, the operation, the expected output, uncertainty and one observation that would contradict the claim. Repeat the calculation under a nearby defensible setting and compare assignments, neighbours or corrections. Use untouched donors, studies or simulated truth where possible because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so every conclusion remains auditable.

Workshop 10: Cell-level neighbours

Start with the mechanism: scmap-cell compares a query with individual reference cells through an approximate neighbour search. The worked case is concrete: A query finds its closest reference examples. Expose the boundary immediately: Large references and density imbalance affect neighbours. Complete this evidence step: Audit exact-search recall on a sample. Write the input vectors or matrices, the operation, the expected output, uncertainty and one observation that would contradict the claim. Repeat the calculation under a nearby defensible setting and compare assignments, neighbours or corrections. Use untouched donors, studies or simulated truth where possible because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so every conclusion remains auditable.

Workshop 11: Neighbour consensus

Start with the mechanism: A cell assignment requires close neighbours to support the same reference label under the declared rule. The worked case is concrete: Three top neighbours agree on one cluster. Expose the boundary immediately: A broad abundant class can dominate the index. Complete this evidence step: Report neighbour labels and margins. Write the input vectors or matrices, the operation, the expected output, uncertainty and one observation that would contradict the claim. Repeat the calculation under a nearby defensible setting and compare assignments, neighbours or corrections. Use untouched donors, studies or simulated truth where possible because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so every conclusion remains auditable.

Workshop 12: Unknown states and open sets

Start with the mechanism: A query can contain cell types absent from the reference. This is an open-set recognition problem. The worked case is concrete: A disease-specific state has no valid owner. Expose the boundary immediately: The highest similarity always exists even when all matches are poor. Complete this evidence step: Include withheld-class challenges. Write the input vectors or matrices, the operation, the expected output, uncertainty and one observation that would contradict the claim. Repeat the calculation under a nearby defensible setting and compare assignments, neighbours or corrections. Use untouched donors, studies or simulated truth where possible because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so every conclusion remains auditable.

Workshop 13: Technology and species shifts

Start with the mechanism: Cross-dataset projection depends on shared genes and conserved expression structure. The worked case is concrete: A plate-based reference can annotate droplet data. Expose the boundary immediately: Protocol, tissue and orthology shifts can lower support. Complete this evidence step: Stratify performance by domain. Write the input vectors or matrices, the operation, the expected output, uncertainty and one observation that would contradict the claim. Repeat the calculation under a nearby defensible setting and compare assignments, neighbours or corrections. Use untouched donors, studies or simulated truth where possible because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so every conclusion remains auditable.

Workshop 14: Held-out study validation

Start with the mechanism: Entire studies should be excluded while thresholds and features are chosen, then used once for final evaluation. The worked case is concrete: A new laboratory tests transfer. Expose the boundary immediately: Random cells share donor and batch signatures. Complete this evidence step: Lock the pipeline before the test. Write the input vectors or matrices, the operation, the expected output, uncertainty and one observation that would contradict the claim. Repeat the calculation under a nearby defensible setting and compare assignments, neighbours or corrections. Use untouched donors, studies or simulated truth where possible because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so every conclusion remains auditable.

Workshop 15: A student similarity project

Start with the mechanism: Create three reference centroids and query points, then compare cosine, Pearson and Spearman assignments. The worked case is concrete: Known geometry reveals agreement and conflict. Expose the boundary immediately: Real data add sparsity and thousands of genes. Complete this evidence step: Add a novel query and tune abstention. Write the input vectors or matrices, the operation, the expected output, uncertainty and one observation that would contradict the claim. Repeat the calculation under a nearby defensible setting and compare assignments, neighbours or corrections. Use untouched donors, studies or simulated truth where possible because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so every conclusion remains auditable.

Workshop 16: Governance and careers

Start with the mechanism: Version references, features, scores, neighbours, thresholds and unassigned outcomes. Projection joins linear algebra, search and biological review. The worked case is concrete: Students can begin with small vectors. Expose the boundary immediately: A projected label guarantees no diagnosis or mechanism. Complete this evidence step: Write a reference-coverage card. Write the input vectors or matrices, the operation, the expected output, uncertainty and one observation that would contradict the claim. Repeat the calculation under a nearby defensible setting and compare assignments, neighbours or corrections. Use untouched donors, studies or simulated truth where possible because random cells can share hidden structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so every conclusion remains auditable.