Chapter map

1. 1. Why heterogeneous collections need panoramas
2. 2. The panorama analogy
3. 3. Shared-gene preprocessing
4. 4. Low-dimensional reduction
5. 5. Approximate nearest neighbours
6. 6. Searching all dataset pairs
7. 7. Mutual links
8. 8. The dataset graph
9. 9. A graph example
10. 10. Correction vectors from matches
11. 11. Merge sequence and panoramas
12. 12. Keeping disjoint datasets disjoint
13. 13. Preserving trajectories and unique cells
14. 14. Held-out and exact-search validation
15. 15. A student panorama project
16. 16. Governance and careers

Visit the How Mathematics Works hub for the wider series and its connections to education, reasoning, science, computing, design and careers.

---

1. Why heterogeneous collections need panoramas

A small example makes the mechanism visible. Large archives contain experiments with partial biological overlap. Requiring every dataset to share every cell type is too strict.

A concrete situation

One dataset may contain blood cells, another blood plus marrow, and another marrow only.

What the mathematics does not license

Partial overlap also creates opportunities for accidental bridges. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Draw expected overlaps as a dataset graph. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

2. The panorama analogy

This is where modelling choices begin to affect evidence. Image panoramas stitch photographs where scenes overlap. Scanorama links cell datasets where shared populations provide correspondences.

A concrete situation

Blood cells can bridge two otherwise different studies.

What the mathematics does not license

A bridge is evidence of overlap, not proof of global equivalence. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Mark shared and unique regions for three datasets. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

3. Shared-gene preprocessing

The central mathematical idea is concrete. Datasets need a common feature representation and documented normalisation. Feature intersection and scaling shape distances.

A concrete situation

Genes measured in all studies can define the search space.

What the mathematics does not license

A small biased intersection can lose cell identity. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Report feature coverage by dataset pair. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

4. Low-dimensional reduction

Here the geometry becomes operational. SVD or related reduction compresses expression for efficient neighbour search while retaining important variation.

A concrete situation

Cells become vectors in dozens rather than thousands of dimensions.

What the mathematics does not license

Omitted dimensions may contain rare-state signals. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Plot reconstruction and neighbour stability across ranks. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

5. Approximate nearest neighbours

This step deserves a careful pause. Exact all-pairs search is expensive. Approximate indices use hashing or tree structures to retrieve likely neighbours quickly.

A concrete situation

Random projection trees split cells using lower-dimensional directions.

What the mathematics does not license

Approximation can miss true pairs or return unstable ones. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Compare approximate and exact neighbours on a sampled subset. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

6. Searching all dataset pairs

The useful question is not only how to calculate, but what the calculation assumes. Scanorama examines pairwise dataset combinations to find shared cell populations wherever they exist.

A concrete situation

Dataset A may match B through T cells while B matches C through progenitors.

What the mathematics does not license

Many pair tests can generate weak accidental edges. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Record match counts, distances and cell-state coverage per pair. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

7. Mutual links

A small example makes the mechanism visible. Reciprocal cross-dataset neighbours are retained as stronger evidence of correspondence.

A concrete situation

Cell a chooses b and b chooses a.

What the mathematics does not license

Reciprocity does not guarantee biological identity. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Build directed lists and intersect them. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

8. The dataset graph

This is where modelling choices begin to affect evidence. Datasets become nodes and supported overlaps become edges. Connected components define panoramas.

A concrete situation

A-B and B-C edges place all three in one panorama.

What the mathematics does not license

One weak B-C edge can join incompatible components. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Perform edge-removal sensitivity and report bridges. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

9. A graph example

The central mathematical idea is concrete. Let A-B have 120 mutual links, B-C have 80 and A-C have none. The graph is connected through B.

A concrete situation

B acts as a biological bridge between A and C.

What the mathematics does not license

Counts alone ignore link quality and coverage. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Weight edges by support and inspect the minimum cut. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

10. Correction vectors from matches

Here the geometry becomes operational. Matched cells provide local difference vectors used to align one dataset toward another in supported regions.

A concrete situation

Shared T cells estimate a local batch shift.

What the mathematics does not license

Correction outside matched populations is extrapolation. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Calculate a weighted vector from two matches. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

11. Merge sequence and panoramas

This step deserves a careful pause. Within a connected component, datasets are assembled using match strength or another guide. The sequence can affect coordinates.

A concrete situation

A well-covered reference can anchor several smaller studies.

What the mathematics does not license

Order sensitivity signals weak geometry. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Repeat plausible orders and compare neighbourhoods. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

12. Keeping disjoint datasets disjoint

The useful question is not only how to calculate, but what the calculation assumes. Datasets with no supported overlap need not be merged. This avoids inventing equivalence.

A concrete situation

A specialised tissue may form its own panorama.

What the mathematics does not license

Failure to integrate can be the correct result. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

State minimum evidence for an edge. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

13. Preserving trajectories and unique cells

A small example makes the mechanism visible. Integration should retain differentiation paths, markers and populations found in only one dataset.

A concrete situation

A time-series lineage can overlap partly across experiments.

What the mathematics does not license

Aggressive smoothing can flatten temporal structure. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Measure pseudotime and neighbourhood preservation. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

14. Held-out and exact-search validation

This is where modelling choices begin to affect evidence. Hold out donors or studies, and compare approximate matches with exact search on manageable subsets.

A concrete situation

A new study should join the expected panorama without label tuning.

What the mathematics does not license

Random cell splits are too weak. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Predefine graph and biology metrics. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

15. A student panorama project

The central mathematical idea is concrete. Create three point clouds with overlapping clusters and one unique cluster. Build pairwise mutual-neighbour edges.

A concrete situation

The graph shows how indirect overlap connects datasets.

What the mathematics does not license

Real data include sparse counts and approximate search. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Remove one bridge and observe the components. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

16. Governance and careers

Here the geometry becomes operational. Version features, reduction, index, matches, graph, merges and corrections. Panorama building combines linear algebra, algorithms and biology.

A concrete situation

Students can begin with graphs and neighbours.

What the mathematics does not license

A panorama proves no mechanism or career outcome. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Create an edge ledger with support and failure zones. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

A seven-day practice lab

1. Day 1: define the decision, observation unit, X variables and Y or reference condition.
2. Day 2: reproduce the smallest worked example and check every dimension, unit and sign.
3. Day 3: build a transparent baseline and write what it assumes.
4. Day 4: fit the main method inside a training-only pipeline; record every parameter.
5. Day 5: challenge one assumption with a deliberately difficult dataset or operating regime.
6. Day 6: compare validation metrics with the practical workload created by errors.
7. Day 7: present one supported conclusion, one unresolved uncertainty and one safe next experiment.

Questions for a student, parent or mentor

• Which part of the model is a mathematical definition, and which part is a choice?
• What small calculation would catch a sign, scale or dimension error?
• What evidence comes from training, validation and the untouched test?
• What benign condition could look like a fault or prediction failure?
• How would the answer change for another response, asset, school project or time period?
• What should a human review before acting on the score?

A method-specific audit before deployment

A Scanorama audit defines datasets, shared genes, preprocessing and the biological overlap expected across experiments. The method creates a panorama by searching for mutual cross-dataset neighbours across all dataset pairs, connecting datasets that share cell populations and using those matches to estimate batch-correction vectors.

Record the low-dimensional reduction, approximate-neighbour index, search parameters, mutual links, dataset graph, connected components, merge sequence and smoothing. Approximate search improves speed but adds another source of sensitivity. Save enough state for a reviewer to reproduce a small exact-neighbour comparison.

Validate panorama membership, pair coverage, preserved neighbourhoods, markers, trajectories and unsupported populations. Heterogeneous datasets need not all overlap, which is a strength, but a weak accidental bridge can join components that should remain separate. Report dataset-graph uncertainty rather than forcing one universal atlas.

---

Frequently asked questions

What is a Scanorama panorama?

It is a connected collection of datasets linked by mutual cross-dataset matches among shared cell populations.

Why compare all dataset pairs?

Pairwise search can discover partial overlap without requiring every dataset to share the same cell types.

How is the search scalable?

Dimensionality reduction and approximate-nearest-neighbour methods reduce the cost of searching large collections.

What are mutual links?

They are reciprocal cross-dataset neighbour relations used as evidence of shared cell states.

Can a wrong bridge merge unrelated datasets?

Yes. Weak or confounded matches can connect components incorrectly, so graph edges and pair coverage must be audited.

How should Scanorama be validated?

Check exact-neighbour subsets, dataset graph stability, anchor purity, neighbourhood and marker preservation, unsupported populations and held-out studies.

---

Primary source and fact-check note

Brian Hie, Bryan Bryson and Bonnie Berger, “Efficient integration of heterogeneous single-cell transcriptomes using Scanorama,” Nature Biotechnology 37, 685–691 (2019), DOI 10.1038/s41587-019-0113-3. Definitions and core claims in this article were checked against this primary publication. Calculations shown here were independently recomputed. Examples are educational illustrations; they are not reported industrial performance results.

Useful next reading

• Mutual Nearest Neighbours
• Harmony integration
• Seurat anchor integration

Keep the options open. Mathematics education supports clearer questions, transferable problem-solving skills and informed choices across science, engineering, computing, AI, statistics, design and many other careers. It does not guarantee intelligence, income, admission or employment; those outcomes depend on far more than one subject or model.

Extended laboratory: validation, transfer and responsible use

Dataset registry

Record studies, donors, technologies, tissues, feature coverage and expected overlap. This registry makes the dataset graph scientifically interpretable.

Reduction sensitivity

Repeat reasonable SVD dimensions and feature sets. Compare pair matches and panorama components. Graph edges that vanish easily should be labelled uncertain.

Approximation audit

On a sampled subset, compare approximate with exact nearest neighbours. Report recall of reciprocal matches and runtime. Speed is valuable only with known search error.

Edge ledger

For each dataset pair, save mutual-link counts, distances, cell-state coverage and donor support. A single edge can determine an entire panorama.

Bridge challenge

Remove weak graph edges and recompute components. If one thin bridge connects very different datasets, inspect it before integration.

Merge-order test

Repeat plausible assembly orders and references. Quantify neighbourhood and correction changes. Order sensitivity is part of the uncertainty.

Unsupported cells

Store effective match count, anchor distance and correction magnitude. Do not hide cells or datasets with no support.

Trajectory preservation

Compare local order, branches and markers before and after correction. A well-mixed endpoint can still destroy developmental geometry.

Unique-population guard

Use simulations or known controls where one dataset has a unique cluster. The algorithm should not invent a counterpart through a weak bridge.

New-study transport

Add an untouched study using locked preprocessing and graph rules. Final labels may evaluate the join but must not choose parameters.

Reconstruction note

Preserve original matrices and embeddings. Corrected coordinates are derived representations, not replacements for measurement evidence.

Reader checkpoint

A trustworthy Scanorama claim names features, reduction, approximate search, mutual links, graph components, merge order, correction and unsupported regions.

Deep dive: how to stress-test the mathematical claim

Deep dive 1

Measure approximate-neighbour recall on subsets and by rare population, not only overall.

Deep dive 2

Audit dataset-graph bridges with edge-removal and label-free stability tests.

Deep dive 3

Compare exact and approximate mutual links for the edges that determine connected components.

Deep dive 4

Vary SVD rank and search parameters. Stable panorama membership is stronger than one fast run.

Deep dive 5

Hold out donors when validating matches; cell-level splits leak local structure.

Deep dive 6

Test unique populations and non-overlapping datasets. Correctly remaining separate is a success.

Deep dive 7

Inspect correction magnitude against link distance and density. Large extrapolation needs rejection.

Deep dive 8

Preserve trajectories and composition differences; panorama stitching should not equalise abundance.

Deep dive 9

Compare with MNN and Harmony under identical preprocessing and evidence gates.

Deep dive 10

Report graph structure, correction support and biology as separate layers of the conclusion.

Worked evidence notebook

SVD dimensions

Write cells-by-genes input and cells-by-r reduced coordinates.

Neighbour recall

Compare approximate and exact neighbour sets using intersection over exact-set size.

Mutual-link graph

Intersect directed links and aggregate them into dataset edges.

Connected components

Find components of a three-node graph before and after removing a weak edge.

Correction vector

Average two match differences with documented weights and direction.

Support note

Store edge, pair and cell coverage beside every corrected conclusion.

Practice, misconception and transfer

A parent-friendly starting point

Use overlapping photographs to introduce graphs and shared regions. Then replace images with cell datasets.

A key misconception

A connected panorama does not mean every dataset shares every cell type. Indirect edges can connect partial overlaps.

When to pause

Pause when one weak bridge determines the component or approximate search cannot reproduce critical links.

A useful extension

Remove one edge and observe graph components. This teaches how local evidence changes global structure.

Career transfer

Approximate neighbours and graphs power search, recommendations and biological integration, with different definitions of a valid match.

Final learning move

Require exact-neighbour verification for one subset and a written explanation of unsupported cells.

A final worked scenario

A final panorama scenario

Twenty datasets share overlapping but non-identical cell populations. Pairwise mutual links create three stable panorama components; a proposed bridge between two components disappears under exact-neighbour verification and is rejected. Within supported components, correction improves mixing while preserving markers and trajectories. Unique populations remain unmatched. The team archives the dataset graph, edge evidence and merge order. The mathematics succeeds by stitching only where overlap is supported, showing that refusing an unsupported merge can be as informative as producing one large atlas.

Mathematical workshop: mechanisms, evidence and limits

A quantitative edge table

For every dataset pair, create a table with cells in each dataset, reciprocal-link count, median and upper-tail distance, cell-state coverage, donor coverage, exact-search recall on a subset and whether the edge survives sensitivity tests. Counts alone can favour large datasets, so include rates and population-specific summaries. An edge with many links from one abundant state may support that state without supporting the rest of the datasets. The table lets reviewers separate broad overlap from a narrow bridge and shows exactly which edges define each panorama component. Version this table with the features and parameters so later reruns can be compared row by row.

Trajectory checks before celebration

Developmental and response trajectories require more than endpoint mixing. Compare local neighbour order, pseudotime rank, branch membership and marker progression before and after correction, using original measurements for biological interpretation. A correction may align endpoints while reversing an intermediate order or collapsing a small branch. Repeat the checks by donor and under nearby graph settings. If a branch is unstable, limit the conclusion to the regions with preserved ordering. This evaluation protects the geometry that matters for the scientific question instead of rewarding mixing as an end in itself.

From local neighbours to a panorama

Scanorama treats integration as a sequence of local correspondences and graph operations. Each dataset is first represented in a lower-dimensional space. Approximate search proposes neighbours across dataset pairs, reciprocal links keep pairs that agree in both directions, and the supported dataset pairs become edges in a graph. Connected components define panoramas that can be assembled through overlap. This local-to-global structure is powerful because datasets need not all share the same populations. It is also risky because one weak edge can connect large regions, so every global merge should be traceable to local evidence.

Approximate search is a measured compromise

Exact nearest-neighbour search can be expensive across millions of cells. Approximate methods trade a small amount of recall for speed. That trade should be measured on representative subsets. If exact neighbours contain ten valid cross-dataset matches and the approximate index recovers eight, recall is 0.8. Overall recall can still hide failure for rare populations, so report it by cell state, density and dataset pair. Critical graph edges deserve exact verification. Efficiency becomes a scientific advantage only when the error introduced by acceleration is visible and acceptable.

Mutual links reduce one-sided mistakes

A directed nearest-neighbour relation can be misleading when one dataset is much denser than another. Many sparse cells may select the same dense neighbour. Requiring reciprocity asks whether the proposed partner also points back, which filters some asymmetric matches. Reciprocity is still not proof of biological identity: two unrelated boundary points can be mutual neighbours when no true counterpart exists. Link counts, distances, marker agreement, donor coverage and stability under parameter changes should accompany the mutual-neighbour rule. The filter strengthens evidence; it does not complete it.

The dataset graph exposes assumptions

Imagine datasets A, B and C. A overlaps B with 500 stable mutual links, B overlaps C with 420, and A has no direct overlap with C. The graph can still place all three in one component through B. This is legitimate if B contains the bridging populations and each edge is independently sound. Remove the B-C edge and the component splits, revealing its structural importance. A bridge audit records how component membership and biological conclusions change when weak edges are removed. Global coherence should not rest invisibly on one fragile connection.

Correction vectors need support

For a cell near two validated matches, suppose the difference vectors are (0.3,0.1) and (0.1,-0.1) with equal weights. The average correction is (0.2,0). A cell far from all matches should not receive the same confident correction by extrapolation. Store effective match count, distance and correction magnitude for every region. Large moves with weak support can be rejected or flagged. This simple arithmetic helps learners understand that correction is an estimated transport vector with a domain of support, not a command to make every dataset overlap.

Merge order and order sensitivity

Panoramas may be assembled through a sequence of pairwise or component-level merges. When several edges have similar support, a different order can change intermediate coordinates and later matches. Repeat plausible orders, compare final neighbourhoods and report unstable regions. Stable conclusions across orders deserve more confidence; sensitive results require a narrower claim. This is a general lesson in computational science: procedures that update a representation sequentially can encode path dependence, and that dependence is part of the uncertainty rather than an inconvenience to hide.

Preserving unique populations

A dataset may contain a disease-specific, developmental or technology-specific population with no counterpart elsewhere. Successful integration should keep that population distinct. Simulations can add a unique cluster and test whether the graph invents weak reciprocal links. Real analyses can inspect raw markers, distances and correction magnitude for unsupported cells. Calling separation a failure encourages overcorrection; treating it as a possible biological result encourages honest follow-up. A panorama is valuable because it shows supported overlap and supported difference, not because it forces one continuous map.

A reproducible panorama claim

A strong report names feature selection, SVD rank, approximate index, search parameters, reciprocal-link rule, dataset graph, connected components, merge order, correction calculation and rejection policy. It verifies important edges exactly, holds out donors or studies and checks markers and trajectories in original data. A careful conclusion might say that two panorama components were supported by stable overlaps while a proposed bridge failed exact verification. That result is informative even without one universal atlas: the mathematics has shown where the evidence connects and where it stops.

Method checklist and conclusion boundary

Reproducibility checklist

Freeze features, reduction, approximate index, pairwise links, dataset graph, merge order, correction and rejection. Compare exact neighbours on subsets, hold out studies and preserve unique populations.

Careful conclusion

A careful conclusion says that supported dataset overlaps formed stable panorama components and improved alignment while preserving named biology. It does not claim that disjoint regions share counterparts.

Final evidence checks

Final evidence checks 1

Archive approximate and exact neighbour audits.

Final evidence checks 2

Report every dataset-graph edge.

Final evidence checks 3

Test bridge and merge-order sensitivity.

Final evidence checks 4

Keep unsupported populations visible.

Final evidence checks 5

Preserve original coordinates and counts.

Final evidence checks 6

Require independent reproduction of one panorama edge and correction.