WHY MATHEMATICS?

Why Mathematics? | BBKNN for Batch-Balanced Neighbour Graphs: Per-Batch Neighbours, Distance Scaling, Connectivity and Validation

How BBKNN changes the neighbour graph by finding a fixed pool within every batch, then merges and weights those pools for scalable integration—and why balance must not force nonexistent counterparts.

Why mathematics is important becomes easier to see when a real decision depends on it. This guide follows the mechanism from definitions and dimensions to a worked calculation, validation, practical learning and responsible use. It keeps factual evidence separate from interpretation and treats every illustrative scenario as an example rather than a promise of results.

Choose a reading route

Begin with PCA geometry and a defensible batch variable
Find neighbours separately within every batch
Merge, scale and weight a balanced graph
Test graph edges, unique populations and downstream stability

Chapter map

1. Why neighbourhood graphs matter
2. Ordinary k nearest neighbours
3. The batch-balance rule
4. Per-batch search
5. PCA geometry
6. Approximate neighbour search
7. Merging neighbour pools
8. Distance scaling
9. UMAP-style connectivity
10. A worked balanced graph
11. Missing populations
12. Batch labels as modelling choices
13. Downstream clustering and trajectories
14. Donor-aware and edge-level validation
15. A student graph project
16. Governance and careers

Visit the How Mathematics Works hub for the wider series and its connections to education, reasoning, science, computing, design and careers.

1. Why neighbourhood graphs matter

The central mathematical idea is concrete. Many single-cell tasks use a graph whose nodes are cells and edges connect nearby cells in PCA space.

A concrete situation

Clustering and UMAP both depend on local neighbours.

What the mathematics does not license

A distorted graph distorts every downstream picture. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Write the graph task before choosing integration. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

2. Ordinary k nearest neighbours

Here the geometry becomes operational. Standard kNN gives each cell its k closest cells overall under a distance metric.

A concrete situation

Euclidean distance in PCs often defines closeness.

What the mathematics does not license

Batch-separated cells may choose neighbours only from their own batch. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Calculate neighbours for a six-point example. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

3. The batch-balance rule

This step deserves a careful pause. BBKNN finds a specified number of neighbours separately in each batch, then merges the pools.

A concrete situation

With three batches and k=3 per batch, a cell can receive nine candidates.

What the mathematics does not license

Balance is imposed even when a batch lacks a counterpart. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

State expected overlap for every batch pair. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

4. Per-batch search

The useful question is not only how to calculate, but what the calculation assumes. For each focal cell, search within batch A, then B, then C using the same representation and metric.

A concrete situation

A cell may connect to within-batch and cross-batch peers.

What the mathematics does not license

Unequal batch size changes density and distance. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Store distances and batch labels for every edge. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

Did You Know? In this chapter, the most important number is useful only together with its construction rule. Changing scaling, sample membership or validation design can change the numerical answer without changing the arithmetic.

5. PCA geometry

A small example makes the mechanism visible. Principal components reduce dimension and determine which variation the neighbour search sees.

A concrete situation

Technical shifts can dominate early PCs.

What the mathematics does not license

Rare biology may sit in later PCs. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Compare graph edges across justified PC counts. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

6. Approximate neighbour search

This is where modelling choices begin to affect evidence. The default implementation can use approximate methods such as Annoy for speed, with exact alternatives at greater cost.

A concrete situation

Trees retrieve likely neighbours without comparing every pair.

What the mathematics does not license

Approximation may miss rare or boundary neighbours. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Measure recall against exact search on subsets. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

7. Merging neighbour pools

The central mathematical idea is concrete. Per-batch lists are combined into one candidate graph before connectivity processing.

A concrete situation

Each cell receives representation from every declared batch.

What the mathematics does not license

A large number of batches expands degree and weak links. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Check degree by cell state and batch. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

8. Distance scaling

Here the geometry becomes operational. Distances from different batch-specific searches need comparable weighting when converted into graph connectivities.

A concrete situation

Local distance scales can temper density differences.

What the mathematics does not license

Scaling can make distant forced neighbours look influential. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Plot raw distance versus final edge weight. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

9. UMAP-style connectivity

This step deserves a careful pause. Neighbour distances become fuzzy graph memberships compatible with Scanpy and UMAP workflows.

A concrete situation

Strong reciprocal local edges receive larger connectivity.

What the mathematics does not license

A layout is only one visualisation of the graph. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Inspect edges directly rather than only UMAP. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

10. A worked balanced graph

The useful question is not only how to calculate, but what the calculation assumes. Let a cell choose two neighbours from each of three batches with distances 0.2, 0.4; 0.3, 0.5; and 1.8, 2.0.

A concrete situation

The third batch is visibly less supported.

What the mathematics does not license

Equal neighbour counts do not imply equal evidence. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Compare thresholding and weighted policies. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

11. Missing populations

A small example makes the mechanism visible. If a cell state exists only in one batch, per-batch balancing can connect it to the least distant wrong states elsewhere.

A concrete situation

A disease-only cluster may have no healthy counterpart.

What the mathematics does not license

Forced edges can erase novelty. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Define distance, marker and support rejection checks. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

12. Batch labels as modelling choices

This is where modelling choices begin to affect evidence. Batch can mean donor, run, protocol or another grouping. The chosen variable changes which pools are balanced.

A concrete situation

Balancing by donor differs from balancing by technology.

What the mathematics does not license

Condition should not be silently treated as nuisance. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Cross-tab batch, condition and cell type. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

13. Downstream clustering and trajectories

The central mathematical idea is concrete. BBKNN supplies a graph used by clustering, UMAP or pseudotime; each downstream method adds parameters and claims.

A concrete situation

A Leiden cluster depends on graph connectivity.

What the mathematics does not license

Graph mixing can alter branch structure. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Report stability across graph and downstream settings. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

14. Donor-aware and edge-level validation

Here the geometry becomes operational. Hold out donors or studies, inspect cross-batch edges and compare markers in original counts.

A concrete situation

A new donor should join supported states.

What the mathematics does not license

Random cell splits leak local structure. A trustworthy workflow records the choice before seeing the final test result.

Try it, check it, transfer it

Publish edge purity, coverage and rejection. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

15. A student graph project

This step deserves a careful pause. Create three shifted point clouds with one cluster missing from one batch. Build ordinary and balanced kNN graphs.

A concrete situation

Known geometry reveals useful and forced edges.

What the mathematics does not license

Real data add sparsity and many batches. Evidence becomes stronger when the limitation is tested instead of hidden.

Try it, check it, transfer it

Vary k, distance and missingness. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

16. Governance and careers

The useful question is not only how to calculate, but what the calculation assumes. Version features, PCs, batch labels, search index, distances, connectivities and downstream parameters. Graph integration joins geometry, algorithms and biology.

A concrete situation

Students can begin with distance matrices and graphs.

What the mathematics does not license

A balanced graph proves no identity or mechanism. Keeping this boundary clear makes the result easier to audit and transfer.

Try it, check it, transfer it

Create an edge ledger and unsupported-region map. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

A seven-day practice lab

Day 1: define the decision, observation unit, X variables and Y or reference condition.
Day 2: reproduce the smallest worked example and check every dimension, unit and sign.
Day 3: build a transparent baseline and write what it assumes.
Day 4: fit the main method inside a training-only pipeline; record every parameter.
Day 5: challenge one assumption with a deliberately difficult dataset or operating regime.
Day 6: compare validation metrics with the practical workload created by errors.
Day 7: present one supported conclusion, one unresolved uncertainty and one safe next experiment.

Questions for a student, parent or mentor

Which part of the model is a mathematical definition, and which part is a choice?
What small calculation would catch a sign, scale or dimension error?
What evidence comes from training, validation and the untouched test?
What benign condition could look like a fault or prediction failure?
How would the answer change for another response, asset, school project or time period?
What should a human review before acting on the score?

A method-specific audit before deployment

A BBKNN audit begins with a PCA representation, a declared batch variable and a biological reason to expect at least some shared cell states. For each cell, the algorithm finds neighbours separately within every batch, merges those per-batch pools and converts distances into a graph compatible with common Scanpy workflows.

Record features, scaling, PCs, batch labels, neighbours per batch, distance metric, approximate-search settings, graph weights and downstream clustering or trajectory parameters. Batch balance is imposed by construction, so unequal composition and missing cell types require explicit review.

Validate graph edges, exact-neighbour recall on subsets, batch mixing, cell-state preservation, rare and unique populations, donor holdouts and sensitivity to batch definitions. The balanced graph changes neighbourhood structure; it does not produce corrected gene counts or prove that cross-batch neighbours are biologically identical.

Frequently asked questions

What does BBKNN stand for?

Batch Balanced k Nearest Neighbours.

What does BBKNN change?

It replaces the ordinary neighbourhood graph with one built from neighbours found separately in each batch.

Does BBKNN create corrected expression values?

No. Its primary output is a balanced graph for embedding, clustering and trajectory analysis.

Why use approximate neighbours?

Approximate search can scale to large data, but recall should be checked on representative subsets.

What is the main biological assumption?

At least some comparable cell states should exist across batches, with batch differences smaller than differences between unrelated states.

How should BBKNN be validated?

Audit cross-batch edges, exact-neighbour recall, composition, unique populations, donor holdouts, marker preservation and downstream stability.

Primary source and fact-check note

Krzysztof Polański, Matthew D. Young, Zhichao Miao et al., “BBKNN: fast batch alignment of single cell transcriptomes,” Bioinformatics 36(3), 964–965 (2020), DOI 10.1093/bioinformatics/btz625. Definitions and core claims in this article were checked against this primary publication. Calculations shown here were independently recomputed. Examples are educational illustrations; they are not reported industrial performance results.

Useful next reading

Keep the options open. Mathematics education supports clearer questions, transferable problem-solving skills and informed choices across science, engineering, computing, AI, statistics, design and many other careers. It does not guarantee intelligence, income, admission or employment; those outcomes depend on far more than one subject or model.

Extended laboratory: validation, transfer and responsible use

Graph provenance

Save every cell ID, feature set, scaling rule, PC score, batch label, neighbour candidate, raw distance and final connectivity. Downstream clusters are only reproducible when the graph is reproducible.

Batch-definition audit

Compare defensible definitions such as donor, run and protocol. A balanced graph is conditional on the declared grouping, and condition must not be silently relabelled as nuisance.

Composition table

Report cell-state counts by batch. Empty combinations identify where the balance rule lacks a true counterpart and where forced edges are most likely.

Exact-search sample

Select dense, sparse, rare and boundary cells. Compare approximate with exact per-batch neighbours and report recall plus changes in influential edges.

Did You Know? A sophisticated adjustment can be algebraically correct and still answer the wrong scientific question when samples, labels, batches or validation boundaries are wrong.

Distance ledger

Summarise raw cross-batch distances by state and batch pair. Equal neighbour counts with sharply different distances do not provide equal biological support.

Connectivity audit

Trace how raw distances become graph memberships. Inspect high-weight cross-state edges and low-weight bridges that connect otherwise separate components.

Unique-population guard

Simulate or identify a population present in only one batch. Measure whether it remains locally distinct or is connected to an unrelated state.

Neighbour-count sensitivity

Vary neighbours per batch and PC count. Report cluster, trajectory and edge stability. A result that exists only at one k is fragile.

Did You Know? A sophisticated adjustment can be algebraically correct and still answer the wrong scientific question when samples, labels, batches or validation boundaries are wrong.

Large-batch-count test

As the number of batches grows, each cell receives larger candidate pools. Monitor degree, memory, weak edges and domination by broad states.

Marker preservation

Use original counts to check markers across graph-defined neighbourhoods. The graph supports organisation; it does not replace expression measurements.

Held-out study

Build settings without one study, then add it under locked rules. Final labels may evaluate the join but must not tune it.

Reader checkpoint

A complete BBKNN claim states PCs, batches, neighbours per batch, search, distance, connectivities, unsupported states and downstream sensitivity.

Did You Know? A sophisticated adjustment can be algebraically correct and still answer the wrong scientific question when samples, labels, batches or validation boundaries are wrong.

Deep dive: how to stress-test the mathematical claim

Deep dive 1

Compare ordinary kNN and BBKNN on the same PCs to isolate the effect of balancing.

Deep dive 2

Measure exact-neighbour recall separately for dense, sparse and rare populations.

Deep dive 3

Plot cross-batch distance distributions before connectivity transformation.

Deep dive 4

Remove weak graph edges and observe component, cluster and trajectory changes.

Deep dive 5

Vary neighbours per batch and PC count under donor holdouts.

Deep dive 6

Test a unique population that lacks a counterpart in one or more batches.

Deep dive 7

Compare batch definitions without using final cell labels to choose the winner.

Deep dive 8

Audit whether equal batch contribution overwhelms a small, high-quality reference.

Deep dive 9

Check markers and differential expression in original counts, not graph weights.

Deep dive 10

Report edge construction, downstream graph analysis and biology as separate layers.

Worked evidence notebook

Distance matrix

Calculate Euclidean distances from one focal cell to candidates in three batches.

Per-batch pools

Select the two closest candidates within each batch and merge the six neighbours.

Recall check

Compare approximate and exact neighbour sets using intersection divided by exact-set size.

Edge ledger

Store source, target, batches, raw distance and final connectivity for six edges.

Did You Know? A sophisticated adjustment can be algebraically correct and still answer the wrong scientific question when samples, labels, batches or validation boundaries are wrong.

Component test

Remove a weak bridge and recompute connected components.

Unique-state note

Identify forced edges from a population with no counterpart and document the rejection rule.

Practice, misconception and transfer

A parent-friendly starting point

Draw classmates as points and require each student to choose neighbours from every classroom. Then ask when that rule helps or forces a bad match.

A key misconception

Balanced neighbour counts do not mean every cross-batch edge is equally close or biologically valid.

When to pause

Pause when batches have disjoint populations or the declared batch variable is confounded with condition.

A useful extension

Remove one population from a batch and measure forced edges.

Did You Know? A sophisticated adjustment can be algebraically correct and still answer the wrong scientific question when samples, labels, batches or validation boundaries are wrong.

Career transfer

Balanced graphs appear in recommendation, network analysis and fair sampling, with different meanings of a valid edge.

Final learning move

Ask the learner to build a six-point graph by hand and explain one rejected edge.

A final worked scenario

A final graph scenario

Four batches share several immune populations, but one batch contains a unique activated state. BBKNN creates per-batch neighbour pools and a balanced graph for the shared states. Edge-distance audits show that the activated cells would require long forced links, so those links are rejected and the state remains separate. Exact-neighbour checks on rare cells and a held-out donor support the remaining graph. Success means balanced, traceable overlap where evidence exists and visible separation where it does not.

Mathematical workshop: mechanisms, evidence and limits

Local density changes what distance means

A distance of 0.5 can be close in a sparse batch and far in a dense one. Per-batch searches therefore interact with local density and the later connectivity transform. Examine distance quantiles within cell states and batches, not only absolute thresholds. Density-aware weighting can help, but it can also elevate weak matches in undersampled regions. Report both raw distance and final weight so reviewers can see where local scaling changed the practical influence of an edge.

Degree grows with the number of batches

If the rule selects k neighbours from each of B batches, the candidate degree grows roughly with k times B before duplicate and connectivity processing. In a collection with many batches, cells can acquire numerous weak cross-batch candidates. Monitor degree distributions, memory, graph density and the fraction of influential edges. Consider whether all batches belong in one analysis or whether a hierarchical design is clearer. Scalability is not only runtime; it also concerns how the graph’s statistical meaning changes as B increases.

Mixing metrics need biological partners

A graph can score highly on batch mixing by connecting different cell states, so mixing alone rewards overcorrection. Pair it with cell-state conservation, marker consistency, neighbourhood preservation, rare-state sensitivity and trajectory stability. Use labels only for evaluation when they were not used to tune the graph. Report a trade-off table rather than one composite winner. A defensible setting improves the intended nuisance structure while preserving independently supported biology, and it leaves unsupported regions visible.

Why the graph is the product

Some integration methods return corrected expression values. BBKNN primarily returns a neighbourhood graph. Clustering, UMAP and trajectory tools then operate on that graph, so edge construction is the central scientific object. Save and inspect it. A beautiful two-dimensional layout is only one projection of weighted connections. Reviewers should be able to identify which cells became neighbours, from which batches, at what raw distances and with what final connectivities. This makes downstream changes traceable to the balancing rule.

Did You Know? A sophisticated adjustment can be algebraically correct and still answer the wrong scientific question when samples, labels, batches or validation boundaries are wrong.

A per-batch example

Suppose a focal cell has nearest distances 0.2 and 0.4 in batch A, 0.3 and 0.5 in batch B, and 1.8 and 2.0 in batch C. With two neighbours per batch, all six enter the candidate pool even though C is much farther away. Distance-to-connectivity transformation may downweight C, but the example warns against equating equal counts with equal support. Plot these distributions and define rejection or review rules for long edges, especially where a batch lacks the relevant population.

The overlap assumption

The primary paper states that at least some cells of the same type should exist across batches and that within-type batch differences should be smaller than differences between unrelated types. This is a testable assumption. Composition tables can reveal missing combinations; simulations can increase batch shift until forced neighbours cross biological classes. When the assumption fails, the right outcome may be separate components or a different method. Graph balance cannot manufacture a counterpart that was never sampled.

Approximate search needs a recall audit

Approximate neighbours make large atlases feasible, but speed introduces search error. Select dense, sparse, rare and boundary cells and compare their approximate per-batch pools with exact search. Compute recall as the size of the intersection divided by the exact neighbour count. Overall recall can look high while rare populations fail, so stratify it. Critical edges that connect graph components deserve exact verification. A scalable result is strongest when its approximation error is measured rather than assumed negligible.

Batch definitions change the graph

Balancing by donor, sequencing run, technology or study creates different pools. None is a neutral synonym. A donor label may preserve technology effects; a technology label may ignore donor heterogeneity; a condition label may erase biology. Cross-tab all design factors and write the intended nuisance structure before building the graph. Compare defensible definitions under held-out donors and biological metrics. Do not choose the version with the prettiest mixing after seeing final labels.

Did You Know? A sophisticated adjustment can be algebraically correct and still answer the wrong scientific question when samples, labels, batches or validation boundaries are wrong.

Unique populations test scientific honesty

Add a cluster present only in one batch. Ordinary kNN may keep it separate; a strictly balanced graph will search every other batch and may connect it to the least distant wrong state. Measure those edge distances, weights and markers. A rejection rule or visible unsupported region can preserve novelty. This experiment reframes separation as potentially correct. Integration succeeds when it aligns supported overlap and protects supported differences, not when every colour is perfectly mixed.

A student graph workshop

Draw twelve points in three colours representing batches, with two shared clusters and one colour-specific cluster. Calculate Euclidean distances, select one or two neighbours per colour, merge the lists and draw weighted edges. Remove the longest cross-colour edges and recompute connected components. Then vary k. The learner sees how a local rule changes global topology, clustering and possible trajectories. The exercise transfers directly to social networks, recommendations and sensor graphs, where the meaning of a valid edge also matters.

A precise final statement

A defensible claim might say that under a named PCA space and batch definition, shared immune states formed stable cross-batch neighbourhoods across held-out donors, approximate search matched exact neighbours at stated recall and unique populations remained unsupported. It does not say that expression values were corrected, that every edge proves cell identity or that batch mixing alone validates biology. The graph, its assumptions and its failure zones are part of the result.

Method checklist and conclusion boundary

Reproducibility checklist

Freeze features, scaling, PCs, batch variable, neighbours per batch, search index, metric, distances, connectivities and downstream graph parameters. Reproduce one focal cell’s complete neighbour pool.

Careful conclusion

A careful conclusion says that supported shared states formed a stable balanced graph under named settings. It does not claim corrected expression or counterparts for every population.

Final evidence checks

Final evidence checks 1

Version PCs and batch labels.

Final evidence checks 2

Archive raw distances and connectivities.

Final evidence checks 3

Verify approximate neighbours exactly on subsets.

Final evidence checks 4

Keep unique populations visible.

Final evidence checks 5

Test graph and downstream sensitivity.

Final evidence checks 6

Require independent reproduction of one neighbour pool.