WHY MATHEMATICS?

Why Mathematics? | scBERT for Cell Annotation: Expression Bins, Gene2vec Embeddings, Performer Attention and Novel-Type Rejection

How scBERT represents every aligned gene with identity and expression vectors, learns masked-expression context using scalable Performer layers, and fine-tunes a classifier that can reject low-confidence cells instead of forcing every cell into a known type.

Why mathematics is important becomes easier to see when a real decision depends on it. This guide follows the mechanism from definitions and dimensions to a worked calculation, validation, practical learning and responsible use. It keeps factual evidence separate from interpretation and treats every illustrative scenario as an example rather than a promise of results.

Choose a reading route

Build gene and expression embeddings with explicit bins
Use masking and Performer attention at transcriptome scale
Fine-tune probabilities for known cell types
Calibrate novel-type rejection and compare strong simple baselines

Chapter map

1. Gene-symbol alignment
2. Normalize and log transform
3. Expression discretization
4. Gene2vec identity vector
5. Combined element representation
6. Masking probability
7. Masked-expression target
8. Performer approximation
9. Heads and encoder depth
10. Attention interpretation
11. Supervised fine-tuning
12. Softmax confidence
13. Novel-type rejection
14. Worked five-gene cell
15. Ablations and simple baselines
16. Governance and careers

Visit the How Mathematics Works hub for the wider series and its connections to education, reasoning, science, computing, design and careers.

1. Gene-symbol alignment

Here the geometry becomes operational. Every query matrix is revised to the model’s expected symbols and ordered columns; unmatched and duplicate genes are handled explicitly.

A concrete situation

A renamed gene maps to one canonical column.

What the mathematics does not license

A one-column shift corrupts every token. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Checksum gene order before inference. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

2. Normalize and log transform

This step deserves a careful pause. Counts are filtered, normalized and log-transformed under the training-compatible pipeline before binning.

A concrete situation

A cell is library-size normalized then log1p transformed.

What the mathematics does not license

A different pipeline changes bin assignments. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Record every preprocessing operation. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

3. Expression discretization

The useful question is not only how to calculate, but what the calculation assumes. Continuous values are mapped to ordered bin IDs plus special tokens.

A concrete situation

Genes fall into zero and six positive ranks for seven bins.

What the mathematics does not license

Hard boundaries make nearby values different. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Test sensitivity near edges. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

4. Gene2vec identity vector

A small example makes the mechanism visible. Each gene receives a pretrained 200-dimensional co-expression embedding shared across cells.

A concrete situation

The same gene starts from the same identity vector.

What the mathematics does not license

Bulk co-expression averages across cell mixtures. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Compare with random or learned identities. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

Did You Know? In this chapter, the most important number is useful only together with its construction rule. Changing scaling, sample membership or validation design can change the numerical answer without changing the arithmetic.

5. Combined element representation

This is where modelling choices begin to affect evidence. Expression and gene-identity vectors are added or otherwise combined so each position carries what gene it is and how strongly it is expressed.

A concrete situation

Gene A in bin 5 differs from gene A in bin 1.

What the mathematics does not license

Additive components can dominate at different scales. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Inspect vector norms and ablations. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

6. Masking probability

The central mathematical idea is concrete. The released pretraining code uses a mask probability and replaces most selected values with a mask token for self-supervision.

A concrete situation

Fifteen percent of eligible positions are selected in the code default.

What the mathematics does not license

Implementation defaults and paper settings must not be conflated. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Archive command-line arguments. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

7. Masked-expression target

Here the geometry becomes operational. The model predicts the original expression category for a hidden gene from other genes in the cell.

A concrete situation

Masked bin 4 receives a probability vector over bins.

What the mathematics does not license

Class imbalance can favour common low bins. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Report loss by target bin. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

8. Performer approximation

This step deserves a careful pause. Random-feature maps rewrite attention so aggregated keys and values can be computed without an explicit n-by-n matrix.

A concrete situation

Complexity grows roughly linearly with gene count.

What the mathematics does not license

Approximation quality depends on features and implementation. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Benchmark against exact attention on short sequences. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

9. Heads and encoder depth

The useful question is not only how to calculate, but what the calculation assumes. The published guidance uses multiple heads and several Performer layers to build contextual gene vectors.

A concrete situation

Ten heads and six layers form the reported default guidance.

What the mathematics does not license

More capacity can overfit small fine-tuning sets. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Sweep depth, heads and seed. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

10. Attention interpretation

A small example makes the mechanism visible. Averaged model attention can highlight genes that strongly influence contextual representations.

A concrete situation

Known marker genes receive high attention in a cell type.

What the mathematics does not license

Average attention is not a causal mechanism. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Use marker, ablation and perturbation checks. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

11. Supervised fine-tuning

This is where modelling choices begin to affect evidence. A classifier head maps the cell representation to logits for reference labels and updates model parameters with cross-entropy.

A concrete situation

A PBMC cell receives scores across nine types.

What the mathematics does not license

Random cell splits can leak dataset signatures. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Split by donor or study. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

12. Softmax confidence

The central mathematical idea is concrete. Exponentiated logits are normalized to one, producing relative scores among known classes.

A concrete situation

Top score 0.62 exceeds runner-up 0.25.

What the mathematics does not license

Closed-set softmax can be high out of domain. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Calibrate on external data. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

13. Novel-type rejection

Here the geometry becomes operational. If the maximum class score is below a threshold, the pipeline can return unassigned rather than the top known label.

A concrete situation

Maximum 0.41 is rejected at threshold 0.5.

What the mathematics does not license

One threshold has different error costs by class. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Plot coverage versus error. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

14. Worked five-gene cell

This step deserves a careful pause. Bin five normalized values, add tiny two-dimensional identity vectors, mask one value and calculate a simplified attention-weighted prediction.

A concrete situation

Context favours bin 3 for the hidden gene.

What the mathematics does not license

Toy linear attention is not the full Performer. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Change one bin and recompute. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

15. Ablations and simple baselines

The useful question is not only how to calculate, but what the calculation assumes. Compare full scBERT with no pretraining, no gene2vec, fewer layers and logistic regression under identical folds.

A concrete situation

A linear baseline reveals whether the task is already separable.

What the mathematics does not license

A single benchmark cannot establish universal value. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Publish all matched results. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

16. Governance and careers

A small example makes the mechanism visible. Version symbols, bins, gene2vec, masks, Performer, checkpoint, labels and rejection. This connects approximation, classification and bioinformatics.

A concrete situation

Students can begin with bins and softmax.

What the mathematics does not license

Novel-type rejection flags uncertainty; it does not name biology. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Write an scBERT audit sheet. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

A seven-day practice lab

Day 1: define the decision, observation unit, X variables and Y or reference condition.
Day 2: reproduce the smallest worked example and check every dimension, unit and sign.
Day 3: build a transparent baseline and write what it assumes.
Day 4: fit the main method inside a training-only pipeline; record every parameter.
Day 5: challenge one assumption with a deliberately difficult dataset or operating regime.
Day 6: compare validation metrics with the practical workload created by errors.
Day 7: present one supported conclusion, one unresolved uncertainty and one safe next experiment.

Questions for a student, parent or mentor

Which part of the model is a mathematical definition, and which part is a choice?
What small calculation would catch a sign, scale or dimension error?
What evidence comes from training, validation and the untouched test?
What benign condition could look like a fault or prediction failure?
How would the answer change for another response, asset, school project or time period?
What should a human review before acting on the score?

A method-specific audit before deployment

An scBERT audit aligns every dataset to the model’s gene order, combines fixed gene2vec identity embeddings with binned expression embeddings and passes the long gene sequence through Performer attention. Self-supervised masking teaches expression recovery; supervised fine-tuning adds a cell-type classifier and can apply a confidence threshold to leave a cell unassigned as a possible novel type.

Record gene-symbol revision, gene order, normalization, bin edges, gene2vec version, mask rate, replacement rule, Performer depth and heads, checkpoint, fine-tuning labels, logits, probabilities and rejection threshold. Binning trades numerical resolution for denoising, while linear-attention approximations trade exact all-pairs attention for scalability.

Validate with technology and donor holdouts, no-pretraining and no-gene2vec ablations, logistic-regression baselines, calibration, omitted-class challenges and repeated seeds. Later re-evaluations reinforce the need to measure where pretraining helps rather than assuming a large model is automatically superior.

Frequently asked questions

What is scBERT?

scBERT is a pretrained Transformer-style model for single-cell RNA-seq annotation. It combines gene identity and expression representations, learns through masked-expression prediction and fine-tunes on labelled reference cells.

What is an expression bin?

Normalized expression is ranked or discretized into a small ordered set of categories. The published work discusses seven bins as a balance between denoising and retaining expression differences.

What is gene2vec?

Gene2vec supplies a fixed 200-dimensional identity embedding learned from bulk co-expression. scBERT combines it with expression embeddings and then learns cell-specific contextual representations.

Why use Performer instead of ordinary attention?

A full transcriptome sequence contains more than sixteen thousand genes. Performer approximates softmax attention with linear-time feature maps, avoiding the quadratic memory of all-pairs attention.

How does scBERT find a novel cell type?

During prediction, a cell can remain unassigned when the maximum known-class confidence is below a chosen threshold. This is rejection of an uncertain known label, not automatic biological discovery.

Did later studies remove the need for validation?

No. Reusability and deeper-evaluation papers reproduced some results while emphasizing ablations and strong baselines. The practical lesson is to test pretraining, gene2vec and model size on each task.

Primary source and fact-check note

Fan Yang, Wenchuan Wang, Fang Wang et al., “scBERT as a large-scale pretrained deep language model for cell type annotation of single-cell RNA-seq data,” Nature Machine Intelligence 4, 852–866 (2022), DOI 10.1038/s42256-022-00534-z. Definitions and core claims in this article were checked against this primary publication. Calculations shown here were independently recomputed. Examples are educational illustrations; they are not reported industrial performance results.

Useful next reading

Keep the options open. Mathematics education supports clearer questions, transferable problem-solving skills and informed choices across science, engineering, computing, AI, statistics, design and many other careers. It does not guarantee intelligence, income, admission or employment; those outcomes depend on far more than one subject or model.

Extended laboratory: topic-specific validation and responsible use

Gene-order audit

Store the canonical symbol revision, ordered 16k-plus gene list, unmatched genes and duplicates for every dataset.

Bin-edge audit

Report normalization, bin boundaries and class frequencies; perturb values near thresholds and measure label stability.

Gene2vec ablation

Compare fixed gene2vec, random identity vectors and task-learned embeddings with all other settings matched.

Performer approximation test

On manageable subsets, compare approximate and exact attention outputs, runtime and downstream predictions.

Did You Know? A sophisticated adjustment can be algebraically correct and still answer the wrong scientific question when samples, labels, batches or validation boundaries are wrong.

Pretraining ablation

Compare the released checkpoint with identical architecture trained from scratch and with logistic regression.

Rejection calibration

Plot accuracy, error and unassigned rate across thresholds for each cell type and external dataset.

Biological holdout

Split by donor, technology and study; random cells from one study do not test transfer.

Final claim

Name bins, gene2vec, mask, Performer, checkpoint and threshold. Low confidence flags uncertainty rather than discovering a type.

Did You Know? A sophisticated adjustment can be algebraically correct and still answer the wrong scientific question when samples, labels, batches or validation boundaries are wrong.

Mechanism workshop: calculate, challenge and transfer

Workshop 1: Gene-symbol alignment

Start with the mechanism: Every query matrix is revised to the model’s expected symbols and ordered columns; unmatched and duplicate genes are handled explicitly. The worked case is concrete: A renamed gene maps to one canonical column. Expose the boundary: A one-column shift corrupts every token. Complete this evidence step: Checksum gene order before inference. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 2: Normalize and log transform

Start with the mechanism: Counts are filtered, normalized and log-transformed under the training-compatible pipeline before binning. The worked case is concrete: A cell is library-size normalized then log1p transformed. Expose the boundary: A different pipeline changes bin assignments. Complete this evidence step: Record every preprocessing operation. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 3: Expression discretization

Start with the mechanism: Continuous values are mapped to ordered bin IDs plus special tokens. The worked case is concrete: Genes fall into zero and six positive ranks for seven bins. Expose the boundary: Hard boundaries make nearby values different. Complete this evidence step: Test sensitivity near edges. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 4: Gene2vec identity vector

Start with the mechanism: Each gene receives a pretrained 200-dimensional co-expression embedding shared across cells. The worked case is concrete: The same gene starts from the same identity vector. Expose the boundary: Bulk co-expression averages across cell mixtures. Complete this evidence step: Compare with random or learned identities. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Did You Know? A sophisticated adjustment can be algebraically correct and still answer the wrong scientific question when samples, labels, batches or validation boundaries are wrong.

Workshop 5: Combined element representation

Start with the mechanism: Expression and gene-identity vectors are added or otherwise combined so each position carries what gene it is and how strongly it is expressed. The worked case is concrete: Gene A in bin 5 differs from gene A in bin 1. Expose the boundary: Additive components can dominate at different scales. Complete this evidence step: Inspect vector norms and ablations. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 6: Masking probability

Start with the mechanism: The released pretraining code uses a mask probability and replaces most selected values with a mask token for self-supervision. The worked case is concrete: Fifteen percent of eligible positions are selected in the code default. Expose the boundary: Implementation defaults and paper settings must not be conflated. Complete this evidence step: Archive command-line arguments. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 7: Masked-expression target

Start with the mechanism: The model predicts the original expression category for a hidden gene from other genes in the cell. The worked case is concrete: Masked bin 4 receives a probability vector over bins. Expose the boundary: Class imbalance can favour common low bins. Complete this evidence step: Report loss by target bin. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 8: Performer approximation

Start with the mechanism: Random-feature maps rewrite attention so aggregated keys and values can be computed without an explicit n-by-n matrix. The worked case is concrete: Complexity grows roughly linearly with gene count. Expose the boundary: Approximation quality depends on features and implementation. Complete this evidence step: Benchmark against exact attention on short sequences. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Did You Know? A sophisticated adjustment can be algebraically correct and still answer the wrong scientific question when samples, labels, batches or validation boundaries are wrong.

Workshop 9: Heads and encoder depth

Start with the mechanism: The published guidance uses multiple heads and several Performer layers to build contextual gene vectors. The worked case is concrete: Ten heads and six layers form the reported default guidance. Expose the boundary: More capacity can overfit small fine-tuning sets. Complete this evidence step: Sweep depth, heads and seed. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 10: Attention interpretation

Start with the mechanism: Averaged model attention can highlight genes that strongly influence contextual representations. The worked case is concrete: Known marker genes receive high attention in a cell type. Expose the boundary: Average attention is not a causal mechanism. Complete this evidence step: Use marker, ablation and perturbation checks. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 11: Supervised fine-tuning

Start with the mechanism: A classifier head maps the cell representation to logits for reference labels and updates model parameters with cross-entropy. The worked case is concrete: A PBMC cell receives scores across nine types. Expose the boundary: Random cell splits can leak dataset signatures. Complete this evidence step: Split by donor or study. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 12: Softmax confidence

Start with the mechanism: Exponentiated logits are normalized to one, producing relative scores among known classes. The worked case is concrete: Top score 0.62 exceeds runner-up 0.25. Expose the boundary: Closed-set softmax can be high out of domain. Complete this evidence step: Calibrate on external data. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Did You Know? A sophisticated adjustment can be algebraically correct and still answer the wrong scientific question when samples, labels, batches or validation boundaries are wrong.

Workshop 13: Novel-type rejection

Start with the mechanism: If the maximum class score is below a threshold, the pipeline can return unassigned rather than the top known label. The worked case is concrete: Maximum 0.41 is rejected at threshold 0.5. Expose the boundary: One threshold has different error costs by class. Complete this evidence step: Plot coverage versus error. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 14: Worked five-gene cell

Start with the mechanism: Bin five normalized values, add tiny two-dimensional identity vectors, mask one value and calculate a simplified attention-weighted prediction. The worked case is concrete: Context favours bin 3 for the hidden gene. Expose the boundary: Toy linear attention is not the full Performer. Complete this evidence step: Change one bin and recompute. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 15: Ablations and simple baselines

Start with the mechanism: Compare full scBERT with no pretraining, no gene2vec, fewer layers and logistic regression under identical folds. The worked case is concrete: A linear baseline reveals whether the task is already separable. Expose the boundary: A single benchmark cannot establish universal value. Complete this evidence step: Publish all matched results. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 16: Governance and careers

Start with the mechanism: Version symbols, bins, gene2vec, masks, Performer, checkpoint, labels and rejection. This connects approximation, classification and bioinformatics. The worked case is concrete: Students can begin with bins and softmax. Expose the boundary: Novel-type rejection flags uncertainty; it does not name biology. Complete this evidence step: Write an scBERT audit sheet. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.