Chapter map

1. 1. Why gene order can carry information
2. 2. Corpus normalization factor
3. 3. Rank-value sequence
4. 4. Vocabulary and truncation
5. 5. Token embedding
6. 6. Masking as a learning signal
7. 7. Self-attention scores
8. 8. Multiple heads and layers
9. 9. Masked-token loss
10. 10. Cell representation
11. 11. Fine-tuning mathematics
12. 12. In-silico gene deletion
13. 13. Embedding shift and cosine distance
14. 14. Worked six-gene ranking
15. 15. External validation and baselines
16. 16. Governance and careers

Visit the How Mathematics Works hub for the wider series and its connections to education, reasoning, science, computing, design and careers.

---

1. Why gene order can carry information

This is where modelling choices begin to affect evidence. Ordering genes by context-adjusted expression creates a sequence in which early tokens are especially informative for one cell.

A concrete situation

A lowly expressed transcription factor can outrank a ubiquitous housekeeping gene after normalization.

What the mathematics does not license

Sequence order is a mathematical encoding, not physical RNA order. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Write the ordering rule before modelling. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

2. Corpus normalization factor

The central mathematical idea is concrete. Each gene receives a reference statistic calculated once from the pretraining corpus and reused for new data.

A concrete situation

Expression is divided by a gene-specific median before ranking.

What the mathematics does not license

A corpus dominated by some tissues can shift priorities. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Audit factors by tissue and gene class. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

3. Rank-value sequence

Here the geometry becomes operational. Detected genes are sorted from highest normalized rank downward and represented by token IDs.

A concrete situation

A cell begins with genes A, C, B after adjustment.

What the mathematics does not license

Ties and low-count noise can swap neighbours. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Define tie handling and stability. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

4. Vocabulary and truncation

This step deserves a careful pause. The reported model uses a gene vocabulary and a finite input size, so only a bounded ranked prefix enters one pass.

A concrete situation

Top-ranked detected genes fill at most 2,048 positions in the reported model.

What the mathematics does not license

Longer cells lose tail genes. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Measure retained expression and pathways. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

5. Token embedding

The useful question is not only how to calculate, but what the calculation assumes. A lookup table maps each gene ID to a learned vector in a common coordinate system.

A concrete situation

Gene A becomes a 256-number vector.

What the mathematics does not license

Vectors inherit pretraining data and objective. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Archive vocabulary and checkpoint together. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

6. Masking as a learning signal

A small example makes the mechanism visible. Selected tokens are concealed and the target is the original gene identity.

A concrete situation

The model sees A, mask, B and predicts C.

What the mathematics does not license

An easy mask policy can reward shortcuts. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Report mask policy and validation loss. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

7. Self-attention scores

This is where modelling choices begin to affect evidence. Queries and keys form scaled similarities; row-wise softmax makes weights that combine value vectors.

A concrete situation

A masked token attends to co-occurring regulators.

What the mathematics does not license

High weight is not causal direction. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Compare attention with perturbation evidence. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

8. Multiple heads and layers

The central mathematical idea is concrete. Parallel heads capture different contextual patterns, and stacked encoder blocks refine them through residual and feed-forward operations.

A concrete situation

One head follows a lineage programme while another follows stress.

What the mathematics does not license

Heads need not have unique biological meanings. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Repeat seeds and inspect stability. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

9. Masked-token loss

Here the geometry becomes operational. Cross-entropy penalizes low probability assigned to the true hidden gene among vocabulary alternatives.

A concrete situation

True-gene probability 0.8 gives loss minus log 0.8.

What the mathematics does not license

Token frequency can make rare genes harder. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Report loss by frequency and tissue. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

10. Cell representation

This step deserves a careful pause. Contextual token vectors can be pooled or otherwise summarized into a representation used by downstream tasks.

A concrete situation

A classifier predicts disease state from a cell embedding.

What the mathematics does not license

Pooling may hide conflicting programmes. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Compare mean, special-token and task-specific summaries. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

11. Fine-tuning mathematics

The useful question is not only how to calculate, but what the calculation assumes. Task heads and selected pretrained weights are updated by gradients on labelled examples.

A concrete situation

A binary head predicts dosage sensitivity.

What the mathematics does not license

Millions of adaptable parameters can overfit a small cohort. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Compare frozen, partial and full tuning. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

12. In-silico gene deletion

A small example makes the mechanism visible. Removing a gene token or altering its context lets the model estimate how representations or predictions change.

A concrete situation

Delete a candidate regulator and measure embedding displacement.

What the mathematics does not license

Model sensitivity is not an experimental knockout. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Validate with real perturbation data. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

13. Embedding shift and cosine distance

This is where modelling choices begin to affect evidence. Vector differences, norms and angles quantify how a perturbation moves a cell relative to reference states.

A concrete situation

A treated embedding moves toward a healthy centroid.

What the mathematics does not license

Geometry depends on checkpoint and pooling. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Publish the metric and uncertainty. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

14. Worked six-gene ranking

The central mathematical idea is concrete. Use six genes with cell counts and corpus medians; calculate adjusted ratios, rank them, mask one token and compare predicted probabilities.

A concrete situation

Gene D rises after normalization despite a smaller raw count.

What the mathematics does not license

A toy corpus has stable medians unlike real data. The habit of checking assumptions is one of the practical benefits of learning mathematics.

Try it, check it, transfer it

Perturb one median and rerank. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

15. External validation and baselines

Here the geometry becomes operational. Hold out donors, tissues and tasks; compare with logistic regression, PCA and task-specific networks under identical splits.

A concrete situation

A simple model can be competitive on an easy label.

What the mathematics does not license

Random cell splits leak donor structure. The same reasoning helps students move from an exercise to a real dataset.

Try it, check it, transfer it

Split by biological unit. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

16. Governance and careers

This step deserves a careful pause. Version corpus factors, vocabulary, sequence, masks, checkpoint, tuning and perturbation outputs. This links ranks, probability and AI.

A concrete situation

Students can begin with ratios and cross-entropy.

What the mathematics does not license

Transfer learning supplies a prior, not certainty. That distinction matters because a correct formula can still answer the wrong operational question.

Try it, check it, transfer it

Write a Geneformer model card. Then explain the result to someone who has not seen the formula. If the explanation needs a hidden assumption, add that assumption to the model record.

---

A seven-day practice lab

1. Day 1: define the decision, observation unit, X variables and Y or reference condition.
2. Day 2: reproduce the smallest worked example and check every dimension, unit and sign.
3. Day 3: build a transparent baseline and write what it assumes.
4. Day 4: fit the main method inside a training-only pipeline; record every parameter.
5. Day 5: challenge one assumption with a deliberately difficult dataset or operating regime.
6. Day 6: compare validation metrics with the practical workload created by errors.
7. Day 7: present one supported conclusion, one unresolved uncertainty and one safe next experiment.

Questions for a student, parent or mentor

• Which part of the model is a mathematical definition, and which part is a choice?
• What small calculation would catch a sign, scale or dimension error?
• What evidence comes from training, validation and the untouched test?
• What benign condition could look like a fault or prediction failure?
• How would the answer change for another response, asset, school project or time period?
• What should a human review before acting on the score?

A method-specific audit before deployment

A Geneformer audit starts before the Transformer. Each cell is converted into a rank-value encoding: genes are ordered by expression within that cell after normalization by gene-level expression statistics from the pretraining corpus. The resulting gene tokens enter a context-aware encoder trained by masked-gene prediction, so attention and hidden vectors are learned from relationships among genes rather than from cell-type labels.

Record the gene vocabulary, corpus-derived normalization factors, ranking rule, truncation length, mask choices, checkpoint, attention configuration, loss, fine-tuning split and downstream head. Rank encodings can reduce sensitivity to absolute count scale, yet the ordering still inherits corpus composition, preprocessing and detection limits.

Validate against count-based and simple baselines, donor and tissue holdouts, alternative ranking factors, frozen versus fine-tuned models, gene perturbations and calibration. Geneformer can transfer statistical structure from many cells, but a model score or in-silico perturbation remains a testable prediction rather than a causal biological result.

---

Frequently asked questions

What is Geneformer?

Geneformer is a context-aware Transformer pretrained on about 30 million human single-cell transcriptomes. It uses rank-value gene sequences and self-supervised masked-gene prediction, then transfers the learned representations to network-biology tasks.

What is rank-value encoding?

For each cell, detected genes are ranked by expression after normalization with gene-level expression statistics from the pretraining corpus. This promotes genes that distinguish cell state and deemphasizes genes that are routinely abundant.

Why does Geneformer omit exact expression values?

The method represents relative gene priority rather than raw counts. That can reduce sensitivity to absolute scale and sparse count magnitude, although ranking still depends on preprocessing and corpus-derived normalization factors.

What does masked-gene prediction learn?

Some gene tokens are hidden and the model predicts their identities from the surrounding gene context. Repeating this task across many cells trains contextual gene representations without cell-type labels.

What is transfer learning here?

A pretrained checkpoint supplies gene and cell representations that are fine-tuned or used for tasks such as gene classification, cell-state prediction and in-silico perturbation with smaller task-specific datasets.

Does an attention weight prove a regulatory connection?

No. Attention describes how a trained model distributes computation. Network or perturbation claims require independent datasets, experiments and comparisons with alternative explanations.

---

Primary source and fact-check note

Christina V. Theodoris, Ling Xiao, Anant Chopra et al., “Transfer learning enables predictions in network biology,” Nature 618, 616–624 (2023), DOI 10.1038/s41586-023-06139-9. Definitions and core claims in this article were checked against this primary publication. Calculations shown here were independently recomputed. Examples are educational illustrations; they are not reported industrial performance results.

Useful next reading

• scGPT masked generative pretraining
• TOSICA pathway-token attention
• CellTypist interpretable linear annotation
• How Mathematics Works

Keep the options open. Mathematics education supports clearer questions, transferable problem-solving skills and informed choices across science, engineering, computing, AI, statistics, design and many other careers. It does not guarantee intelligence, income, admission or employment; those outcomes depend on far more than one subject or model.

Extended laboratory: topic-specific validation and responsible use

Rank-factor audit

Store the corpus-derived normalization factor for every gene and quantify how alternative corpora change within-cell order.

Sequence-retention audit

Report detected genes, retained prefix length, omitted pathways and truncation by cell type and sequencing depth.

Mask-policy test

Repeat pretraining or probing with different eligible genes and mask patterns; rule out frequency shortcuts.

Fine-tuning ladder

Compare frozen embeddings, linear probes, partial layers, full tuning and a task-specific baseline under identical biological splits.

Perturbation reality check

Separate token deletion, embedding movement and downstream prediction from observed knockout or treatment evidence.

Attention stability

Repeat seeds and checkpoints; compare attention ranks with ablations and independent network evidence.

Domain holdout

Keep complete tissues, diseases, donors and technologies out of feature selection and tuning.

Final claim

Name factors, vocabulary, checkpoint, tuning and validation. A transferred representation is a statistical prior, not a causal map.

Mechanism workshop: calculate, challenge and transfer

Workshop 1: Why gene order can carry information

Start with the mechanism: Ordering genes by context-adjusted expression creates a sequence in which early tokens are especially informative for one cell. The worked case is concrete: A lowly expressed transcription factor can outrank a ubiquitous housekeeping gene after normalization. Expose the boundary: Sequence order is a mathematical encoding, not physical RNA order. Complete this evidence step: Write the ordering rule before modelling. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 2: Corpus normalization factor

Start with the mechanism: Each gene receives a reference statistic calculated once from the pretraining corpus and reused for new data. The worked case is concrete: Expression is divided by a gene-specific median before ranking. Expose the boundary: A corpus dominated by some tissues can shift priorities. Complete this evidence step: Audit factors by tissue and gene class. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 3: Rank-value sequence

Start with the mechanism: Detected genes are sorted from highest normalized rank downward and represented by token IDs. The worked case is concrete: A cell begins with genes A, C, B after adjustment. Expose the boundary: Ties and low-count noise can swap neighbours. Complete this evidence step: Define tie handling and stability. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 4: Vocabulary and truncation

Start with the mechanism: The reported model uses a gene vocabulary and a finite input size, so only a bounded ranked prefix enters one pass. The worked case is concrete: Top-ranked detected genes fill at most 2,048 positions in the reported model. Expose the boundary: Longer cells lose tail genes. Complete this evidence step: Measure retained expression and pathways. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 5: Token embedding

Start with the mechanism: A lookup table maps each gene ID to a learned vector in a common coordinate system. The worked case is concrete: Gene A becomes a 256-number vector. Expose the boundary: Vectors inherit pretraining data and objective. Complete this evidence step: Archive vocabulary and checkpoint together. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 6: Masking as a learning signal

Start with the mechanism: Selected tokens are concealed and the target is the original gene identity. The worked case is concrete: The model sees A, mask, B and predicts C. Expose the boundary: An easy mask policy can reward shortcuts. Complete this evidence step: Report mask policy and validation loss. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 7: Self-attention scores

Start with the mechanism: Queries and keys form scaled similarities; row-wise softmax makes weights that combine value vectors. The worked case is concrete: A masked token attends to co-occurring regulators. Expose the boundary: High weight is not causal direction. Complete this evidence step: Compare attention with perturbation evidence. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 8: Multiple heads and layers

Start with the mechanism: Parallel heads capture different contextual patterns, and stacked encoder blocks refine them through residual and feed-forward operations. The worked case is concrete: One head follows a lineage programme while another follows stress. Expose the boundary: Heads need not have unique biological meanings. Complete this evidence step: Repeat seeds and inspect stability. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 9: Masked-token loss

Start with the mechanism: Cross-entropy penalizes low probability assigned to the true hidden gene among vocabulary alternatives. The worked case is concrete: True-gene probability 0.8 gives loss minus log 0.8. Expose the boundary: Token frequency can make rare genes harder. Complete this evidence step: Report loss by frequency and tissue. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 10: Cell representation

Start with the mechanism: Contextual token vectors can be pooled or otherwise summarized into a representation used by downstream tasks. The worked case is concrete: A classifier predicts disease state from a cell embedding. Expose the boundary: Pooling may hide conflicting programmes. Complete this evidence step: Compare mean, special-token and task-specific summaries. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 11: Fine-tuning mathematics

Start with the mechanism: Task heads and selected pretrained weights are updated by gradients on labelled examples. The worked case is concrete: A binary head predicts dosage sensitivity. Expose the boundary: Millions of adaptable parameters can overfit a small cohort. Complete this evidence step: Compare frozen, partial and full tuning. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 12: In-silico gene deletion

Start with the mechanism: Removing a gene token or altering its context lets the model estimate how representations or predictions change. The worked case is concrete: Delete a candidate regulator and measure embedding displacement. Expose the boundary: Model sensitivity is not an experimental knockout. Complete this evidence step: Validate with real perturbation data. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 13: Embedding shift and cosine distance

Start with the mechanism: Vector differences, norms and angles quantify how a perturbation moves a cell relative to reference states. The worked case is concrete: A treated embedding moves toward a healthy centroid. Expose the boundary: Geometry depends on checkpoint and pooling. Complete this evidence step: Publish the metric and uncertainty. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 14: Worked six-gene ranking

Start with the mechanism: Use six genes with cell counts and corpus medians; calculate adjusted ratios, rank them, mask one token and compare predicted probabilities. The worked case is concrete: Gene D rises after normalization despite a smaller raw count. Expose the boundary: A toy corpus has stable medians unlike real data. Complete this evidence step: Perturb one median and rerank. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 15: External validation and baselines

Start with the mechanism: Hold out donors, tissues and tasks; compare with logistic regression, PCA and task-specific networks under identical splits. The worked case is concrete: A simple model can be competitive on an easy label. Expose the boundary: Random cell splits leak donor structure. Complete this evidence step: Split by biological unit. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.

Workshop 16: Governance and careers

Start with the mechanism: Version corpus factors, vocabulary, sequence, masks, checkpoint, tuning and perturbation outputs. This links ranks, probability and AI. The worked case is concrete: Students can begin with ratios and cross-entropy. Expose the boundary: Transfer learning supplies a prior, not certainty. Complete this evidence step: Write a Geneformer model card. Write the input matrix or token list, every transformation, the output, uncertainty and one observation that would contradict the conclusion. Repeat under a nearby defensible setting and compare ranks, masks, value projections, attention approximations, mixture probabilities, KL terms, reconstruction errors, embeddings, probabilities or calibration. Use untouched donors, studies, tissues or controlled downsampling because random cells can share hidden technical and biological structure with training data. Keep measurement, model estimate, interpretation and decision on separate notebook lines so students and reviewers can see what the mathematics establishes.