A transcript table contains thousands of abundance estimates. A gene table comes out the other end. The totals look plausible, yet some transcripts never reached a gene. Before interpreting a biological difference, check the join that connected the two tables.
This lesson separates a harmless difference in identifier formatting from a change in the underlying annotation. It uses small, invented records to show how unmatched identifiers can remove counts, how duplicate mappings can add them twice, and why a successful import is only the beginning of a defensible analysis.
The wider subject is The Gold Standard of Bioinformatics. As part of The Gold Standard of Education and How X Works, the aim is to make the reasoning portable: the same questions apply whenever one scientific table is joined to another.
eduKateSG · The Gold Standard of Bioinformatics
Choose your reading route
Identify the reference relationship · Chapters 1–2. Build the quantities before using a formula.
Trace lost and duplicated counts · Chapters 3–5. Follow original examples, assumptions and changed conditions.
Audit the repair and transfer · Chapters 6–8. Test transfer, answer common questions and follow connected reading.
Full chapter index · Practice and evidence checks · Return to Bioinformatics
Chapter index
Identify the reference relationship · Chapters 1–2
Trace lost and duplicated counts · Chapters 3–5
Audit the repair and transfer · Chapters 6–8
CHAPTER 1 OF 8 · Identify the reference relationship
1. Start with what a transcript-to-gene mapping means
A transcript quantifier estimates quantities for the transcript sequences in its reference. A transcript-to-gene table supplies a relationship: this transcript belongs to this gene under this annotation. Summarising transcript estimates by gene therefore depends on two things agreeing—the quantified reference and the mapping used to group its records.
Think of a library catalogue. A shelf label helps find a book, but changing the punctuation of the label does not establish that two editions contain the same text. Likewise, matching character strings is necessary for an exact join, while biological compatibility requires the correct reference and annotation history.
There are several distinct identifiers to keep: the assembly, the annotation release, the transcript identifier and its version, and the gene identifier. A transcript version is not the same as a database release number. Many records may be unchanged between releases; another record may change while retaining the stable part of its identifier.
The Ensembl stable-ID documentation explains that transcript version increments can reflect changes in splicing, chromosomal location or cDNA sequence. The practical consequence is modest but important: removing the suffix changes the string being compared, not the history of the sequence it names.
The FASTA reference guide and GTF annotation guide help distinguish the sequence records from the feature relationships that accompany them.
CHAPTER 2 OF 8 · Identify the reference relationship
2. Recognise three different reasons for a failed match
First, there can be a representation mismatch. The quantifier might report a versioned identifier while a verified mapping from the same reference omits its version suffix. Once that provenance is established and the transformation remains one-to-one, a documented format conversion may be reasonable.
Second, there can be a reference mismatch. The quantified sequences might come from one annotation release while the gene map comes from another. A transcript might have changed, disappeared, or been regrouped. Making its stable portion match does not show that the new record represents the quantified sequence correctly.
Third, there can be a namespace or parsing mismatch. A sequence header may contain several fields, while the mapping expects one particular field. Two resources may use entirely different identifiers. Taking everything before a punctuation mark can damage names when the delimiter has a different meaning in that resource.
The tximport vignette documents identifier-handling options, including ignoring transcript versions or text after a bar in appropriate imports. These are explicit parsing choices, not evidence that arbitrary references are interchangeable. Check the installed software’s documented behaviour and retain its messages.
A warning, an error, a partially matched import and a custom inner join are different outcomes. We do not assume every program silently discards records. The examples below deliberately use a simple exact-key join so the consequence of a missing key is visible.
CHAPTER 3 OF 8 · Trace lost and duplicated counts
3. Audit a three-transcript join by hand
Suppose a fictional quantification contains TXA.1 with estimated count 60, TXB.2 with 30, and TXC.1 with 10. These are invented labels, not actual Ensembl records. Their total is 100.
The supplied map contains TXA.1 → Gene A, TXB.3 → Gene B and TXD.1 → Gene D. An exact inner join retains only TXA.1. TXB.2 does not equal TXB.3, and TXC.1 has no corresponding key. The presence of TXD.1 in the map does not help because it was not quantified.
| Quantified record | Estimated count | Exact key in supplied map? | Immediate result |
|---|---|---|---|
| TXA.1 | 60 | Yes | Count can be assigned to Gene A |
| TXB.2 | 30 | No; map contains TXB.3 | Investigate the version/reference difference |
| TXC.1 | 10 | No | Investigate absent annotation or incomplete mapping |
The row-match fraction is 1/3, approximately 33.3%. The retained-count fraction is 60/100 = 60%. Both are useful. A high row-match percentage can conceal loss of one abundant transcript, while many unmatched zero-count records may have little immediate count impact. Neither statistic proves biological compatibility.
If we remove the final version suffix from every identifier, TXA and TXB now match, retaining 90 counts. That is a larger numerical match, but it leaves two questions: is TXB.3 an appropriate representation of the quantified TXB.2, and why is TXC missing? Reporting “90% mapped” is not a complete repair.
Contrast this with a map documented as belonging to the exact quantification reference but storing TXA, TXB and TXC without versions. If the version-removal rule creates unique keys and the verified reference crosswalk assigns all three correctly, the conversion can be justified. The decisive difference is provenance, not the attractiveness of the new match percentage.
CHAPTER 4 OF 8 · Trace lost and duplicated counts
4. Follow the missing counts into a misleading comparison
Loss can vary between samples even when the same mapping is applied to all of them. Imagine three transcript estimates, each assigned to a different gene under the intended reference. Gene C’s transcript is absent from the map. The following numbers are deliberately simple estimated-count units.
| Quantity | Sample 1 | Sample 2 |
|---|---|---|
| Gene A transcript | 40 | 40 |
| Gene B transcript | 50 | 10 |
| Gene C transcript, unmatched | 10 | 50 |
| Total before the join | 100 | 100 |
| Total retained after losing C | 90 | 50 |
| A as a fraction of retained counts | 40/90 ≈ 0.444 | 40/50 = 0.8 |
Dividing the last two values gives 0.8/0.444… = 1.8. A naive analysis could describe an 80% increase in A’s retained-count share. Before the faulty join, its share was 40/100 = 0.4 in both samples. The altered denominator created this particular change.
This example models a simple proportion calculation. It is not a claim that every differential-expression method will return a 1.8-fold change; model-based normalisation has its own assumptions and implementation. The lesson is that upstream losses can become downstream analytical differences, so the retained fraction must be checked separately for every sample.
The opposite problem is count duplication. Suppose TXA.1, carrying 40 counts, appears twice in a mapping, once for Gene A and once for Gene B. A naive join followed by summation can turn a 100-count input into a 140-count output. A table that looks more complete can therefore violate conservation rather than fix missing data.
For a declared one-transcript-to-one-gene grouping, verify that each transcript has one permitted gene assignment and is used once. Repeated identical rows can sometimes be removed after confirming they are accidental duplicates. Conflicting assignments require investigation; choosing the first row is an arbitrary biological decision disguised as data cleaning.
CHAPTER 5 OF 8 · Trace lost and duplicated counts
5. Repair the relationship before repairing the totals
Recover the reference manifest first: species, assembly, annotation source and release, sequence-file identity, quantifier index and the method used to construct the gene map. A filename containing “latest” is not a reproducible reference description. Retain checksums or other suitable content identifiers alongside human-readable names.
Then compare the actual quantified transcript keys with the mapping keys. Preserve the original values, generate an unmatched-record table, and inspect high-count failures first. Separate absent keys, version differences, duplicate keys and conflicting gene assignments. They need different remedies.
If the issue is proven to be formatting alone, apply the smallest documented transformation to both sides as appropriate. Verify that it does not collapse two distinct identifiers into one. Record the number of affected keys and reassess both row coverage and count-weighted coverage.
If the issue is a different biological reference, use the annotation associated with the quantified reference or a validated, explicit crosswalk. Re-quantification may be necessary when the intended sequence set itself changes. Renaming output rows cannot add transcripts absent from the original index or redistribute ambiguous fragments according to a new sequence model.
Finally, keep unmatched evidence visible in the report. “Not represented after mapping” is different from “measured as zero.” A zero implies a defined measurement or estimate for a represented feature; a missing relationship describes a failure to establish where the record belongs.
The refget guide develops sequence identity. The isoform-mixture and effective-length lesson explains why even a correct grouping does not make raw gene counts identical to molecule abundance.
CHAPTER 6 OF 8 · Audit the repair and transfer
6. Test the repair in a changed context
Use three checkpoints before interpreting a repaired matrix. Confirm that the key transformation is justified by the resource’s identifier rules. Confirm that each included transcript has the intended assignment. Confirm that the chosen aggregation conserves the quantity it claims to aggregate, allowing explicitly documented exclusions.
For direct summation of estimated transcript counts with a unique gene assignment, the sum of mapped transcript counts should equal the sum of the resulting gene counts, apart from numerical rounding. Do not apply that exact invariant blindly to abundance-derived counts or other transformations that deliberately rescale values.
Transfer task one: 99 of 100 transcript rows match, but the unmatched row contains half of the sample’s estimated counts. Is 99% coverage reassuring? No. Row coverage and count-weighted coverage answer different questions; the high-abundance missing record could materially affect the analysis.
Transfer task two: removing versions yields a complete key match, but two quantified identifiers collapse onto the same stable key. Is the repair complete? No. Investigate whether the reference intentionally contains distinct versions or alternative records, and whether the target mapping can represent them. A many-to-one text conversion needs an explicit aggregation decision.
Transfer task three: every identifier matches, but all samples were quantified against one release and the gene map belongs to another. Does complete matching prove compatibility? No. A string can persist while its relationships change. Check the reference provenance and relevant annotation differences.
Save an audit summary containing input rows, unique keys, matched rows, unmatched estimated counts by sample, duplicated assignments, the transformation rule and the final aggregation check. This is a compact explanation of what the analysis actually did, not a substitute for reading the biological results.
CHAPTER 7 OF 8 · Audit the repair and transfer
7. Frequently asked questions about identifier repairs
Should I always remove transcript version numbers?
No. Keep them unless the import requires a documented conversion and you can establish that the relationship remains valid. A version suffix can carry useful evidence about the sequence model.
Can gene symbols replace stable identifiers?
They are useful reader-facing labels, but require their own mapping and ambiguity checks. Do not assume a familiar symbol uniquely identifies a feature across species, resources and releases. Retain the underlying identifier and provenance.
Is a successful software import enough?
It establishes that the program accepted the inputs under its rules. It does not establish that the annotation answers your biological question. Inspect messages and unmatched records, then verify the reference relationship.
Can I insert zero for every missing gene?
Only when the data model justifies what that zero means. An absent mapping, an unquantified sequence and a represented feature with estimated count zero are different states. Preserve those distinctions rather than making the table look complete.
CHAPTER 8 OF 8 · Audit the repair and transfer
8. Follow the records into the wider subject
Reference and annotation foundations
- Bioinformatics subject root connects computational steps to biological evidence.
- FASTA sequence records explains the reference sequences being named.
- GTF gene and transcript annotations develops feature relationships and identifiers.
- GFF3 feature annotations offers a related annotation format with its own relationship rules.
- refget and sequence identity connects labels to identifiable sequence content.
Interpreting a correctly assembled matrix
- Isoform mixtures, effective length and gene counts examines the difference between fragments and molecules.
- Spike-in reference units and RNA per cell explains why correct labels alone do not establish an absolute biological scale.
Authoritative documentation
- Ensembl stable identifiers describes identity and versioning rules.
- Bioconductor tximport vignette documents transcript-to-gene import and relevant identifier options.
A defensible gene table should let another reader trace a value back through its transcript assignment to the reference that was actually quantified. That trace is the foundation for interpreting the number.