eduKateSG · Why Science?
Index long references and varied genomic formats with a configurable hierarchy—without losing the coordinate model, sort contract or source-file identity
Choose the closest route, then return to the full index whenever you need the wider evidence chain.
Science learning becomes powerful when students can tell a measured pattern from a model, a statistical decision from a mechanism, and a useful lead from a finished conclusion. Coordinate-Sorted Index, or CSI, is a general genomic index format for coordinate-sorted files. The official HTS specifications describe CSI as the successor to BAI and parameterise its hierarchical binning with min_shift, the size exponent of the smallest bins, and depth, the number of subdivision levels below the root. Using min_shift 14 and depth 5 reproduces the BAI-style six-level hierarchy, while other values can address longer references. CSI supports multiple data formats through auxiliary metadata, but the correct parser, coordinate convention, sort order and exact source bytes remain essential. This guide supports Primary Science, PSLE Science, Secondary Science, O-Level Science and STEM education as a longform evidence-reading route; it is not medical advice, a laboratory protocol, an admission promise or a career guarantee.
Related eduKate reading: BAM and binary sequence alignment storage; BCF and binary variant call storage; BED and genomic feature coordinates; Science Learning Hub; Education Hub; STEM Education; Careers by Subject Capability.
Primary and official sources checked on 12 October 2026: Official CSI coordinate-sorted index specification; Official SAM/BAM Format Specification Working Group specification, version 1.6, 12 August 2025; Official HTS format specifications index; 2026 Singapore–Cambridge O-Level Biology syllabus; MOE G2/G3 Lower Secondary Science teaching and learning syllabus; 2026 MOE G2 Computing syllabus. Specifications, tools and recommendations can change, so preserve the dated documentation, exact source bytes, parser rules and software versions used.
A deeper reading route
CSI takes the useful geometric idea behind BAI and makes it configurable. Instead of fixing one hierarchy forever, the file records how many low bits define a smallest window and how many times the coordinate space is divided into eight-way levels. This makes the index suitable for assemblies with long reference sequences and for formats beyond BAM.
The parameters are part of the scientific access contract. min_shift controls the smallest bin width as a power of two, while depth controls the total span reachable by the hierarchy. Smaller bins can reduce over-reading for narrow queries but can enlarge the index. A larger depth expands the coordinate range. Good choices respond to reference length and workload rather than a copied command.
Like BAI, CSI maps bins to chunks described by virtual offsets in a BGZF-backed file. A query is converted to candidate bins, the chunks are gathered and refined, and records are decoded and checked for exact overlap. The index deliberately returns candidates. It cannot decide whether an alignment, feature or variant is scientifically convincing.
CSI can carry auxiliary data describing how a generic tabular file should be interpreted: preset format, sequence column, begin and end columns, meta character, skipped lines and reference names. That flexibility makes provenance crucial. A one-column shift or wrong zero-versus-one-based preset can build a syntactically valid index that retrieves the wrong scientific intervals.
The loffset stored for bins can help readers avoid data ending before a query start. Implementations may choose different internal optimisations while producing compatible results. Validation should compare retrieved logical records, not assume that independent index files must be byte-identical.
Metadata pseudo-bins and optional counts extend observability without becoming biological evidence. A count can help quality control, but it inherits the file’s classification of mapped, unmapped or placed records. The safest report names what was counted and which file state produced the index.
CSI readers also need defensive arithmetic. Bin identifiers and coordinate limits are calculated from shifts, depth and counts supplied by the file. Unchecked shifts or multiplications can overflow, and a malicious count can request unreasonable memory. A scientific parser therefore applies resource bounds, validates every chunk range and reports structural failure rather than returning a reassuring empty result.
Interoperability should be tested as a result, not assumed from a shared extension. One tool may create CSI for BAM, another for BCF, and a third for generic tabular text with auxiliary configuration. Each consumer must agree about reference names, interval semantics and BGZF virtual offsets. A small cross-tool fixture suite is often more informative than a feature checklist because it tests the complete contract under real boundary conditions.
Index design can be tuned to workload. Very fine smallest bins may help many narrow queries but add index entries, while broader bins can increase decoding of irrelevant records. Remote object storage adds latency and range-request costs to that trade-off. A responsible benchmark records query width, cache state, object location and returned records so a speed result remains interpretable. It should also repeat cold-cache and warm-cache runs, because caching can dominate the apparent benefit while leaving coordinate correctness entirely untested.
For students, CSI is a powerful example of a scalable mathematical model. One hierarchy can be tuned for a short teaching genome or a very long real reference by changing two documented parameters. The cheerful lesson is that generality comes from making assumptions explicit, not from pretending every dataset is the same.
| Stage | Question | Evidence | Responsible output |
|---|---|---|---|
| Parameterise | What coordinate span and smallest bins are needed? | min_shift, depth and reference lengths | A suitable hierarchy |
| Index | Which file chunks may contain each interval? | Bins, loffsets and virtual-offset chunks | A compact search structure |
| Retrieve | Which records truly overlap the query? | Candidate chunks plus format-aware decoding | A verified regional result |
| Govern | Does the CSI match source bytes and parser rules? | Hashes, auxiliary metadata and equivalence tests | A reproducible access path |
Inside this guide
1–12 · Foundations and design
- 1. Start with the coordinate workload
- 2. Read the official quick reference
- 3. Treat CSI as format-general
- 4. Validate the CSI magic
- 5. Understand min_shift
- 6. Understand depth
- 7. Recover the BAI-style hierarchy
- 8. Choose parameters for long references
- 9. Use zero-based half-open coordinates
- 10. Assign records to bins
- 11. Enumerate query bins
- 12. Read chunks as virtual ranges
13–24 · Models and estimation
- 13. Use loffset correctly
- 14. Merge candidate chunks
- 15. Read auxiliary data
- 16. Keep reference names ordered
- 17. Respect sort order
- 18. Bind the index to exact bytes
- 19. Use metadata pseudo-bins carefully
- 20. Handle absent optional fields
- 21. Distinguish index compatibility
- 22. Compare index size and query cost
- 23. Check overflow and allocation
- 24. Test reference-end queries
25–36 · Diagnostics and interpretation
- 25. Test very long records
- 26. Compare with a sequential scan
- 27. Plan remote access
- 28. Connect to Primary Science
- 29. Connect to Secondary Science
- 30. Connect to mathematics
- 31. Connect to computing
- 32. Connect to pathways
- 33. Did you know?
- 34. Write a reader-ready CSI note
- 35. Build a validation ladder
- 36. Finish with a reproducible bundle
Section 1 of 36
1. Start with the coordinate workload
CSI should be chosen from the longest reference, typical query widths, source format and downstream tool support.
Working checkpoint — CSI: Write those constraints before selecting parameters. Keep source and CSI hashes, format parser, reference order, min_shift, depth, builder and consumer versions together.
For a classroom model, build an eight-way nested map with invented observations.
Section 2 of 36
2. Read the official quick reference
The official hts-specs index describes CSI as a successor to BAI, and CSIv1 defines its binary fields and algorithms.
Working checkpoint — coordinate index: Record the document revision and implementation version. Read query speed beside coordinate range, index size, over-read, boundary correctness and sequential equivalence.
For future study, connect indexing to genomics, discrete mathematics and databases while checking pathways separately.
Section 3 of 36
3. Treat CSI as format-general
CSI can index coordinate-sorted BAM, BCF and suitable tabular formats when the parser contract is defined.
Working checkpoint — min_shift: Name the source format explicitly rather than saying CSI data. Preserve records outside requested regions and all parser failures; a successful query is a bounded view.
For scientific writing, publish parameters, format rules, hashes and equivalence tests.
Section 4 of 36
4. Validate the CSI magic
A CSI file begins with defined magic bytes followed by its hierarchy parameters and optional auxiliary data.
Working checkpoint — depth: Check type and bounds before parsing variable-length arrays. Separate a configurable search hierarchy from the validity of source records and biological conclusions.
For students, ask how two parameters resize one nested map for different worlds.
Section 5 of 36
5. Understand min_shift
min_shift is the base-two exponent for the smallest bin width. A value of 14 corresponds to 16-kilobase smallest bins.
Working checkpoint — bin hierarchy: Translate the parameter into bases when documenting it. Treat hierarchy parameters, reference identifiers and coordinate convention as scientific inputs: silent defaults can change retrieval.
For parents, ask whether long references and software compatibility were checked.
Section 6 of 36
6. Understand depth
Depth specifies how many subdivision levels sit below the single root bin. Each level divides coordinate space into eightfold finer regions.
Working checkpoint — auxiliary data: Calculate the maximum coordinate range before indexing. Keep source and CSI hashes, format parser, reference order, min_shift, depth, builder and consumer versions together.
For a classroom model, build an eight-way nested map with invented observations.
Section 7 of 36
7. Recover the BAI-style hierarchy
The CSI specification notes that min_shift 14 and depth 5 represent the BAI-style scheme with six levels including the root.
Working checkpoint — CSI: Use this relationship to compare formats, not to erase their file differences. Read query speed beside coordinate range, index size, over-read, boundary correctness and sequential equivalence.
For future study, connect indexing to genomics, discrete mathematics and databases while checking pathways separately.
Section 8 of 36
8. Choose parameters for long references
Adding depth or changing shift can extend addressable coordinates beyond BAI’s fixed limit. Excessive range can increase bins and compatibility requirements.
Working checkpoint — coordinate index: Use the smallest documented configuration that safely covers every contig. Preserve records outside requested regions and all parser failures; a successful query is a bounded view.
For scientific writing, publish parameters, format rules, hashes and equivalence tests.
Section 9 of 36
9. Use zero-based half-open coordinates
CSI binning algorithms operate on zero-based half-open intervals. Input formats may present coordinates differently.
Working checkpoint — min_shift: Test first-base, one-base and end-exclusive conversions. Separate a configurable search hierarchy from the validity of source records and biological conclusions.
For students, ask how two parameters resize one nested map for different worlds.
Section 10 of 36
10. Assign records to bins
A record interval is placed in the smallest bin that completely contains it under the chosen hierarchy.
Working checkpoint — depth: Test long features that cross fine-bin boundaries. Treat hierarchy parameters, reference identifiers and coordinate convention as scientific inputs: silent defaults can change retrieval.
For parents, ask whether long references and software compatibility were checked.
Section 11 of 36
11. Enumerate query bins
A query interval maps to all hierarchy bins that could contain an overlapping record. Omitting one level creates false negatives.
Working checkpoint — bin hierarchy: Compare implementation output with the specification pseudocode. Keep source and CSI hashes, format parser, reference order, min_shift, depth, builder and consumer versions together.
For a classroom model, build an eight-way nested map with invented observations.
Section 12 of 36
12. Read chunks as virtual ranges
Bins contain chunks bounded by BGZF virtual offsets. They describe where to read, not what conclusion to draw.
Working checkpoint — auxiliary data: Decode and filter every candidate record for exact overlap. Read query speed beside coordinate range, index size, over-read, boundary correctness and sequential equivalence.
For future study, connect indexing to genomics, discrete mathematics and databases while checking pathways separately.
Section 13 of 36
13. Use loffset correctly
CSI stores a lower-offset value for a bin to help skip chunks that cannot overlap the query start. Its interpretation depends on bin relationships.
Working checkpoint — CSI: Test sparse files and queries beginning in empty regions. Preserve records outside requested regions and all parser failures; a successful query is a bounded view.
For scientific writing, publish parameters, format rules, hashes and equivalence tests.
Section 14 of 36
14. Merge candidate chunks
Readers can merge overlapping or nearby chunks to reduce seeks. Merging must preserve all candidate bytes and avoid duplicate processing.
Working checkpoint — coordinate index: Compare record identities before and after optimisation. Separate a configurable search hierarchy from the validity of source records and biological conclusions.
For students, ask how two parameters resize one nested map for different worlds.
Section 15 of 36
15. Read auxiliary data
CSI may include opaque auxiliary bytes interpreted by a format-specific convention, such as generic tabix configuration and reference names.
Working checkpoint — min_shift: Preserve and decode auxiliary data with the appropriate format rules. Treat hierarchy parameters, reference identifiers and coordinate convention as scientific inputs: silent defaults can change retrieval.
For parents, ask whether long references and software compatibility were checked.
Section 16 of 36
16. Keep reference names ordered
Reference indexes align with the source header or auxiliary name list. Reordering names changes the meaning of every numeric reference ID.
Working checkpoint — depth: Compare names, order and lengths where available. Keep source and CSI hashes, format parser, reference order, min_shift, depth, builder and consumer versions together.
For a classroom model, build an eight-way nested map with invented observations.
Section 17 of 36
17. Respect sort order
CSI assumes records are sorted compatibly by reference and coordinate. The index cannot repair an out-of-order source file.
Working checkpoint — bin hierarchy: Verify actual order rather than trusting a header label. Read query speed beside coordinate range, index size, over-read, boundary correctness and sequential equivalence.
For future study, connect indexing to genomics, discrete mathematics and databases while checking pathways separately.
Section 18 of 36
18. Bind the index to exact bytes
Virtual offsets change when the source BGZF stream is recompressed or rewritten. A logically equivalent file still needs a new CSI.
Working checkpoint — auxiliary data: Store source and index hashes together. Preserve records outside requested regions and all parser failures; a successful query is a bounded view.
For scientific writing, publish parameters, format rules, hashes and equivalence tests.
Section 19 of 36
19. Use metadata pseudo-bins carefully
CSI can include reference-level metadata pseudo-bins similar to BAI. Their bin number is derived from the configured hierarchy.
Working checkpoint — CSI: Do not enumerate pseudo-bins as genomic intervals. Separate a configurable search hierarchy from the validity of source records and biological conclusions.
For students, ask how two parameters resize one nested map for different worlds.
Section 20 of 36
20. Handle absent optional fields
Optional metadata or unmapped counts may be absent. Absence must remain unknown instead of becoming zero.
Working checkpoint — coordinate index: Use explicit missing states in reports. Treat hierarchy parameters, reference identifiers and coordinate convention as scientific inputs: silent defaults can change retrieval.
For parents, ask whether long references and software compatibility were checked.
Section 21 of 36
21. Distinguish index compatibility
Two tools may both read CSI yet differ in supported source formats, presets or maximum parameters.
Working checkpoint — min_shift: Test every production consumer with representative files. Keep source and CSI hashes, format parser, reference order, min_shift, depth, builder and consumer versions together.
For a classroom model, build an eight-way nested map with invented observations.
Section 22 of 36
22. Compare index size and query cost
Finer bins may reduce extra decoding but increase index entries; coarse bins can do the reverse.
Working checkpoint — depth: Benchmark narrow, broad and empty-region queries. Read query speed beside coordinate range, index size, over-read, boundary correctness and sequential equivalence.
For future study, connect indexing to genomics, discrete mathematics and databases while checking pathways separately.
Section 23 of 36
23. Check overflow and allocation
Malicious or corrupt counts and parameter values can produce enormous calculated structures.
Working checkpoint — bin hierarchy: Bound shifts, depth, counts and multiplications before allocation. Preserve records outside requested regions and all parser failures; a successful query is a bounded view.
For scientific writing, publish parameters, format rules, hashes and equivalence tests.
Section 24 of 36
24. Test reference-end queries
Intervals at the maximum representable coordinate and at actual contig ends expose range and conversion bugs.
Working checkpoint — auxiliary data: Include both successful and refused boundary cases. Separate a configurable search hierarchy from the validity of source records and biological conclusions.
For students, ask how two parameters resize one nested map for different worlds.
Section 25 of 36
25. Test very long records
Structural variants, spliced alignments or long annotations may span many smallest bins and belong in broader bins.
Working checkpoint — CSI: Use fixtures that cross several hierarchy levels. Treat hierarchy parameters, reference identifiers and coordinate convention as scientific inputs: silent defaults can change retrieval.
For parents, ask whether long references and software compatibility were checked.
Section 26 of 36
26. Compare with a sequential scan
For representative intervals, CSI-backed results should equal a format-aware full scan after applying the same overlap rule.
Working checkpoint — coordinate index: Compare complete record identities, not counts alone. Keep source and CSI hashes, format parser, reference order, min_shift, depth, builder and consumer versions together.
For a classroom model, build an eight-way nested map with invented observations.
Section 27 of 36
27. Plan remote access
CSI supports selective retrieval when the server returns stable byte ranges for the indexed BGZF object.
Working checkpoint — min_shift: Validate object identity, caching, range semantics and retries. Read query speed beside coordinate range, index size, over-read, boundary correctness and sequential equivalence.
For future study, connect indexing to genomics, discrete mathematics and databases while checking pathways separately.
Section 28 of 36
28. Connect to Primary Science
A nested map can divide a fictional habitat into large, medium and small regions, guiding pupils toward relevant observation cards.
Working checkpoint — depth: Explain that the map finds candidates, not conclusions. Preserve records outside requested regions and all parser failures; a successful query is a bounded view.
For scientific writing, publish parameters, format rules, hashes and equivalence tests.
Section 29 of 36
29. Connect to Secondary Science
CSI shows how one explicit mathematical model can scale across datasets while scientific interpretation remains format-specific.
Working checkpoint — bin hierarchy: Separate indexing assumptions from measurement assumptions. Separate a configurable search hierarchy from the validity of source records and biological conclusions.
For students, ask how two parameters resize one nested map for different worlds.
Section 30 of 36
30. Connect to mathematics
Powers of two, octree-like subdivision, half-open intervals and unions of ranges make the hierarchy a rich discrete-mathematics example.
Working checkpoint — auxiliary data: Calculate one toy hierarchy by hand. Treat hierarchy parameters, reference identifiers and coordinate convention as scientific inputs: silent defaults can change retrieval.
For parents, ask whether long references and software compatibility were checked.
Section 31 of 36
31. Connect to computing
Configurable binary formats, overflow safety, random access and interoperability tests show robust systems design.
Working checkpoint — CSI: Fuzz parameter and chunk parsing with strict resource limits. Keep source and CSI hashes, format parser, reference order, min_shift, depth, builder and consumer versions together.
For a classroom model, build an eight-way nested map with invented observations.
Section 32 of 36
32. Connect to pathways
CSI connects genomics, algorithms, databases and cloud storage and can illuminate STEM education and career pathways.
Working checkpoint — coordinate index: Check current official qualifications separately. Read query speed beside coordinate range, index size, over-read, boundary correctness and sequential equivalence.
For future study, connect indexing to genomics, discrete mathematics and databases while checking pathways separately.
Section 33 of 36
33. Did you know?
With min_shift 14 and depth 5, CSI represents the same six-level bin geometry associated with BAI while retaining its more general file structure.
Working checkpoint — min_shift: Use the equivalence to teach parameterisation. Preserve records outside requested regions and all parser failures; a successful query is a bounded view.
For scientific writing, publish parameters, format rules, hashes and equivalence tests.
Section 34 of 36
34. Write a reader-ready CSI note
Name source format and hash, reference, sort order, min_shift, depth, auxiliary configuration, builder, consumers and tests.
Working checkpoint — depth: State the exact overlap rule used for validation. Separate a configurable search hierarchy from the validity of source records and biological conclusions.
For students, ask how two parameters resize one nested map for different worlds.
Section 35 of 36
35. Build a validation ladder
Check magic, parameters, coordinate range, auxiliary data, reference order, bins, loffsets, chunks and full-scan equivalence.
Working checkpoint — bin hierarchy: Stop at any mismatch before scientific use. Treat hierarchy parameters, reference identifiers and coordinate convention as scientific inputs: silent defaults can change retrieval.
For parents, ask whether long references and software compatibility were checked.
Section 36 of 36
36. Finish with a reproducible bundle
Deliver source and CSI hashes, header or name list, parameters, command, environment, boundary fixtures and comparison logs.
Working checkpoint — auxiliary data: Make every indexed query reconstructable from declared assumptions. Keep source and CSI hashes, format parser, reference order, min_shift, depth, builder and consumer versions together.
For a classroom model, build an eight-way nested map with invented observations.
Contents · Previous section · Continue to the Science Learning Hub