eduKateSG · Why Science?
Compress enormous genomic files into independently addressable blocks—then preserve the offsets, integrity checks and provenance that make retrieval trustworthy
Choose the closest route, then return to the full index whenever you need the wider evidence chain.
Science learning becomes powerful when students can tell a measured pattern from a model, a statistical decision from a mechanism, and a useful lead from a finished conclusion. Blocked GNU Zip Format, or BGZF, is a gzip-compatible representation made from a series of independently compressed blocks. The SAM/BAM specification defines BGZF for BAM and describes a virtual file offset that combines a block’s compressed-file position with a position inside its uncompressed data. The current HTSlib bgzip documentation says input is divided into blocks no larger than 64 kilobytes before or after compression, including headers. This architecture supports compression and random access together. It does not prove that the genomic records are correctly sorted, scientifically valid, complete or paired with the right index. This guide supports Primary Science, PSLE Science, Secondary Science, O-Level Science and STEM education as a longform evidence-reading route; it is not medical advice, a laboratory protocol, an admission promise or a career guarantee.
Related eduKate reading: BAM and binary sequence alignment storage; BCF and binary variant call storage; htsget and secure genomic data streaming; Science Learning Hub; Education Hub; STEM Education; Careers by Subject Capability.
Primary and official sources checked on 12 October 2026: Official SAM/BAM Format Specification Working Group specification, version 1.6, 12 August 2025; Official HTSlib bgzip manual, current documentation viewed 12 October 2026; Official SAMtools HTS specifications repository; 2026 Singapore–Cambridge O-Level Biology syllabus; MOE G2/G3 Lower Secondary Science teaching and learning syllabus; 2026 MOE G2 Computing syllabus. Specifications, tools and recommendations can change, so preserve the dated documentation, exact source bytes, parser rules and software versions used.
A deeper reading route
Ordinary whole-stream compression is excellent when a reader starts at byte zero and continues to the end. Genomic analysis often asks a different question: retrieve the records overlapping one region from a file containing millions of records. BGZF answers by concatenating bounded gzip members. Each block can be decompressed independently, so software can jump near the desired records instead of expanding everything before them.
A BGZF block remains recognisable as a gzip member. Its header contains an extra subfield identified as BC, and that field records the total compressed block size minus one. Readers use the size to find the next member. Compatibility with gzip is useful for sequential decompression, but generic gzip software does not understand genomic indexes or the meaning of virtual offsets.
A virtual offset is a two-part address. The high portion identifies the compressed-file offset of a BGZF block, while the low portion identifies a byte inside the uncompressed block. Comparing or storing that address only makes sense for one exact byte stream. Recompressing equivalent text can change block boundaries and invalidate every saved virtual offset without changing the visible records.
The HTSlib bgzip manual also documents a .gzi index containing pairs of compressed and uncompressed offsets. That index supports seeking by positions in the uncompressed stream. A .gzi file is different from a genomic coordinate index such as BAI, CSI or TBI: one maps byte spaces, while the other maps reference intervals to chunks of the encoded file.
Parallel compression, thread counts and compression levels can improve throughput, yet they can also produce different block layouts. Reproducibility therefore separates logical equality from byte-for-byte identity. If an archive is cited by checksum or accompanied by an index, the exact compressed bytes matter; if only decoded content matters, a semantic comparison is needed as well.
Integrity checks exist at several layers. Gzip members carry CRC and size checks; the BGZF end-of-file marker can help detect truncation; file hashes can verify transfer identity; parsers can test block boundaries; and record-aware tools can validate decoded content. No single check replaces the others. A valid checksum can faithfully preserve the wrong file, and a valid block can contain a scientifically unsuitable record.
For students, BGZF is a delightful library analogy. Imagine sealing a long encyclopedia into numbered transparent packets: each packet is compact, and an index tells you which packet to open. The lesson links compression, coordinates, data structures and scientific care—speed becomes useful only when meaning and provenance travel with it.
| Stage | Question | Evidence | Responsible output |
|---|---|---|---|
| Partition | How is the stream split for independent decoding? | Bounded gzip-compatible BGZF blocks | A seekable compressed stream |
| Address | How is a location inside the compressed stream represented? | Compressed block offset plus in-block offset | A virtual file position |
| Retrieve | How does software find relevant blocks? | GZI or genomic coordinate index plus sorting | Efficient selective access |
| Verify | Did the exact stream and decoded content survive? | CRC, EOF marker, hashes and semantic tests | A trustworthy compressed dataset |
Inside this guide
1–12 · Foundations and design
- 1. Start with the access question
- 2. Use the current documentation
- 3. Treat BGZF as a blocked stream
- 4. Read the gzip header first
- 5. Find the BC subfield
- 6. Respect block size limits
- 7. Decompress blocks independently
- 8. Understand concatenated gzip behaviour
- 9. Read a virtual offset as two numbers
- 10. Respect the in-block range
- 11. Keep offsets tied to exact bytes
- 12. Do not subtract offsets casually
13–24 · Models and estimation
- 13. Understand the GZI index
- 14. Separate GZI from genomic indexes
- 15. Connect coordinate order
- 16. Connect to BAM
- 17. Connect to BCF and compressed VCF
- 18. Connect to tabix
- 19. Use the end-of-file marker
- 20. Check every block CRC
- 21. Detect truncation beyond the marker
- 22. Validate concatenation deliberately
- 23. Plan parallel compression
- 24. Plan parallel decompression
25–36 · Diagnostics and interpretation
- 25. Benchmark the right workload
- 26. Protect index freshness
- 27. Test remote range access
- 28. Connect to Primary Science
- 29. Connect to Secondary Science
- 30. Connect to mathematics
- 31. Connect to computing
- 32. Connect to pathways
- 33. Did you know?
- 34. Write a reader-ready compression note
- 35. Build a validation ladder
- 36. Finish with a reproducible bundle
Section 1 of 36
1. Start with the access question
BGZF is valuable when a large data stream must be compressed yet still support selective access. Begin by naming whether the task is sequential reading, uncompressed seeking or genomic regional retrieval.
Working checkpoint — BGZF: Write the file type, query pattern and required integrity guarantees. Keep original and compressed hashes, bgzip and library versions, commands, block validation and companion indexes together.
For a classroom model, seal invented observation cards into numbered packets.
Section 2 of 36
2. Use the current documentation
The current SAM/BAM specification is version 1.6 dated 12 August 2025, and the HTSlib documentation page consulted here is dated 9 July 2026.
Working checkpoint — compressed block: Record the specification, bgzip release and access date. Read every performance gain beside integrity checks, random-access latency, decoded equality and object identity.
For future study, connect compression to mathematics, genomics and computing while checking official pathways separately.
Section 3 of 36
3. Treat BGZF as a blocked stream
BGZF is a sequence of gzip-compatible members rather than one indivisible compressed stream. Each member creates an independently decompressible unit.
Working checkpoint — virtual offset: Inspect more than the filename extension before claiming BGZF. Preserve failed blocks, truncation warnings and rejected offsets; successful early reads do not prove the whole stream.
For scientific writing, report exact bytes, software, options, validation and equality criteria.
Section 4 of 36
4. Read the gzip header first
Every member follows the gzip framing rules and uses the extra-field mechanism. A BGZF-aware parser then checks the special BC subfield.
Working checkpoint — gzip member: Validate magic bytes, compression method, flags and extra-field length. Separate compression compatibility from index compatibility, record validity and biological meaning.
For students, ask how small packets can remain compressed yet independently reachable.
Section 5 of 36
5. Find the BC subfield
The BGZF BC extra subfield carries the total block size minus one. It lets a reader skip directly to the next member.
Working checkpoint — GZI index: Reject missing, duplicated or inconsistent block-size declarations. Treat block boundaries, virtual offsets and object versions as linked scientific inputs: recompression can invalidate every address.
For parents, ask whether fast storage was checked for integrity, privacy and faithful decoding.
Section 6 of 36
6. Respect block size limits
The bgzip manual says blocks are no larger than 64 kilobytes before and after compression, with the compressed limit including headers.
Working checkpoint — regional access: Test boundary-sized and incompressible blocks, not only ordinary examples. Keep original and compressed hashes, bgzip and library versions, commands, block validation and companion indexes together.
For a classroom model, seal invented observation cards into numbered packets.
Section 7 of 36
7. Decompress blocks independently
A reader can inflate one member without decoding the complete preceding stream, provided it begins at a valid block boundary.
Working checkpoint — BGZF: Use an index or validated scan to locate boundaries. Read every performance gain beside integrity checks, random-access latency, decoded equality and object identity.
For future study, connect compression to mathematics, genomics and computing while checking official pathways separately.
Section 8 of 36
8. Understand concatenated gzip behaviour
General gzip tools can usually decompress concatenated BGZF members sequentially, but they do not expose the genomic random-access model.
Working checkpoint — compressed block: Distinguish decompression compatibility from index-aware access. Preserve failed blocks, truncation warnings and rejected offsets; successful early reads do not prove the whole stream.
For scientific writing, report exact bytes, software, options, validation and equality criteria.
Section 9 of 36
9. Read a virtual offset as two numbers
The BAM specification combines a block’s compressed-file offset with an uncompressed offset inside that block. The pair identifies a logical position.
Working checkpoint — virtual offset: Display both components during debugging rather than treating the value as a plain byte count. Separate compression compatibility from index compatibility, record validity and biological meaning.
For students, ask how small packets can remain compressed yet independently reachable.
Section 10 of 36
10. Respect the in-block range
The low part of a virtual offset addresses bytes inside one uncompressed block. Values outside the allowed range signal corruption or a coding mistake.
Working checkpoint — gzip member: Range-check before seeking or comparing offsets. Treat block boundaries, virtual offsets and object versions as linked scientific inputs: recompression can invalidate every address.
For parents, ask whether fast storage was checked for integrity, privacy and faithful decoding.
Section 11 of 36
11. Keep offsets tied to exact bytes
Changing compression level, thread scheduling or block boundaries can change virtual offsets even when decoded records are equivalent.
Working checkpoint — GZI index: Hash the BGZF file associated with every stored index. Keep original and compressed hashes, bgzip and library versions, commands, block validation and companion indexes together.
For a classroom model, seal invented observation cards into numbered packets.
Section 12 of 36
12. Do not subtract offsets casually
Virtual offsets are ordered addresses, not ordinary linear distances across compressed and uncompressed spaces. Arithmetic between them can be meaningless.
Working checkpoint — regional access: Use library operations designed for BGZF positions. Read every performance gain beside integrity checks, random-access latency, decoded equality and object identity.
For future study, connect compression to mathematics, genomics and computing while checking official pathways separately.
Section 13 of 36
13. Understand the GZI index
A .gzi index lists pairs of compressed and corresponding uncompressed offsets for BGZF blocks. It supports seeking in the uncompressed stream.
Working checkpoint — BGZF: Keep the GZI beside the exact compressed file and regenerate it after rewriting. Preserve failed blocks, truncation warnings and rejected offsets; successful early reads do not prove the whole stream.
For scientific writing, report exact bytes, software, options, validation and equality criteria.
Section 14 of 36
14. Separate GZI from genomic indexes
GZI maps byte positions, while BAI, CSI and TBI map genomic coordinates to file chunks. Similar file companions answer different questions.
Working checkpoint — compressed block: Name the index type and coordinate model explicitly. Separate compression compatibility from index compatibility, record validity and biological meaning.
For students, ask how small packets can remain compressed yet independently reachable.
Section 15 of 36
15. Connect coordinate order
Genomic regional indexes normally depend on records being ordered by reference and coordinate. BGZF alone imposes no scientific sort order.
Working checkpoint — virtual offset: Verify sorting separately before building a genomic index. Treat block boundaries, virtual offsets and object versions as linked scientific inputs: recompression can invalidate every address.
For parents, ask whether fast storage was checked for integrity, privacy and faithful decoding.
Section 16 of 36
16. Connect to BAM
BAM commonly uses BGZF so binary alignment records can be compressed and addressed with virtual offsets. BAM semantics come from its own header and record specification.
Working checkpoint — gzip member: Validate BAM structure after validating the BGZF container. Keep original and compressed hashes, bgzip and library versions, commands, block validation and companion indexes together.
For a classroom model, seal invented observation cards into numbered packets.
Section 17 of 36
17. Connect to BCF and compressed VCF
BCF and bgzip-compressed VCF can also use BGZF-based access patterns. Their records and coordinate rules remain format-specific.
Working checkpoint — GZI index: Do not reuse a BAM interpretation merely because the compression layer matches. Read every performance gain beside integrity checks, random-access latency, decoded equality and object identity.
For future study, connect compression to mathematics, genomics and computing while checking official pathways separately.
Section 18 of 36
18. Connect to tabix
Tabix indexes suitable position-sorted textual genomic files compressed with bgzip. The parser preset and column rules determine what each line means.
Working checkpoint — regional access: Record compression command, preset and indexing parameters together. Preserve failed blocks, truncation warnings and rejected offsets; successful early reads do not prove the whole stream.
For scientific writing, report exact bytes, software, options, validation and equality criteria.
Section 19 of 36
19. Use the end-of-file marker
The SAM/BAM specification defines an empty BGZF member commonly used as an end marker and recommends warning when it is absent in seekable files.
Working checkpoint — BGZF: Check for the marker while remembering that it is not a cryptographic proof. Separate compression compatibility from index compatibility, record validity and biological meaning.
For students, ask how small packets can remain compressed yet independently reachable.
Section 20 of 36
20. Check every block CRC
Each gzip member includes integrity fields for its uncompressed payload. A CRC failure signals damaged or wrongly decoded block data.
Working checkpoint — compressed block: Fail closed on checksum errors and preserve the failing offset. Treat block boundaries, virtual offsets and object versions as linked scientific inputs: recompression can invalidate every address.
For parents, ask whether fast storage was checked for integrity, privacy and faithful decoding.
Section 21 of 36
21. Detect truncation beyond the marker
A file can be incomplete even when earlier blocks are valid, and careless concatenation can place data after an expected end marker.
Working checkpoint — virtual offset: Compare object size, final structure and expected record counts. Keep original and compressed hashes, bgzip and library versions, commands, block validation and companion indexes together.
For a classroom model, seal invented observation cards into numbered packets.
Section 22 of 36
22. Validate concatenation deliberately
Concatenating BGZF streams may be mechanically readable, but file-format headers, EOF markers and coordinate order can make the result semantically invalid.
Working checkpoint — gzip member: Use a format-aware merge instead of raw byte concatenation. Read every performance gain beside integrity checks, random-access latency, decoded equality and object identity.
For future study, connect compression to mathematics, genomics and computing while checking official pathways separately.
Section 23 of 36
23. Plan parallel compression
bgzip can use multiple threads, improving throughput while changing resource needs and sometimes block layout.
Working checkpoint — GZI index: Record thread count, compression level and tool version. Preserve failed blocks, truncation warnings and rejected offsets; successful early reads do not prove the whole stream.
For scientific writing, report exact bytes, software, options, validation and equality criteria.
Section 24 of 36
24. Plan parallel decompression
Independent blocks permit concurrent work, but output order and error propagation still need control.
Working checkpoint — regional access: Test determinism and complete failure reporting. Separate compression compatibility from index compatibility, record validity and biological meaning.
For students, ask how small packets can remain compressed yet independently reachable.
Section 25 of 36
25. Benchmark the right workload
A faster full scan does not guarantee faster small-region queries, and maximum compression may cost excessive CPU.
Working checkpoint — BGZF: Measure file size, sequential throughput, random access and total cost separately. Treat block boundaries, virtual offsets and object versions as linked scientific inputs: recompression can invalidate every address.
For parents, ask whether fast storage was checked for integrity, privacy and faithful decoding.
Section 26 of 36
26. Protect index freshness
An index can remain on disk after the BGZF file is replaced or recompressed. Matching names are weak evidence of matching bytes.
Working checkpoint — compressed block: Store hashes, sizes and creation provenance for both file and index. Keep original and compressed hashes, bgzip and library versions, commands, block validation and companion indexes together.
For a classroom model, seal invented observation cards into numbered packets.
Section 27 of 36
27. Test remote range access
Cloud and web retrieval may request byte ranges containing chosen blocks. Server range support, caching and authentication affect correctness and performance.
Working checkpoint — virtual offset: Validate start, end, retries and partial-response handling. Read every performance gain beside integrity checks, random-access latency, decoded equality and object identity.
For future study, connect compression to mathematics, genomics and computing while checking official pathways separately.
Section 28 of 36
28. Connect to Primary Science
Packets of compressed fictional observations can show why labelled containers and a contents map make a large collection easier to retrieve.
Working checkpoint — gzip member: Use invented data and compare sequential versus indexed lookup. Preserve failed blocks, truncation warnings and rejected offsets; successful early reads do not prove the whole stream.
For scientific writing, report exact bytes, software, options, validation and equality criteria.
Section 29 of 36
29. Connect to Secondary Science
BGZF demonstrates that data representation changes access speed without changing the scientific claim encoded in each record.
Working checkpoint — GZI index: Separate container evidence, record evidence and biological interpretation. Separate compression compatibility from index compatibility, record validity and biological meaning.
For students, ask how small packets can remain compressed yet independently reachable.
Section 30 of 36
30. Connect to mathematics
Powers of two, bounded blocks, paired offsets, checksums and interval queries connect the format to discrete mathematics.
Working checkpoint — regional access: Decode a small virtual offset into its two components. Treat block boundaries, virtual offsets and object versions as linked scientific inputs: recompression can invalidate every address.
For parents, ask whether fast storage was checked for integrity, privacy and faithful decoding.
Section 31 of 36
31. Connect to computing
Streaming, binary headers, compression, random access, concurrency and fault detection show how systems engineering supports science.
Working checkpoint — BGZF: Build tests for malformed headers, bad sizes, CRC failures and truncation. Keep original and compressed hashes, bgzip and library versions, commands, block validation and companion indexes together.
For a classroom model, seal invented observation cards into numbered packets.
Section 32 of 36
32. Connect to pathways
Blocked compression connects genomics, storage engineering, cloud computing and data stewardship and can illuminate STEM study and career pathways.
Working checkpoint — compressed block: Check current official qualifications separately. Read every performance gain beside integrity checks, random-access latency, decoded equality and object identity.
For future study, connect compression to mathematics, genomics and computing while checking official pathways separately.
Section 33 of 36
33. Did you know?
A BGZF file is compatible with ordinary gzip for sequential decompression, yet its special block structure lets genomic tools jump to chosen regions.
Working checkpoint — virtual offset: Explain both compatibility and the extra access contract. Preserve failed blocks, truncation warnings and rejected offsets; successful early reads do not prove the whole stream.
For scientific writing, report exact bytes, software, options, validation and equality criteria.
Section 34 of 36
34. Write a reader-ready compression note
Name the source format, bgzip and library versions, options, block or index companions, hashes, validation and retrieval tests.
Working checkpoint — gzip member: Report whether equality was bytewise or semantic. Separate compression compatibility from index compatibility, record validity and biological meaning.
For students, ask how small packets can remain compressed yet independently reachable.
Section 35 of 36
35. Build a validation ladder
Check gzip framing, BC fields, sizes, decompression, CRC, virtual offsets, EOF marker, file-format records and indexed retrieval in that order.
Working checkpoint — GZI index: Stop at the first failed layer and retain the failing fixture. Treat block boundaries, virtual offsets and object versions as linked scientific inputs: recompression can invalidate every address.
For parents, ask whether fast storage was checked for integrity, privacy and faithful decoding.
Section 36 of 36
36. Finish with a reproducible bundle
Deliver original and compressed hashes, command, environment, companion indexes, validation logs, sample retrievals and bounded scientific claims.
Working checkpoint — regional access: Make every retrieved record traceable to exact compressed bytes. Keep original and compressed hashes, bgzip and library versions, commands, block validation and companion indexes together.
For a classroom model, seal invented observation cards into numbered packets.
Contents · Previous section · Continue to the Science Learning Hub