中文版 · Blockchain learning guide · Previous article
Blockchain learning · Article BC056
An indexer reaching a high block number does not prove that it has a complete, correctly interpreted history. A historical answer also depends on the starting boundary, the exact branch, the contract and event filters, decoder versions, retained data and the meaning of the processing checkpoint.
Before accepting an analytics result, ask a narrower question: does this dataset’s documented coverage support this particular query? A service may answer a short interval completely while being unable to substantiate a longer interval or a different kind of historical claim.
This lesson builds a coverage manifest for a fictional catalogue-event dataset. It is a design and evidence exercise, not a live indexing job. No provider credentials, production deployment or chain writes are involved.
Choose your reading route
In this guide
- 1. Define the dataset before advertising its history
- 2. What the coverage manifest records
- 3. A dataset can reach 510 and still stop at 504
- 4. Raw completeness does not guarantee semantic completeness
- 5. Height coverage must follow one defined branch
- 6. Subscriptions help freshness, not complete backfill by themselves
- 7. A checkpoint must mean completed work, not attempted work
- 8. Historical events and historical entity state are different products
- 9. Admit or narrow the query
- 10. Release a corrected generation with its evidence
- 11. Exercises and worked answers
- 12. Frequently asked questions
- Sources and next reading
Sections 1–3 define the dataset contract and its first gap. Sections 4–7 test decoder coverage, branch identity and restart evidence. Sections 8–10 examine retention and query admission. The exercise answers show how to reject an overbroad historical claim without discarding useful partial data.
For the general machinery of change feeds, retries, refresh visibility and rebuilding an index, see Incremental Indexing and Freshness. This guide adds a blockchain-specific question: which identified historical blocks and interpretation rules does a dataset actually cover?
Back to contents · Next section
1. Define the dataset before advertising its history
Our target dataset contains specified catalogue events from contract C on fictional chain S. Its requested interval is height 500 through height 510, inclusive, on the branch ending at identified block B510. B499 is the recognised predecessor anchor.
The packet supplies correct network identity, the linked block identities B499 through B510, and sufficient finality evidence for the target history under the chain’s stated assumptions. This is a pinned historical release, not a request to follow an undefined moving latest state.
The source collection procedure obtains the complete required receipts for each selected block and applies the declared contract and event filter. Correct source responses and complete receipt retrieval are supplied assumptions. Merely receiving some log notifications is not our completeness criterion.
The dataset promises an event-history query for that scope. It does not promise the complete current application state, all transactions on S, all contracts using a similarly named event, or events before height 500.
This boundary resembles the role of network, contract, ABI, start-block and handler declarations in an indexing configuration. The Graph’s manifest documentation makes those choices explicit. Our coverage document is an original audit model, not a ready-to-deploy subgraph configuration. The Graph: Subgraph manifest
Back to contents · Next section
2. What the coverage manifest records
For every required block, the manifest records its exact identity, parent identity, retrieval status, receipt-set completeness, applied filter, decoder rule, derived-output status and durable completion marker. Symbolic B500 is shorthand for a unique identity, not just the number 500.
At the dataset level, record the network, predecessor anchor, target block, inclusive interval, contract scope, event scope, decoder schedule and generation identifier. Keep a list of unresolved gaps rather than hiding them behind a single progress number.
Three progress fields have different meanings:
- Highest observed block: the largest height whose data have been seen.
- Raw coverage: which required blocks have complete retained source material.
- Query-ready coverage: which blocks additionally have correct interpretation and committed derived results for the declared query.
The contiguous query-ready frontier starts at the declared beginning and stops before the first unresolved block. It is not automatically equal to the highest observed height.
The manifest is a service’s evidence record, not a cryptographic proof merely because it is called a manifest. Its entries need support from the actual collection, verification and storage process. In the worksheet, those supporting facts are explicitly supplied or withheld.
Back to contents · Next section
3. A dataset can reach 510 and still stop at 504
Generation G1 has complete, correctly decoded and durably committed coverage for B500–B504 and B506–B510. B505 is missing. All covered blocks are on the pinned target branch; the decoder conditions for this card are correct.
The requested interval contains eleven blocks: 510 minus 500 plus 1. Five covered blocks lie before the gap, and five after it. Ten required blocks are covered, but the contiguous query-ready frontier from 500 is 504.
A dashboard may truthfully say “highest observed 510”. It may not translate that into “complete through 510 from 500”. The latter claim would erase B505.
For a complete event report over 500–504, G1 has the required coverage under the supplied assumptions. For a complete report over 500–510, it does not. It may provide an explicitly partial report identifying the missing block, but the answer’s scope must change with the evidence.
A missing block is not an empty block. If B505 were fetched completely, checked under the correct filter and found to contain no matching events, its completed empty result would close this gap. Without that work, absence of indexed rows supplies no such conclusion.
Back to contents · Next section
4. Raw completeness does not guarantee semantic completeness
Use a separate generation G2 for the next card. All eleven required blocks have complete raw receipt coverage. However, the dataset uses decoder V1 everywhere, while the supplied interpretation schedule requires V1 through B506 and V2 from B507 onward.
For this card, V1 is explicitly unsuitable for the relevant V2-era events. We do not infer that from a version number alone. The authoritative schedule and incompatibility are given facts of the exercise.
G2 can substantiate raw coverage for the entire interval. It cannot substantiate a correctly decoded event report over 500–510 until the B507–B510 interpretation gap is repaired. Four blocks are affected. Its contiguous semantic frontier is 506, assuming all other requirements are satisfied.
Using the same human-readable event name across a change would not fix the problem. The query contract depends on the actual schema and mapping rule. Conversely, a version change does not automatically imply incompatibility; the exercise supplies that incompatibility rather than treating every new number as a breaking change.
The Graph’s documentation separates ABI declarations, event handlers and mapping code, illustrating why selecting records and interpreting them are distinct configuration responsibilities. The Graph: ABI and mapping declarations
For a reliable historical answer, preserve which decoder applied to which identified block interval. A single “current decoder” field is insufficient if historical interpretation varies.
Back to contents · Next section
5. Height coverage must follow one defined branch
Suppose an alternative manifest has a completed record at every height from 500 to 510, but its height 508 entry belongs to a competing block C508 that is not an ancestor of target B510. The row count is eleven, yet the dataset does not cover the requested branch.
The correction is not to add the two histories together. The dataset must distinguish branch identities and establish the ancestry required by its release contract. A row keyed only by height can hide the substitution.
Ethereum RPC records expose block identities and contextual log fields. Those references are useful for preserving the branch context of collected data, although an RPC response still has its own verification assumptions. Ethereum JSON-RPC reference
Our base release pins a target with supplied finality evidence, so the exercise does not model an ordinary later reorganisation of that finalised history. The competing-block card is a collection error relative to the chosen target. If a real service publishes unfinalised data, it needs an explicit policy for later branch changes instead.
A manifest should therefore say both what history it represents and how that history was selected. “All heights present” is a structural check, not a complete ancestry or finality argument.
Back to contents · Next section
6. Subscriptions help freshness, not complete backfill by themselves
Imagine a worker starts listening when the chain is already at height 510. Receiving a current notification does not populate the missing history 500–509. The historical contract still requires an explicit source of past data and evidence that the required interval was covered.
Geth’s subscription documentation states that notifications concern current events rather than past events and that subscriptions are tied to their connection. It also describes how new-head notifications behave during catch-up and reorganisations. These are reasons to distinguish a live notification channel from a historical completeness record. Geth real-time events
For the worksheet, notifications can tell the collector that there may be new work. They cannot themselves certify that every earlier block’s required receipts were collected, filtered and interpreted.
After a disconnect, a trustworthy historical pipeline resumes from its last supported coverage point and reconciles the identified interval. The exact implementation can vary, but its release claim must remain tied to durable evidence rather than the most recent notification received.
Do not convert this into a claim that subscriptions are useless. They solve a useful notification problem. The error is asking that mechanism to prove a different property that its contract does not supply.
Back to contents · Next section
7. A checkpoint must mean completed work, not attempted work
The next card concerns one block’s processing boundary. Before B505 is processed, the durable query-ready frontier is B504. The worker starts reading B505 and crashes before its complete derived result and completion marker have been committed.
Under the toy ingestion contract, partial staged work is not query-ready. After restart, the published frontier remains B504. The worker must reconcile or reprocess B505 under the same block identity, filter and decoder before advertising it as complete.
The contract requires the block’s completed derived output and its durable completion marker to become visible consistently. This is a stated design obligation, not a claim that merely writing a checkpoint file achieves atomicity.
Suppose instead a progress counter advanced to 505 before the required outputs were committed. The counter cannot support the coverage claim. Its meaning would be “attempted up to 505”, not “the declared query is complete through 505”. The publication layer must not silently substitute one meaning for the other.
This is why the dataset record stores a generation and a branch-aware processing reference. A bare integer cannot establish which interpretation rules or source history produced the rows that a query will see.
Back to contents · Next section
8. Historical events and historical entity state are different products
An immutable event-history table and a table representing an application’s state at a past block are not the same dataset. The first may retain event records; the second may require versioned entity states or a supported reconstruction procedure.
Our release promises filtered event history. It does not promise that every application change emits one of those events. Therefore, even complete coverage of the declared events is not automatically enough to reconstruct all application state at B510.
Similarly, keeping today’s derived entity state does not prove that older entity versions remain queryable. The Graph’s documentation explains that pruning historical entity data affects time-travel queries and related historical capabilities. That is a separate retention choice from whether the indexer once processed a block. The Graph: Historical data retention
A service may still be able to rebuild an older view if it retains or can recover the necessary raw data and interpretation rules. “Can be rebuilt under these conditions” is different from “can answer this query now”. A response should not conceal the distinction.
No universal storage duration is supplied in this lesson. The useful requirement is that the retention policy match the historical query being promised, with evidence for the data and transformations that promise needs.
Back to contents · Next section
9. Admit or narrow the query
Consider four requests against the supplied manifests.
Request A: all declared events in 500–504 on the B510 branch. G1 can provide a complete scoped answer because this interval is entirely query-ready.
Request B: all declared events in 500–510 on that branch. G1 cannot provide a complete answer because B505 is unresolved. It can reject the completeness claim or explicitly return a partial answer with the gap identified.
Request C: correctly interpreted events in 500–510 using the supplied historical decoder schedule. G2 cannot provide this answer while V1 is incorrectly used for 507–510, even though raw receipt coverage is complete.
Request D: the complete application state at B510. Neither manifest’s event-history promise alone establishes that capability. The query asks for a different product, and the exercise supplies no guarantee that the selected events capture every state change.
This admission step is the lesson’s main output. An index is useful when it can honestly state what questions it supports, not when every request returns a plausible-looking table.
Back to contents · Next section
10. Release a corrected generation with its evidence
A repaired generation G3 has complete correct raw data for all eleven blocks, the proper decoder schedule, ancestry aligned to B510, durable completion markers and no unresolved processing gaps. Its release record identifies exactly that generation and scope.
Only after these conditions are established may the service publish the complete event-history claim for 500–510. If an older generation remains visible during repair, label its actual coverage rather than implying that the new generation’s work has already reached users.
The release check should verify that the query endpoint serves the same generation the manifest describes. Correct files sitting in a staging area do not prove that public answers use them. Conversely, changing a public alias without matching the coverage record can produce an ambiguous answer provenance.
General rebuilding and cutover techniques belong to the indexing foundation linked above. The blockchain-specific obligation here is to preserve the exact chain, block boundary, branch and decoder context through that release.
Even G3 does not prove that the catalogue’s off-chain statements are true. Its assurance concerns the declared records and their interpretation. Dataset completeness is valuable, but it is not a substitute for the evidential standard of every question asked about the world.
Back to contents · Next section
11. Exercises and worked answers
Question 1. How many blocks are required for 500–510 inclusive? How many does G1 cover?
Answer. Eleven are required. G1 covers ten: five before missing 505 and five after it.
Question 2. What is G1’s contiguous query-ready frontier from 500?
Answer. B504. Highest observed 510 does not close the gap at 505.
Question 3. All raw blocks are present, but V1 is wrongly applied from 507 through 510. How many blocks have the supplied semantic problem?
Answer. Four. Raw completeness and semantic completeness are separate.
Question 4. Can a completed zero-match result for B505 close the raw-and-query coverage gap?
Answer. Yes, if complete source collection, the correct filter and decoder, ancestry and durable completion are established. Merely having no rows cannot do so.
Question 5. A crash occurs before B505’s complete outputs and completion marker commit. May the published frontier advance to 505?
Answer. No. Under the contract it remains B504 until the required work is durably established.
Question 6. Does a complete event dataset automatically answer the complete application state at B510?
Answer. No. The exercise supplies no guarantee that these events capture every state change or a complete reconstruction procedure.
Back to contents · Next section
12. Frequently asked questions
Is the highest indexed block a sufficient completeness measure?
No. Check the declared beginning, gaps, branch identity, decoder coverage and query-ready state.
Can I treat a missing interval as having no events?
No. An unscanned interval and a completely checked empty interval have different evidence.
Does a live subscription replace historical collection?
Not by itself. Its notification contract must not be mistaken for complete past coverage.
Is a coverage manifest itself proof that every row is correct?
No. It is a structured claim supported by collection and processing evidence. The underlying verification and source assumptions must remain visible.
Back to contents · Next section
Sources and next reading
Primary references checked on 12 October 2026:
- Geth real-time events, last edited 16 August 2023; current-notification and connection limitations only.
- Ethereum JSON-RPC reference, for block and log context.
- The Graph subgraph manifest, for declared sources, ABI/mapping choices and historical entity retention.
The generations, block intervals, decoder activation and query-admission decisions are original teaching constructions, not a claim about any live indexer’s completeness.