THE GOLD STANDARD OF INFORMATION · SPECIALIST STUDY
Treat a cached SPARQL plan as a versioned decision with dependencies, not a timeless shortcut
Preparing a query can save parsing and planning work, but a reused plan can outlive the data distribution, statistics, indexes, inference rules, dataset mapping or engine build that made it sensible. Parameters can also make one logical query selective for one value and enormous for another. A defensible plan cache therefore records what was compiled, which evidence shaped it, when that evidence changed, how reuse was bounded and how an uncached comparison can be reproduced without turning every slow query into an emergency cache flush.
This guide belongs to the Information subject root, the Gold Standard of Education, and the How X Works hub. It begins with a concrete decision, then tests whether the method still holds when the context changes.
Choose a reading route
Define the cached object · chapters 1, 2, 3
Track dependencies and epochs · chapters 4, 5
Test parameter sensitivity · chapters 6, 7, 8
Bypass and compare safely · chapters 9, 10, 11
Govern, rehearse and transfer · chapters 12, 13
The whole problem at a glance
| Cached object | Dependency | Invalidation evidence |
|---|---|---|
| Parsed query | Syntax and base IRI | Canonical text fingerprint |
| Optimised algebra | Statistics and optimiser settings | Statistics epoch and strategy |
| Physical plan | Indexes, extensions and engine build | Plan fingerprint and capability set |
| Prepared handle | Connection, dataset and access context | Scope and terminal state |
Invalidate the smallest object whose assumptions are no longer true.
CHAPTER 1 OF 13 · Define the cached object
1. Name what “prepared” means in the deployed stack
A client API may merely parse a query, a repository may optimise it, and a server may cache an executable plan. Do not infer server-side reuse from a client object name.
Evidence record: Record client library, repository, endpoint, connection lifetime, canonical query text, parameter binding point and whether parse, algebra or physical plan is reused. Preserve versions, timestamps, denominators, excluded cases, failures and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Accept a cache claim only when the reuse boundary and observable plan identity are documented. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Move from an embedded RDF store to a remote SPARQL service; the same API call can cross a different preparation boundary. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 1 check · what should survive an audit
- State the decision unit for “Name what “prepared” means in the deployed stack”.
- Bind every claim to an observed version, time boundary and comparison set.
- Connect the evidence to an owner, bounded action and later verification.
CHAPTER 2 OF 13 · Define the cached object
2. Fingerprint query shape without erasing semantics
Normalisation helps group equivalent requests, but careless replacement of literals, graph names, SERVICE targets, language tags or datatypes can merge queries whose plans and meanings differ.
Evidence record: Keep original text hash, parsed algebra hash, normalisation rules, dataset clauses, entailment mode, security context and parameter schema. Preserve versions, timestamps, denominators, excluded cases, failures and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Reject a cache key that aliases requests with different result semantics, access scope or optimisation dependencies. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Add a parameter inside a property path or SERVICE clause; decide whether the old shape key remains safe. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 2 check · what should survive an audit
- State the decision unit for “Fingerprint query shape without erasing semantics”.
- Bind every claim to an observed version, time boundary and comparison set.
- Connect the evidence to an owner, bounded action and later verification.
CHAPTER 3 OF 13 · Define the cached object
3. Declare the reuse scope and lifetime
Prepared handles may be valid only for a connection or transaction, while server caches may span users and deployments. Scope determines both correctness and leakage risk.
Evidence record: Capture tenant, role, dataset, transaction snapshot, connection, node, deployment, creation time, last use and eviction reason. Preserve versions, timestamps, denominators, excluded cases, failures and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Never reuse across a boundary the implementation does not guarantee, especially identity, dataset or entailment boundaries. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Fail over to another node with different extensions; capability equivalence must be proved before reuse. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 3 check · what should survive an audit
- State the decision unit for “Declare the reuse scope and lifetime”.
- Bind every claim to an observed version, time boundary and comparison set.
- Connect the evidence to an owner, bounded action and later verification.
CHAPTER 4 OF 13 · Track dependencies and epochs
4. Version statistics as an explicit epoch
Optimisers use cardinality estimates to choose join order and algorithms. Jena TDB, for example, can use a statistics file whose approximate counts guide reordering. A plan compiled under one distribution needs a traceable statistics generation.
Evidence record: Store statistics file or catalogue identifier, generation time, dataset generation, sampling method, rule order, count summaries and optimiser strategy. Preserve versions, timestamps, denominators, excluded cases, failures and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Advance the statistics epoch only through an atomic publication step; plans dependent on the old epoch become stale by policy. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Bulk-load one predicate that was previously rare; relative counts may change enough to reverse a join order. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 4 check · what should survive an audit
- State the decision unit for “Version statistics as an explicit epoch”.
- Bind every claim to an observed version, time boundary and comparison set.
- Connect the evidence to an owner, bounded action and later verification.
CHAPTER 5 OF 13 · Track dependencies and epochs
5. Map every plan dependency before invalidating
Plans can depend on indexes, schema mappings, inference rules, custom functions, endpoint capabilities, configuration flags and engine code. A generic time-to-live cannot express those relationships.
Evidence record: Maintain dependency type, version, affected operators, compatibility rule, invalidation event and proof that each cache node received it. Preserve versions, timestamps, denominators, excluded cases, failures and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Invalidate on a declared dependency change or quarantine plans whose dependency lineage cannot be reconstructed. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Change a custom function from deterministic to context-sensitive; cached constant folding is no longer valid. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 5 check · what should survive an audit
- State the decision unit for “Map every plan dependency before invalidating”.
- Bind every claim to an observed version, time boundary and comparison set.
- Connect the evidence to an owner, bounded action and later verification.
Worked parameter-sensitive comparison
| Parameter | Estimated rows | Observed rows | Plan | Elapsed |
|---|---|---|---|---|
| Rare identifier | 1 | 1 | Index lookup | 18 ms |
| Common class | 120 | 1,240,000 | Same cached lookup chain | 19.4 s |
| Common class, fresh plan | 1,180,000 | 1,240,000 | Scan then hash join | 2.2 s |
The numbers illustrate the method; they are not a product benchmark.
CHAPTER 6 OF 13 · Test parameter sensitivity
6. Separate correctness invalidation from performance refresh
Most stale plans remain semantically correct but slow; some changes, such as dataset mapping or function semantics, can change meaning. The response and urgency should differ.
Evidence record: Classify dependency changes by result correctness, access control, resource risk and expected performance impact, with owner and maximum stale window. Preserve versions, timestamps, denominators, excluded cases, failures and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Fail closed for semantic or access uncertainty; use bounded refresh for performance-only staleness. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: An index is rebuilt without logical change, then an inference configuration changes; the two events need different gates. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 6 check · what should survive an audit
- State the decision unit for “Separate correctness invalidation from performance refresh”.
- Bind every claim to an observed version, time boundary and comparison set.
- Connect the evidence to an owner, bounded action and later verification.
CHAPTER 7 OF 13 · Test parameter sensitivity
7. Detect parameter-sensitive plans from distributions
A value identifying one subject and a value denoting a common class can share syntax while producing radically different cardinalities. Median latency can hide the harmful tail.
Evidence record: Retain parameter class, privacy-safe frequency band, estimated and observed rows per operator, elapsed time, spills, memory and result size. Preserve versions, timestamps, denominators, excluded cases, failures and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Split plan families or re-optimise when stable parameter bands show materially different safe plans. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: A formerly rare value becomes common after a data import; band membership and plan choice must be refreshed. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 7 check · what should survive an audit
- State the decision unit for “Detect parameter-sensitive plans from distributions”.
- Bind every claim to an observed version, time boundary and comparison set.
- Connect the evidence to an owner, bounded action and later verification.
CHAPTER 8 OF 13 · Test parameter sensitivity
8. Use estimates and observations without circular tuning
A slow execution can teach the optimiser, but tuning only to yesterday’s troublesome literals overfits. Compare estimated and observed cardinalities across representative parameter strata.
Evidence record: Keep workload sampling frame, stratum definitions, estimate error, operator timings, censored timeouts and excluded executions. Preserve versions, timestamps, denominators, excluded cases, failures and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Promote a new rule only after it improves a held-out workload without violating correctness or resource floors. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Shift from interactive lookups to nightly analytical queries; the earlier workload sample no longer represents use. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 8 check · what should survive an audit
- State the decision unit for “Use estimates and observations without circular tuning”.
- Bind every claim to an observed version, time boundary and comparison set.
- Connect the evidence to an owner, bounded action and later verification.
CHAPTER 9 OF 13 · Bypass and compare safely
9. Expose an explain path with known side effects
RDF4J explain levels distinguish parsed, optimised and fully executed evidence; higher levels may evaluate the query and can be expensive. Explain tooling needs the same access and resource controls as execution.
Evidence record: Record explain level, whether results were fully iterated, timeout, dataset snapshot, plan text, estimates, actual rows and telemetry collection overhead. Preserve versions, timestamps, denominators, excluded cases, failures and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Never run an executing explain in production merely to inspect syntax; choose the least invasive evidence level that answers the question. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: A federated query triggers remote calls during explanation; include downstream cost and permissions. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 9 check · what should survive an audit
- State the decision unit for “Expose an explain path with known side effects”.
- Bind every claim to an observed version, time boundary and comparison set.
- Connect the evidence to an owner, bounded action and later verification.
Cache-bypass gate
| Question | Pass evidence | Stop condition |
|---|---|---|
| Can results be compared? | Same dataset snapshot and entailment | Generation changed |
| Can plans be identified? | Normalised algebra and plan fingerprints | Opaque plan identity |
| Is bypass bounded? | One cohort, expiry and capacity budget | Uncontrolled global flush |
| Can regression be attributed? | Statistics, build and index lineage | Multiple unrecorded changes |
Preserve both the cached and fresh executions under comparable conditions.
CHAPTER 10 OF 13 · Bypass and compare safely
10. Design a bounded cache-bypass experiment
Bypass should compare a cached plan with a freshly optimised plan under the same snapshot, parameters and limits. A global flush changes load for everyone and contaminates the comparison.
Evidence record: Assign experiment ID, query cohort, cached plan, fresh plan, snapshot, warm-up policy, concurrency, resource caps, expiry and rollback. Preserve versions, timestamps, denominators, excluded cases, failures and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Permit bypass only for a scoped cohort and automatically end it when the evidence window closes. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Run on a hot replica and then a cold recovery node; separate cache warmth from plan quality. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 10 check · what should survive an audit
- State the decision unit for “Design a bounded cache-bypass experiment”.
- Bind every claim to an observed version, time boundary and comparison set.
- Connect the evidence to an owner, bounded action and later verification.
CHAPTER 11 OF 13 · Bypass and compare safely
11. Attribute regressions through controlled contrasts
Latency changed after a release does not prove the plan cache caused it. Dataset growth, index state, JVM warm-up, remote endpoints and concurrency can move together.
Evidence record: Preserve before-and-after build, plan, statistics, snapshot, index health, endpoint versions, workload mix, concurrency and hardware envelope. Preserve versions, timestamps, denominators, excluded cases, failures and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Call a plan regression only when the comparison controls material alternatives or reports them as unresolved. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: A new plan is faster alone but worse under contention; include service-level and resource evidence. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 11 check · what should survive an audit
- State the decision unit for “Attribute regressions through controlled contrasts”.
- Bind every claim to an observed version, time boundary and comparison set.
- Connect the evidence to an owner, bounded action and later verification.
CHAPTER 12 OF 13 · Govern, rehearse and transfer
12. Rehearse invalidation loss and mixed epochs
In distributed services, one node may miss an invalidation or continue serving an old plan while another uses new statistics. Drills should expose split-brain plan behaviour.
Evidence record: Inject a missed event, delayed statistics publication, node restart and rollback; retain cache inventories, epoch gauges, request routing and convergence time. Preserve versions, timestamps, denominators, excluded cases, failures and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Pass only when stale nodes are detected, isolated or converged within the declared window without semantic ambiguity. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Repeat during a rolling engine upgrade where two plan formats coexist. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 12 check · what should survive an audit
- State the decision unit for “Rehearse invalidation loss and mixed epochs”.
- Bind every claim to an observed version, time boundary and comparison set.
- Connect the evidence to an owner, bounded action and later verification.
CHAPTER 13 OF 13 · Govern, rehearse and transfer
13. Publish a plan-cache evidence contract
Operators and consumers need stable fields for query shape, plan, statistics epoch, dataset generation, cache outcome, parameter band and terminal result. Proprietary plan internals can remain private.
Evidence record: Version the telemetry schema, redaction rules, retention, sampling, known opaque operators and investigation procedure. Preserve versions, timestamps, denominators, excluded cases, failures and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Treat missing plan lineage as an observability defect; do not invent certainty from latency alone. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Transfer the contract to SQL prepared statements while replacing SPARQL-specific algebra and entailment fields. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 13 check · what should survive an audit
- State the decision unit for “Publish a plan-cache evidence contract”.
- Bind every claim to an observed version, time boundary and comparison set.
- Connect the evidence to an owner, bounded action and later verification.
Contents · Previous chapter · Return to the Information guide
Verification lab: work the decision twice
Run the fictional rare and common parameters through one cached plan. Decide which evidence justifies a second plan family rather than a global flush.
Advance a statistics epoch after a bulk load while one node misses the event. Design detection, quarantine, convergence and rollback evidence.
Compare cached and fresh plans on one snapshot, then repeat after the snapshot changes. Explain why the second comparison cannot isolate cache effects.
Limits and misconceptions
Prepared does not necessarily mean server-cached
The preparation boundary is implementation- and API-specific.
A stale plan is not always a wrong result
Classify semantic risk separately from performance degradation.
Median latency can hide parameter harm
Inspect representative parameter bands and tail resources.
A global flush is weak evidence
It changes load and destroys the controlled comparison.
Questions readers ask
When should a SPARQL plan be invalidated?
- When a declared dependency changes beyond its compatibility rule or its statistics epoch expires.
Are optimiser statistics required to be exact?
- No. They guide choices, but their version, representativeness and estimate error still matter.
What is a parameter-sensitive plan?
- A plan whose safety or efficiency varies materially across parameter values or value classes.
How can cache bypass be safe?
- Scope it to a cohort, preserve comparable snapshots, cap resources and enforce expiry.
Does EXPLAIN execute the query?
- It depends on product and level; some RDF4J explanation levels fully evaluate results.
What proves a plan regression?
- A controlled contrast with plan, statistics, data, build, workload and resource lineage.
Practise, verify and continue
Design a plan-cache record for one parameterised SPARQL query. Include query and plan fingerprints, dataset and statistics epochs, parameter band, cache outcome, invalidation causes and a scoped bypass experiment.
Move the design to a graph analytics service with compiled traversals. Preserve dependency lineage, parameter strata, bounded bypass and controlled regression evidence while replacing SPARQL-specific operators.
Practical evidence check
Before implementation, write a one-page evidence map with the decision, unit of analysis, version or time boundary, inputs, outcomes, uncertainty, authority and stop rule. Attach one worked case that passes and one that fails. A reviewer should be able to reproduce the reasoning without relying on the memory of the person who built the system.
Run one changed-context rehearsal before approval. Alter a mechanism rather than a cosmetic detail, then identify which calculation, safeguard or governance gate must change. Record any conclusion that does not transfer. This short rehearsal is often the fastest way to expose a hidden assumption before it becomes an operational dependency.
Related eduKate reading
SPARQL index rebuilds and query-plan regression
SPARQL query cancellation and retry safety
SPARQL rolling upgrades and mixed-version nodes
SPARQL result-set comparison and regression testing
Authoritative sources and scope
W3C SPARQL 1.2 Query Language
Apache Jena TDB optimiser and statistics documentation
Apache Jena TDB command documentation
Eclipse RDF4J Repository API and query explanation
The standards and product documentation establish query semantics, statistics-guided optimisation and explain capabilities. They do not prescribe a universal prepared-plan cache. The epochs, parameter bands, bypass gate and fictional timings are an original operational framework and must be adapted to the deployed repository and client versions.