THE GOLD STANDARD OF INTELLIGENCE · SPECIALIST STUDY
Change an RDS survey language without quietly changing the construct, network path or time trend
A translated question can be grammatically accurate yet alter recall, stigma, category boundaries or the meaning of network size. In respondent-driven sampling, language choice can also follow recruitment chains, sites and interviewer availability, so a mid-wave version change may be entangled with population composition and estimator weights. A defensible amendment versions every wording, tests conceptual equivalence, records exposure, protects recruitment continuity and estimates how conclusions change under plausible language effects.
This guide belongs to the Intelligence subject root, the Gold Standard of Education, and the How X Works hub. It begins with a concrete decision, then tests whether the method still holds when the context changes.
Choose a reading route
Define language and construct versions · chapters 1, 2, 3
Translate and test meaning · chapters 4, 5
Control the mid-wave amendment · chapters 6, 7, 8
Estimate comparability and bias · chapters 9, 10, 11
Report, govern and transfer · chapters 12, 13
The whole problem at a glance
| Layer | Question | Evidence |
|---|---|---|
| Concept | Is the same construct intended? | Definition and item intent |
| Language | Is meaning equivalent in context? | Forward translation, reconciliation and cognitive testing |
| Fieldwork | Who received each version? | Version receipt, interviewer, site and time |
| Analysis | Would conclusions change? | DIF, bridge sample and sensitivity analysis |
Word agreement alone cannot establish construct or estimator comparability.
CHAPTER 1 OF 13 · Define language and construct versions
1. Define the construct before translating words
Write what each item is meant to measure, its recall period, response process and decision use. Translation begins from that intent, not from matching individual words.
Evidence record: Keep source wording, item intent, definitions, prohibited reinterpretations, response options and the populations for whom the item was designed. Preserve timestamps, versions, failures, excluded cases and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Do not approve a translation whose intended construct cannot be explained independently of the source sentence. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Move an item from behaviour frequency to perceived risk; even fluent wording cannot preserve a construct that changed. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 1 check · what should survive an audit
- State the decision unit for “Define the construct before translating words”.
- Bind the claim to versions, denominators, missing cases and time boundaries.
- Connect evidence to an owner, bounded action and later verification.
CHAPTER 2 OF 13 · Define language and construct versions
2. Treat every language form as a versioned instrument
Assign immutable identifiers to language, locale, script, interviewer mode, response options and revision. “Chinese” or “English” alone is too coarse when wording and dialect differ.
Evidence record: Record version ID on consent, survey, coupon explanation, network-size questions, payment scripts and analytic exports. Preserve timestamps, versions, failures, excluded cases and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Exclude or reconcile records with unknown instrument versions before claiming cross-version comparability. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Add an audio-assisted form; mode becomes part of the version even when visible text is unchanged. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 2 check · what should survive an audit
- State the decision unit for “Treat every language form as a versioned instrument”.
- Bind the claim to versions, denominators, missing cases and time boundaries.
- Connect evidence to an owner, bounded action and later verification.
CHAPTER 3 OF 13 · Define language and construct versions
3. Map language choice onto the recruitment network
RDS recruitment chains may cluster by language, site or community. Language exposure is therefore not necessarily independent of network position, seed, degree or outcome.
Evidence record: Retain recruiter and recruit version, preferred language, site, interviewer, time, seed, degree and switch reason without exposing identities. Preserve timestamps, versions, failures, excluded cases and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Inspect whether version clusters within recruitment trees before attributing outcome gaps to translation alone. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: A bilingual participant recruits into another language group; allow cross-language edges rather than forcing one chain label. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 3 check · what should survive an audit
- State the decision unit for “Map language choice onto the recruitment network”.
- Bind the claim to versions, denominators, missing cases and time boundaries.
- Connect evidence to an owner, bounded action and later verification.
CHAPTER 4 OF 13 · Translate and test meaning
4. Use a controlled translation and reconciliation process
Independent forward translations, reconciliation, review and back-translation can reveal alternatives, but the goal is conceptual equivalence in the target context. Harmonise recurring terms across instruments.
Evidence record: Preserve translator roles, decisions, alternatives, problematic concepts, reconciled wording and reasons for rejecting literal matches. Preserve timestamps, versions, failures, excluded cases and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Advance wording only after disagreements are resolved against item intent and target-user comprehension. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Translate for a new country sharing the same language; repeat cultural review rather than reusing vocabulary by nationality label. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 4 check · what should survive an audit
- State the decision unit for “Use a controlled translation and reconciliation process”.
- Bind the claim to versions, denominators, missing cases and time boundaries.
- Connect evidence to an owner, bounded action and later verification.
CHAPTER 5 OF 13 · Translate and test meaning
5. Test comprehension with target participants
Cognitive interviewing asks how people understand, recall, decide and map answers to response categories. It can expose stigma, ambiguous kinship, unsafe disclosure or network terms that translation review misses.
Evidence record: Record sampling of testers, probes, paraphrases, hesitation, category use, privacy concerns and each resulting revision. Preserve timestamps, versions, failures, excluded cases and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Retest material changes; do not treat one successful bilingual reviewer as population validation. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Move from interviewer administration to self-completion; test navigation and privacy as part of comprehension. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 5 check · what should survive an audit
- State the decision unit for “Test comprehension with target participants”.
- Bind the claim to versions, denominators, missing cases and time boundaries.
- Connect evidence to an owner, bounded action and later verification.
Worked bridge sample
| Group | Version A yes | Version B yes | Observed gap |
|---|---|---|---|
| Bilingual bridge participants | 42% | 51% | 9 percentage points |
| Earlier wave, A only | 39% | — | Not a direct bridge |
| Later wave, B only | — | 55% | Confounded with time and recruitment |
Randomised or counterbalanced exposure needs ethical and practical review; these numbers are illustrative.
CHAPTER 6 OF 13 · Control the mid-wave amendment
6. Amend mid-wave only through a declared change boundary
A necessary correction can improve ethics or validity, but the effective time, sites, participants and instruments must be explicit. Do not silently overwrite the earlier wording in the data dictionary.
Evidence record: Keep amendment approval, trigger, effective timestamp, affected items, old and new versions, retraining receipts and already-enrolled participant treatment. Preserve timestamps, versions, failures, excluded cases and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Deploy only when every field route can identify the new version and unknown mixed use is bounded. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: One offline device syncs late; use administration-time version evidence rather than server upload time. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 6 check · what should survive an audit
- State the decision unit for “Amend mid-wave only through a declared change boundary”.
- Bind the claim to versions, denominators, missing cases and time boundaries.
- Connect evidence to an owner, bounded action and later verification.
CHAPTER 7 OF 13 · Control the mid-wave amendment
7. Preserve consent and participant understanding
Language amendments can affect consent, withdrawal, payment and referral explanations as well as outcome items. Participant-facing obligations must remain understandable and consistent.
Evidence record: Retain consent language version, interpreter use, comprehension check, reconsent rule, participant notice and available remedy. Preserve timestamps, versions, failures, excluded cases and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Reconsent or notify when the amendment changes material participation information under the approved protocol. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: A participant changes preferred language at follow-up; preserve both receipts and avoid treating the switch as misconduct. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 7 check · what should survive an audit
- State the decision unit for “Preserve consent and participant understanding”.
- Bind the claim to versions, denominators, missing cases and time boundaries.
- Connect evidence to an owner, bounded action and later verification.
CHAPTER 8 OF 13 · Control the mid-wave amendment
8. Build a bridge instead of relying on before-and-after averages
A bridge sample exposes some participants or comparable groups to both versions under controlled order or parallel allocation. Design must consider learning, fatigue, disclosure risk and interview burden.
Evidence record: Keep allocation, order, washout if relevant, bilingual proficiency, interviewer, response pair, missingness and reasons a bridge was not feasible. Preserve timestamps, versions, failures, excluded cases and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Use bridge evidence to estimate plausible version effects, not to erase genuine time or composition change. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: When dual administration is unsafe, use qualitative linking plus bounded sensitivity rather than pretending equivalence was measured. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 8 check · what should survive an audit
- State the decision unit for “Build a bridge instead of relying on before-and-after averages”.
- Bind the claim to versions, denominators, missing cases and time boundaries.
- Connect evidence to an owner, bounded action and later verification.
CHAPTER 9 OF 13 · Estimate comparability and bias
9. Test differential item functioning carefully
Differential item functioning asks whether people with comparable underlying construct levels respond differently by language version. Model choice, sample size and anchor items matter, especially in hidden populations.
Evidence record: Record latent model, anchors, grouping rule, thresholds, uncertainty, multiplicity handling and whether weights or network dependence were incorporated. Preserve timestamps, versions, failures, excluded cases and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Flag items for review when version differences are material and robust; do not label cultural difference as bias without a construct argument. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Small language groups yield unstable estimates; report bounds and combine quantitative with cognitive evidence. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 9 check · what should survive an audit
- State the decision unit for “Test differential item functioning carefully”.
- Bind the claim to versions, denominators, missing cases and time boundaries.
- Connect evidence to an owner, bounded action and later verification.
Amendment decision gates
| Gate | Pass condition | Failure response |
|---|---|---|
| Concept | Same intended construct | Rewrite or split estimands |
| Comprehension | Target users interpret as intended | Revise and retest |
| Operations | Every interview logs exact version | Quarantine unknown receipts |
| Analysis | Differences bounded or modelled | Stratify, bridge or limit trend claims |
Approval should state which comparisons remain supportable.
CHAPTER 10 OF 13 · Estimate comparability and bias
10. Separate language effects from fieldwork and composition
A version introduced later may coincide with new seeds, sites, interviewers, incentives or epidemic conditions. Before-and-after comparison blends these mechanisms.
Evidence record: Model calendar time, site, seed, interviewer, recruitment wave, degree, covariates and version; show overlap and positivity limits. Preserve timestamps, versions, failures, excluded cases and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Avoid causal language when version and time have little overlap; restrict the claim to observed association and sensitivity. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Roll out the new form by site; site and language become inseparable without a bridge or staggered design. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 10 check · what should survive an audit
- State the decision unit for “Separate language effects from fieldwork and composition”.
- Bind the claim to versions, denominators, missing cases and time boundaries.
- Connect evidence to an owner, bounded action and later verification.
CHAPTER 11 OF 13 · Estimate comparability and bias
11. Recompute RDS estimands under version scenarios
Outcome classification changes can alter prevalence, subgroup estimates and possibly recruitment-adjusted comparisons. Network-size translation can also change weights directly.
Evidence record: Produce estimates using observed version, harmonised coding, version-stratified outcomes and plausible reclassification or degree-error scenarios. Preserve timestamps, versions, failures, excluded cases and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Publish conclusions only if the decision survives the declared language-effect range or explicitly narrow the conclusion. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: A translation changes only degree bins; outcome counts stay fixed while weights and uncertainty move. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 11 check · what should survive an audit
- State the decision unit for “Recompute RDS estimands under version scenarios”.
- Bind the claim to versions, denominators, missing cases and time boundaries.
- Connect evidence to an owner, bounded action and later verification.
CHAPTER 12 OF 13 · Report, govern and transfer
12. Keep dashboards from splicing incomparable trends
A trend line that crosses a wording change needs a visible version marker, bridge estimate and uncertainty. Automatic dashboards often hide these boundaries.
Evidence record: Version metric definitions, annotate release dates, retain old series, show bridged and unbridged views and prevent silent backfill. Preserve timestamps, versions, failures, excluded cases and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Break or qualify the trend when comparability cannot be defended; do not smooth across an unknown measurement discontinuity. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: A later harmonisation revision changes the bridge; issue a new series version rather than overwrite history. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 12 check · what should survive an audit
- State the decision unit for “Keep dashboards from splicing incomparable trends”.
- Bind the claim to versions, denominators, missing cases and time boundaries.
- Connect evidence to an owner, bounded action and later verification.
CHAPTER 13 OF 13 · Report, govern and transfer
13. Publish a bounded comparability statement
State which languages, versions, sites, dates, items and estimands were tested; what evidence supports equivalence; where DIF or uncertainty remains; and which comparisons should not be made.
Evidence record: Release the version map, amendment history, testing summary, analytic decisions, sensitivity ranges and governance approvals without exposing participants. Preserve timestamps, versions, failures, excluded cases and rejected alternatives so another reviewer can reconstruct the decision rather than inherit a conclusion.
Decision rule: Require future reuse to cite the exact instrument and series version, not a generic study name. Name the authorised owner, trigger, stop condition, remedy and verification window. A control is incomplete until a changed-context case tests the mechanism it is meant to protect.
Evidence discipline: record the observation, version or time boundary that supports this chapter’s conclusion. Preserve the rejected alternative as well as the selected action. That makes a later review able to distinguish a changed world from an undocumented change of mind.
Changed-context transfer: Transfer the method to a non-RDS longitudinal survey; network-specific diagnostics may drop, but version, bridge and DIF duties remain. The method transfers only if the decision question and measurement meaning survive the change. If either changes, rebuild the comparison rather than copying the earlier threshold or weight.
Chapter 13 check · what should survive an audit
- State the decision unit for “Publish a bounded comparability statement”.
- Bind the claim to versions, denominators, missing cases and time boundaries.
- Connect evidence to an owner, bounded action and later verification.
Contents · Previous chapter · Return to the Intelligence guide
Verification lab: work the decision twice
Translate a fictional network-size item whose word for “know” can mean recognise or have recent contact. Compare literal, construct-based and cognitively tested versions.
Use the fictional 42% versus 51% bridge result and a later 55% prevalence. Show why 55−39 does not estimate a pure time trend.
Create an RDS sensitivity table in which the translated degree item shifts high-degree reports. Recompute one weighted estimate and describe the limit.
Limits and misconceptions
Back-translation is not proof of equivalence
It can reveal discrepancies but does not replace target-user comprehension or construct evidence.
Bilingual participants are not always representative
Their education, migration and language use may differ from monolingual groups.
DIF is not automatically bias
Interpretation requires a construct and context argument.
A visible trend break can be more honest than a smooth line
Unknown comparability should not be hidden by imputation.
Questions readers ask
Must every translation use identical words?
- No. The aim is equivalent construct and response meaning in context.
What makes a mid-wave amendment defensible?
- A version boundary, approval, deployment receipts, testing, bridge evidence and qualified analysis.
Can two language versions be pooled?
- Only when conceptual and empirical evidence supports the intended estimand or a model explicitly accounts for differences.
Why test network-size wording?
- Degree reports often enter RDS weights, so a language effect can change both measurement and estimation.
What if a bridge sample is impossible?
- Use cognitive evidence, external anchors and bounded sensitivity, then limit comparability claims.
Should dashboards back-cast the new wording?
- Only through a documented bridge model with retained original series and uncertainty.
Practise, verify and continue
Write a version-control and bridge plan for a two-language RDS study introducing a revised network-size item halfway through fieldwork. Include consent, devices, interviewers, recruitment trees, estimands and dashboards.
Move the framework to a multilingual school survey. Remove coupon and recruitment-tree controls, but retain construct definition, version receipts, cognitive testing, bridge design, DIF analysis and trend qualification.
Practical evidence check
Before implementation, write a one-page evidence map with the decision, unit of analysis, version or time boundary, inputs, outcomes, uncertainty, authority and stop rule. Attach one worked case that passes and one that fails. A reviewer should be able to reproduce the reasoning without relying on the memory of the person who built the system.
Run one changed-context rehearsal before approval. Alter a mechanism rather than a cosmetic detail, then identify which calculation, safeguard or governance gate must change. Record any conclusion that does not transfer. This short rehearsal is often the fastest way to expose a hidden assumption before it becomes an operational dependency.
Related eduKate reading
RDS degree-question design and construct validity
RDS multi-site harmonisation and pooled estimands
RDS adaptive fieldwork and mid-course validity
RDS participant withdrawal and estimator sensitivity
Authoritative sources and scope
HealthMeasures PROMIS measure development and translation methodology
HealthMeasures requirements for new translations
HealthMeasures translation use and administration guidance
PROMIS instrument development and validation scientific standards
The sources provide instrument-development and translation principles. Their branded measures are examples, not RDS-specific rules. The bridge sample and calculations are original teaching cases; actual protocols require ethics review, community input and jurisdiction-specific consent decisions.