Short answer: a participant who has not completed when observation stops supplies a lower bound on completion time, not an exact completion at the time limit.
This guide develops a focused reader question within the User Experience subject guide. It adds an explicit measurement chain, original worked examples, changed-context checks and limits. Read it with The Gold Standard of Education | Learning That Transfers into Life and the How X Works.
Every dataset and calculation below is an invented teaching case. The article does not certify a food, laboratory result, product, network or service. Use the current method, responsible specialist and applicable authority for a real decision.
eduKateSG · User Experience
Find your next learning step
Preserve outcome, assistance, stop reason and observation horizon before comparing task-time summaries.
ROUTE 1 · CHAPTERS 1–6
Define the timed outcome
Separate task boundaries, outcomes, censoring and completer-only summaries.
ROUTE 2 · CHAPTERS 7–11
Build an honest timeline
Use completion curves while preserving assistance, stops and interruptions.
ROUTE 3 · CHAPTERS 12–17
Change the context
Test unequal limits, accessibility, alternate routes, coarse time and uncertainty.
ROUTE 4 · CHAPTERS 18–22
Communicate and continue
Avoid blame and causal overreach, repair misconceptions and follow sources.
Chapter contents
Define the timed outcome · Chapters 1–6
1. Start with the short answer
2. Define the task and completion criterion
3. Record outcomes before summarising time
Build an honest timeline · Chapters 7–11
7. Use a completion-over-time curve
8. Do not borrow survival terminology carelessly
9. Distinguish administrative limits from abandonment
Change the context · Chapters 12–17
12. Changed context: different time limits
13. Changed context: accessibility and pace
14. Changed context: voluntary alternate route
15. Use intervals when timing is coarse
Communicate and continue · Chapters 18–22
18. Communicate without blaming participants
19. Avoid causal claims from descriptive timing
20. Repair common misconceptions
When a usability session ends before some participants finish, their task times are not ordinary completed times. A participant observed for 120 seconds without completion has a known lower bound, not a 120-second completion. Preserve completion state, assistance and stopping reason before comparing speed.
Begin with a measurement ledger. Name the quantity, its unit, the sample or observation that produced it, the time window and the transformation rule. Mark each input as observed, stipulated, estimated or assumed. This stops a familiar label from smuggling in a new denominator or a stronger claim. A reader should be able to replace one input and see exactly which conclusion changes. Read the ledger as a sentence before calculating; if the sentence is incomplete, the arithmetic is premature.
In an invented test, completion times are 42, 55, 78 and 96 seconds; two participants reach the 120-second limit without finishing. The median of the four completed times is 66.5 seconds, but that is a completer-only summary, not the median for all six attempts.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to change the limit to 90 seconds and identify which records and claims change. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
CHAPTER 2 OF 22 · Define the timed outcome
2. Define the task and completion criterion
Timing becomes meaningful only after the intended goal, start event, successful end event and permitted paths are declared. A click, saved transaction and confirmed outcome are different boundaries.
Keep the mechanism beside the number. Ask what physical, biological, statistical or behavioural relationship connects the observation to the conclusion, and which conditions could break that link. A correct calculation can answer the wrong question when a method selects only part of a sample or a changed setting alters the process. Draw one arrow for every inferential step, then challenge each arrow with a counterexample. State why the relationship is plausible and where it stops.
A task begins when the account page loads and completes when a confirmation number appears. Stopping at the final button click shortens every recorded time and can misclassify an error response as success.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to move the start boundary from page load to instruction end and state why results cannot be mixed silently. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
CHAPTER 3 OF 22 · Define the timed outcome
3. Record outcomes before summarising time
Use separate fields for unassisted completion, assisted completion, non-completion, abandonment, technical interruption and observation limit. Do not infer outcome from duration.
Use an independent check that follows a different route. Reconstruct totals from components, compare a bound with a point estimate, inspect a distribution behind an average, or solve a changed case with the original definitions. Agreement does not prove every assumption, but disagreement locates a definition, unit or boundary that needs repair. Write the expected check before seeing the answer so the check does not become a second performance of the same mistake.
Participant A completes in 110 seconds; B abandons after 35; C is stopped at 120; D completes after a hint at 80. Sorting only the seconds makes B look fastest and erases assistance.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to rebuild the table with status and assistance chronology beside every time. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
A right-censored record says the unobserved completion time, if completion would eventually occur under the same conditions, exceeds the observed duration. It does not say the eventual time equals the limit or that completion was inevitable.
Match the claim to the evidence. A tidy decimal cannot repair selective observation, and one convenient run cannot establish every product, participant or network. Report the result, the conditions that make it interpretable, and the next observation needed for the actual decision. Separate the numerical conclusion from the practical conclusion. The first says what follows from the supplied model; the second names the authority, replication, context or threshold still required before action.
A participant is still working when the 120-second session window ends. The record is greater than 120 seconds under this observation, not completed in 120 seconds.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to change the stopping reason to voluntary abandonment and explain why the interpretation may differ. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
CHAPTER 5 OF 22 · Define the timed outcome
5. Worked example: completer-only median
A completer-only statistic can describe the successful subgroup when labelled. It can mislead when designs have different completion rates because each design contributes a different selected set.
Begin with a measurement ledger. Name the quantity, its unit, the sample or observation that produced it, the time window and the transformation rule. Mark each input as observed, stipulated, estimated or assumed. This stops a familiar label from smuggling in a new denominator or a stronger claim. A reader should be able to replace one input and see exactly which conclusion changes. Read the ledger as a sentence before calculating; if the sentence is incomplete, the arithmetic is premature.
Design A has completed times 40, 50, 60, 70 and five limits at 120; its completer median is 55. Design B has eight completions from 70 to 105 and two limits; its completer median is 87.5. Calling A more usable from 55 versus 87.5 ignores selection.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to compare completion rates and the full status table before interpreting speed. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
CHAPTER 6 OF 22 · Define the timed outcome
6. Worked example: replacing limits creates bias
Substituting the time limit for every non-completion invents exact completion times and generally understates how long completion would have taken. Excluding them also selects easier successes.
Keep the mechanism beside the number. Ask what physical, biological, statistical or behavioural relationship connects the observation to the conclusion, and which conditions could break that link. A correct calculation can answer the wrong question when a method selects only part of a sample or a changed setting alters the process. Draw one arrow for every inferential step, then challenge each arrow with a counterexample. State why the relationship is plausible and where it stops.
For times 42, 55, 78, 96 and two censored at 120, replacement produces 42, 55, 78, 96, 120, 120 with median 87. Exclusion gives 66.5. Neither number is the unknown all-attempt completion median without assumptions.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to report transparent descriptive bounds and the censoring pattern instead of choosing a convenient answer. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
A curve can show the proportion completing by each observed time while retaining participants still under observation. A nonparametric product-limit estimate is one possible statistical tool when its assumptions and event definitions are appropriate.
Use an independent check that follows a different route. Reconstruct totals from components, compare a bound with a point estimate, inspect a distribution behind an average, or solve a changed case with the original definitions. Agreement does not prove every assumption, but disagreement locates a definition, unit or boundary that needs repair. Write the expected check before seeing the answer so the check does not become a second performance of the same mistake.
Six participants begin. Completions occur at 42, 55, 78 and 96 seconds; two remain incomplete at 120. The empirical observed completion fraction reaches 4/6 by 96 and stays there through the limit.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to add one participant who leaves for an unrelated reason at 70 seconds and update the risk set carefully. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
CHAPTER 8 OF 22 · Build an honest timeline
8. Do not borrow survival terminology carelessly
Reliability and survival methods provide mathematics for censored event times, but a user completing a task is not a machine failing. Translate only the data structure, not the meaning or causal assumptions.
Match the claim to the evidence. A tidy decimal cannot repair selective observation, and one convenient run cannot establish every product, participant or network. Report the result, the conditions that make it interpretable, and the next observation needed for the actual decision. Separate the numerical conclusion from the practical conclusion. The first says what follows from the supplied model; the second names the authority, replication, context or threshold still required before action.
The event is declared task completion. The curve estimates completion timing under the study definition; it does not measure a person worth, persistence or general ability.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to rewrite labels so the chart remains humane and task-specific. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
CHAPTER 9 OF 22 · Build an honest timeline
9. Distinguish administrative limits from abandonment
A predeclared session limit is imposed by the study. A participant choice to stop may reflect difficulty, irrelevance, distress, misunderstanding or a rational alternative. These can require different outcome categories and analyses.
Begin with a measurement ledger. Name the quantity, its unit, the sample or observation that produced it, the time window and the transformation rule. Mark each input as observed, stipulated, estimated or assumed. This stops a familiar label from smuggling in a new denominator or a stronger claim. A reader should be able to replace one input and see exactly which conclusion changes. Read the ledger as a sentence before calculating; if the sentence is incomplete, the arithmetic is premature.
Two records both end at 90 seconds. One is stopped by the facilitator schedule; the other ends when the participant says the task is no longer worth doing. Equal durations do not make the observations equivalent.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to record stopping reason without pressuring the participant to continue. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
CHAPTER 10 OF 22 · Build an honest timeline
10. Keep facilitator assistance on the timeline
A hint changes the task conditions. Timing before and after assistance can be useful, but assisted completion should not silently enter the unaided distribution.
Keep the mechanism beside the number. Ask what physical, biological, statistical or behavioural relationship connects the observation to the conclusion, and which conditions could break that link. A correct calculation can answer the wrong question when a method selects only part of a sample or a changed setting alters the process. Draw one arrow for every inferential step, then challenge each arrow with a counterexample. State why the relationship is plausible and where it stops.
A participant works unaided for 70 seconds, receives a task-revealing hint and completes 25 seconds later. Report 70 seconds to assistance and 95 seconds total assisted completion, not a 95-second unaided success.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to replace the hint with a planned accessibility accommodation and apply the protocol intended category. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
CHAPTER 11 OF 22 · Build an honest timeline
11. Handle technical interruptions explicitly
A server outage, recorder failure or accidental notification can invalidate or pause a timing record. A technical stop is not user failure and should not be converted into a censored difficulty record without justification.
Use an independent check that follows a different route. Reconstruct totals from components, compare a bound with a point estimate, inspect a distribution behind an average, or solve a changed case with the original definitions. Agreement does not prove every assumption, but disagreement locates a definition, unit or boundary that needs repair. Write the expected check before seeing the answer so the check does not become a second performance of the same mistake.
At 64 seconds the service returns an error for every participant. The affected attempts are marked technical interruptions. Their times do not estimate how long the usable design takes.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to repeat after recovery and keep the repeated-session learning effect visible. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
Comparisons with unequal limits expose different portions of the time distribution. A design tested for 60 seconds cannot be compared directly with one observed for 180 seconds using raw completion rates.
Match the claim to the evidence. A tidy decimal cannot repair selective observation, and one convenient run cannot establish every product, participant or network. Report the result, the conditions that make it interpretable, and the next observation needed for the actual decision. Separate the numerical conclusion from the practical conclusion. The first says what follows from the supplied model; the second names the authority, replication, context or threshold still required before action.
Design C has 12 of 20 completions by 60 seconds. Design D has 17 of 20 by 180. At the common 60-second point D has only 10 completions. The ranking depends on the declared question.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to choose a common horizon and show both curves rather than mixing final percentages. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
A longer completion time can be a successful, accessible path. Segment analysis should be planned and respectful; it must not turn disability or assistive-technology use into a hidden penalty.
Begin with a measurement ledger. Name the quantity, its unit, the sample or observation that produced it, the time window and the transformation rule. Mark each input as observed, stipulated, estimated or assumed. This stops a familiar label from smuggling in a new denominator or a stronger claim. A reader should be able to replace one input and see exactly which conclusion changes. Read the ledger as a sentence before calculating; if the sentence is incomplete, the arithmetic is premature.
A participant using a screen reader completes accurately in 150 seconds under an allowed 180-second window. Excluding the result from a 120-second benchmark would change the represented context of use.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to state whether the goal is universal completion, a service target or a comparative diagnostic. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
CHAPTER 14 OF 22 · Change the context
14. Changed context: voluntary alternate route
A participant may meet the underlying goal through a supported alternative rather than the tested interface path. Decide whether that is task completion, a service-level success or an out-of-scope route before the study.
Keep the mechanism beside the number. Ask what physical, biological, statistical or behavioural relationship connects the observation to the conclusion, and which conditions could break that link. A correct calculation can answer the wrong question when a method selects only part of a sample or a changed setting alters the process. Draw one arrow for every inferential step, then challenge each arrow with a counterexample. State why the relationship is plausible and where it stops.
A user begins an online form, switches to a staffed callback and resolves the request. The interface task is incomplete, while the service goal may be complete. Both outcomes can be true.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to report interface and service outcomes separately. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
If observers record to the nearest ten seconds or only at checkpoints, exact-looking times are false precision. Preserve intervals or rounding rules.
Use an independent check that follows a different route. Reconstruct totals from components, compare a bound with a point estimate, inspect a distribution behind an average, or solve a changed case with the original definitions. Agreement does not prove every assumption, but disagreement locates a definition, unit or boundary that needs repair. Write the expected check before seeing the answer so the check does not become a second performance of the same mistake.
A completion observed between the 70- and 80-second checks is reported as occurring in that interval. Entering 75 seconds can be a midpoint convention, not a direct observation.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to compare two designs using the same observation resolution. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
CHAPTER 16 OF 22 · Change the context
16. Quantify uncertainty without hiding the records
Small usability studies are often diagnostic. Confidence intervals or bootstrap summaries can orient uncertainty, but they do not repair unrepresentative participants or inconsistent facilitation.
Match the claim to the evidence. A tidy decimal cannot repair selective observation, and one convenient run cannot establish every product, participant or network. Report the result, the conditions that make it interpretable, and the next observation needed for the actual decision. Separate the numerical conclusion from the practical conclusion. The first says what follows from the supplied model; the second names the authority, replication, context or threshold still required before action.
Eight participants yield five completions before 120 seconds. The observed 62.5% is a session result, not a precise population constant. Show the count, denominator and context beside any interval.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to change one outcome and observe how unstable the percentage is. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
Predeclare task boundary, success categories, assistance rules, stopping policy, maximum time, technical-exclusion rules and planned summaries. Precommitment reduces opportunistic switching after seeing results.
Begin with a measurement ledger. Name the quantity, its unit, the sample or observation that produced it, the time window and the transformation rule. Mark each input as observed, stipulated, estimated or assumed. This stops a familiar label from smuggling in a new denominator or a stronger claim. A reader should be able to replace one input and see exactly which conclusion changes. Read the ledger as a sentence before calculating; if the sentence is incomplete, the arithmetic is premature.
A plan states: report unassisted completion by 90 seconds, assisted completion separately, and a completion-over-time display through 180 seconds. It prevents a later choice of whichever horizon favours a design.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to write a plan for a second task whose natural duration is much longer. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
CHAPTER 18 OF 22 · Communicate and continue
18. Communicate without blaming participants
Use language such as the participant did not complete under these test conditions rather than the user failed. Record the interface obstacle and context that the design team can act on.
Keep the mechanism beside the number. Ask what physical, biological, statistical or behavioural relationship connects the observation to the conclusion, and which conditions could break that link. A correct calculation can answer the wrong question when a method selects only part of a sample or a changed setting alters the process. Draw one arrow for every inferential step, then challenge each arrow with a counterexample. State why the relationship is plausible and where it stops.
Instead of three slow users failed, write three participants had not completed by the 120-second limit; all encountered the address-format step. The sentence preserves evidence and responsibility.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to remove an unsupported explanation while keeping the observed pattern. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
CHAPTER 19 OF 22 · Communicate and continue
19. Avoid causal claims from descriptive timing
A faster or higher completion curve in one session does not prove the design caused the difference. Participant mix, order, learning, facilitation and context can contribute.
Use an independent check that follows a different route. Reconstruct totals from components, compare a bound with a point estimate, inspect a distribution behind an average, or solve a changed case with the original definitions. Agreement does not prove every assumption, but disagreement locates a definition, unit or boundary that needs repair. Write the expected check before seeing the answer so the check does not become a second performance of the same mistake.
Group E uses design 1 first; group F uses design 2 after practice. A timing difference is confounded with order. The existing counterbalancing branch explains why balanced order does not erase unequal learning transfer.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to design a follow-up that addresses the most important confound without promising perfect control. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
A timeout is not a completion at the limit; excluding non-completions is not neutral; a completer median does not describe every attempt; and a longer successful path is not automatically worse.
Match the claim to the evidence. A tidy decimal cannot repair selective observation, and one convenient run cannot establish every product, participant or network. Report the result, the conditions that make it interpretable, and the next observation needed for the actual decision. Separate the numerical conclusion from the practical conclusion. The first says what follows from the supplied model; the second names the authority, replication, context or threshold still required before action.
Diagnose: Design A median is 55 seconds, so it beats B at 87.5. The missing fact is that A completed for half the participants and B for most of them.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to construct a case where the faster completer median accompanies a worse completion curve. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
The facilitator-hint guide preserves assistance chronology. Why Counterbalancing Does Not Erase Task Learning preserves order effects. This branch handles incomplete event times and stopping rules.
Begin with a measurement ledger. Name the quantity, its unit, the sample or observation that produced it, the time window and the transformation rule. Mark each input as observed, stipulated, estimated or assumed. This stops a familiar label from smuggling in a new denominator or a stronger claim. A reader should be able to replace one input and see exactly which conclusion changes. Read the ledger as a sentence before calculating; if the sentence is incomplete, the arithmetic is premature.
Together the routes protect outcome, assistance, order and time. None alone proves causation or universal usability.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to select the branch that matches a hint, repeated-design order or timeout problem. State what would change the result and what the example cannot establish. That limit belongs inside the answer.
NIST usability-testing overview treats time on task, errors and completion as distinct evidence. Its practical CIF guide connects efficiency to task time and effectiveness to completion. The NIST handbook explains right censoring and Kaplan–Meier estimation. Sources checked on 26 October 2026.
Keep the mechanism beside the number. Ask what physical, biological, statistical or behavioural relationship connects the observation to the conclusion, and which conditions could break that link. A correct calculation can answer the wrong question when a method selects only part of a sample or a changed setting alters the process. Draw one arrow for every inferential step, then challenge each arrow with a counterexample. State why the relationship is plausible and where it stops.
The statistical tools require appropriate assumptions and competent analysis. The examples are diagnostic teaching cases, not product certification or research findings.
Before moving on, record the observation that would distinguish the two most plausible explanations. If no feasible observation exists, narrow the claim instead of pretending the ambiguity has disappeared.
A changed-context transfer check is to audit a new usability table by naming task, boundaries, outcome, help, stop reason, observation horizon and defensible summary. State what would change the result and what the example cannot establish. That limit belongs inside the answer.