How The World Works | Latency

An answer can be correct and still arrive too late to matter.

The ambulance reaches the right address after the critical window. The server returns the right result after the user has abandoned the page. A company finally notices demand has changed after warehouses are full. A student receives excellent feedback three weeks after the reasoning that produced the mistake has faded. A government measure works, but only after the condition it was designed for has already changed.

Nothing in those examples is necessarily wrong about the information itself.

The problem is time.

The technical word for one important form of this problem is latency.

Latency is delay between an initiating event and a response. In computing it may be the time from a request to a reply. In a control system it may be the delay between a change in the world, measurement, decision and corrective action. In an organisation it may be the time between a problem appearing and someone with authority seeing it. In learning it may be the interval between an attempt and useful feedback.

Once you learn to see latency, a great many “quality” problems turn out to be timing problems in disguise.

Quick Read

Latency is the elapsed time between an input, event, request, observation or decision and the corresponding response or effect.

Low latency can make systems responsive. High latency can make information stale, feedback late, queues grow, users abandon tasks and controllers overcorrect.

But lower latency is not always better. Deliberation, verification, cooling-off periods, batching and safety reviews can add intentional delay for good reasons.

The useful question is therefore:

How much delay can this system tolerate before the meaning or value of the response changes?

That question links latency to capacity, feedback, thresholds, risk, scheduling, monitoring and decision-making.

The One-Sentence Answer

Latency works by separating cause from response in time, allowing the world to continue changing during the gap, so the eventual response may act on a state that no longer exists, encounter a queue that has grown, miss a deadline or trigger further behaviour before correction arrives.

Delay Is Not Empty Time

A common mental model treats delay as if the world freezes while a system waits.

It does not.

During the delay, customers place more orders. Traffic continues arriving. Prices change. Students forget. Machines heat. Competitors respond. Rumours spread. Queues grow. Inventory moves. Weather changes. A medical condition can progress. New evidence can arrive.

Latency is therefore not simply “extra minutes.”

It is a period during which the system and its environment may diverge from the state on which the response was based.

The Latency Chain

event → detection delay → transmission delay → queue → processing delay → decision delay → action delay → effect delay → observation of result

A system can be “slow” for very different reasons depending on which segment dominates.

The sensor noticed late.
The message travelled slowly.
The request waited behind other work.
Processing itself was expensive.
A human decision took time.
Implementation required coordination.
The intervention had a delayed physical effect.
The system measured the outcome only after another long interval.

Calling all of this “latency” is useful at the top level. Repair requires decomposing the chain.

Latency Is Not the Same as Throughput

Throughput asks how much work a system completes per unit time.

Latency asks how long one unit of work waits for completion.

A system can have high throughput and poor latency. A warehouse may process enormous daily volume while one urgent parcel waits too long. A computer cluster may maximise total jobs completed while interactive requests experience unacceptable delay.

Google’s Site Reliability Engineering guidance explicitly distinguishes workloads that care about low latency from workloads that care mainly about throughput. The queue state that is desirable for one can be undesirable for the other.

This is a recurring world lesson: the metric of success depends on the receiver and the job.

Latency Is Not the Same as Capacity

Capacity is how much load a system can handle under specified conditions.

Latency often rises as capacity becomes saturated, but the concepts are distinct.

A system with enormous capacity can still contain a long fixed delay. A low-capacity system can sometimes provide fast responses under light load.

As utilisation approaches the system’s processing limit, queues often become the bridge between capacity and latency.

See How Capacity Works.

The Queue Is Where Time Accumulates

Imagine a café with one barista.

If customers arrive slowly, the barista begins each order almost immediately.

If customers arrive at roughly the same rate the barista can serve them, small random bursts begin creating a queue.

If arrivals exceed sustainable service capacity, the queue grows.

The coffee itself may still take exactly three minutes to make. Yet the customer waits twenty minutes because seventeen minutes were spent before service began.

This is why measuring only processing time can produce an absurdly optimistic picture of user experience.

Little’s Law: Work in the System, Arrival Rate and Time

Queueing theory gives us one of the most useful equations in operations:

L = λW

In steady-state form, Little’s Law relates the average number of items in a system, L, to the average arrival rate, λ, and the average time each item spends in the system, W.

MIT queueing notes use the same relationship for vehicles and waiting systems.

The equation is simple enough to become dangerous if misused. Its conditions matter, and it does not explain the full distribution of waiting times. But it gives an immediate systems intuition:

If arrival rate stays high and time in system rises, more work accumulates in the system.

Backlog and latency are linked.

A Backlog Is Stored Latency

A queue is work waiting for time.

That makes backlog one of the clearest visible forms of accumulated latency.

Emails await replies. Court cases await hearings. maintenance tasks await technicians. Student work awaits feedback. Software tickets await engineers. Ships await berths. Patients await appointments.

Backlog can be rational. Batching can improve efficiency. Priority systems can protect urgent work.

But a growing backlog is a warning that arrival and service dynamics have become misaligned.

If the system cannot reduce incoming demand, increase effective capacity or shorten service time, waiting becomes structural.

Average Latency Can Lie

Suppose 95 users receive a response in 50 milliseconds and five users wait one second.

The average can look respectable while a meaningful minority experiences a very different service.

Google SRE guidance recommends examining latency distributions and percentiles rather than relying only on averages because tail latency can hide behind a good mean.

This principle extends beyond computers.

An average processing time of three days can conceal a small group waiting thirty. An average commute can conceal extreme peak-period delays. Average feedback time can conceal one class whose work is consistently returned late.

The tail often contains the people for whom the system is failing.

Tail Latency Becomes More Dangerous in Multi-Step Systems

Imagine a request that depends on twenty services.

If nineteen respond quickly but one is unusually slow, the final user may still wait for the slow component.

As systems become more distributed, the probability of encountering a slow component somewhere along the route can increase.

This is why modern systems care about high-percentile latency, timeouts, redundancy and graceful degradation.

The same pattern appears in human organisations. A project with twenty approvals does not feel like the average approval. It feels like the slowest required one.

Latency Changes the Value of Information

Information has a timestamp even when the timestamp is not printed.

A stock count from last week may be useless for dispatch today. A weather observation from six hours ago may mislead a current decision. A child’s reading diagnosis from two years ago may describe a learner who no longer exists in that state.

Latency therefore creates staleness.

The faster the underlying world changes, the shorter the useful life of old information.

This gives us a simple relationship:

The value of low latency rises when the state of the world changes quickly.

Latency and Feedback

Feedback is useful only if it returns in time to influence the next relevant action.

A thermostat measures temperature and adjusts heating. If sensing and actuation are heavily delayed, the system can overshoot. It heats based on old information. By the time the correction arrives, the room has already moved past the target. The controller then compensates too late in the opposite direction.

Delayed feedback can create oscillation.

Human systems do the same thing. Organisations cut inventory after observing a downturn, but the data arrives late. Orders then recover. The cut was based on yesterday’s world, so shortages follow. The organisation reacts again.

See How Feedback Works and How Control Systems Work.

The Bullwhip Intuition

Supply chains illustrate how delay and local reaction can amplify variability.

A retailer sees demand rise and orders more from a distributor. The distributor sees orders rise and responds. Manufacturers adjust production. But information and goods take time to move. Each participant may add buffers because the future is uncertain.

Small changes in customer demand can become larger swings upstream.

Latency is not the only cause of bullwhip effects, but delay is one of the mechanisms that makes coordination harder because participants act on incomplete and lagged signals.

This is emergence with memory: local decisions, delayed information and feedback produce a macro-pattern nobody intended.

Latency and Emergence

Latency is often one of the hidden parameters behind emergent behaviour.

Drivers react to the car ahead with finite reaction time. Traders respond to information that arrives at different speeds. People imitate behaviours after observing others. Software retries after timeouts. Organisations update plans after reporting cycles.

Change the delay and the collective pattern can change.

That is why the companion article How The World Works | Emergence sits next to this one.

Latency and Monitoring

A monitor does not observe “now.”

It observes a measurement made at some time, transmitted through some path, processed with some delay and displayed later.

When delays are tiny relative to system change, this approximation is harmless.

When delays are large, the dashboard can become a picture of the past pretending to be the present.

Good monitoring therefore records timestamps, collection intervals, processing delays and freshness. It distinguishes event time from arrival time.

See How Monitoring Works.

Latency and Decision-Making

A decision has a validity window.

Some decisions remain useful for years. Others decay in seconds.

Choosing a university programme may tolerate weeks of research. Stopping a machine after a dangerous sensor reading may tolerate milliseconds. Responding to a student misconception may be useful tomorrow but less useful after several weeks of reinforcing the wrong method.

The faster the decision environment changes and the less reversible the consequence, the more carefully latency should be engineered.

See How Decision-Making Works.

Fast Can Be Wrong

Latency reduction is seductive because speed is visible.

A faster decision looks better than a slower one.

But verification takes time. Evidence collection takes time. Independent review takes time. Some physical processes require time. Some social decisions benefit from reflection.

Optimising latency without regard to error can create fast failure.

The real design target is often timely correctness, not minimum delay.

Slow Can Also Be Wrong

The opposite mistake is using caution as an excuse for delay.

If a process repeatedly misses the window in which action can help, its accuracy after the fact may have little operational value.

A perfectly reviewed safety recommendation issued after the hazard has passed is not the same service as a sufficiently reliable warning issued in time.

Systems therefore need explicit latency requirements tied to receiver deadlines.

Deadlines Turn Latency Into a Threshold Problem

Suppose an answer is worth almost the same whether it arrives in one second or two.

But after ten seconds, the user leaves.

The value function contains a threshold.

Many real systems are like this. A plane connection departs. A financial market moves. A crop window closes. A student submits the examination. A machine crosses a temperature limit. A legal deadline expires.

Latency should therefore be measured against the time-to-use, not against an abstract desire to be fast.

See How Thresholds Work.

Retries Can Make Delay Worse

A user sends a request.

The system is overloaded and responds slowly.

The user or client assumes failure and retries.

Now the overloaded system has even more work.

Google SRE documentation warns that retries can amplify low error rates into higher traffic and contribute to cascading failure. It recommends backoff, jitter and overload protection rather than unlimited immediate retry.

This is an important general pattern:

When delay causes actors to generate more demand, latency can feed itself.

Queueing Can Create a Phase Change

Below capacity, additional work may be absorbed with modest delay.

Near saturation, small demand increases can create much larger waiting times.

Above sustainable capacity, backlog grows continuously unless work is rejected, shed or capacity changes.

This nonlinear behaviour is why overload can feel sudden.

The system was “fine” at 80%. It was “mostly fine” at 90%. Then delay explodes near the boundary.

The lesson is not to worship a particular utilisation number. Different systems behave differently. The lesson is to understand the curve rather than extrapolate linearly toward saturation.

Graceful Degradation Is a Latency Strategy

When a system cannot complete every request at full quality in time, it may be better to return a cheaper, partial result than to let every request wait until useless.

Google SRE describes graceful degradation and load shedding as ways to preserve useful service under overload.

This creates a trade-off:

perfect but late versus imperfect but usable now

The correct answer depends on the receiver.

A search result can sometimes degrade gracefully. A dosage calculation cannot casually trade correctness for speed. System design must know which dimensions are negotiable.

Latency Budgets

A useful design technique is to treat the total acceptable delay as a budget.

If the receiver needs a result within one second, the full path must share that second.

Detection gets some time.
Network transmission gets some.
Queueing gets some.
Processing gets some.
Verification gets some.
Delivery gets some.

If one stage consumes the entire budget, upstream speed elsewhere cannot save the result.

This is the temporal equivalent of an end-to-end capacity budget.

Latency Hides in Handoffs

Work often moves quickly inside departments and slowly between them.

A team completes its task in an hour, then waits three days for the next team to notice the handoff. The second team completes work in two hours, then waits for a weekly meeting.

Each team reports excellent processing time.

The receiver waits a week.

This is why interfaces and queues matter. Organisational latency often lives in the white space between boxes on the chart.

See How Interfaces Work.

Batching Trades Latency for Efficiency

Why do organisations process some work in batches?

Because setup is costly.

It may be more efficient to process fifty similar items together than handle each immediately. Trucks wait for loads. Payroll runs on a schedule. Reports are compiled monthly. Manufacturing changes tools between batches.

Batching can raise throughput and reduce unit cost while increasing latency for the first item waiting in the batch.

Neither mode is universally better.

The correct design depends on setup cost, urgency, perishability, capacity and receiver deadlines.

Latency and Markets

Markets depend on information arriving at different speeds.

A seller learns inventory conditions. A buyer learns prices. A lender receives financial information. A regulator receives reports. A supply chain receives demand signals.

When information is delayed, participants act on stale states. When some participants receive information faster than others, latency becomes a form of informational advantage.

This creates a bridge to information asymmetry. Difference in knowledge is not only about what people know. It can also be about when they know it.

Latency and Information Asymmetry

Suppose two parties eventually receive the same information, but one receives it tomorrow and the other receives it next month.

At the final date, their information sets may look equal.

During the month, they were not.

That timing difference can affect bargaining, prices, risk and trust.

Information architecture therefore has at least three dimensions:

accuracy;
access;
freshness.

The next article in this batch develops the unequal-knowledge problem directly.

Latency and Externalities

Delayed consequences can make externalities harder to see.

A cost appears years after the action that created it. The original decision-maker may be gone. The receiver may be elsewhere. The causal chain may be disputed.

Long latency between action and harm weakens intuitive attribution even when the underlying mechanism remains real.

This is one reason environmental and maintenance problems are easy to defer: the benefit of delay is immediate while much of the cost arrives later.

See How The World Works | Externalities.

Latency and Defaults

A default can become more powerful when correction is slow.

If a mistaken default can be changed instantly, the cost of being wrong may be small. If correction requires weeks, paperwork or a scheduled review, the same default has much larger consequences.

This is why reversibility and latency belong together.

See How The World Works | Defaults.

Latency and Path Dependence

Slow correction gives a path time to build structure around itself.

A poor standard is noticed, but migration planning takes years. During those years more systems adopt it. More data accumulates. More people train. More complements appear.

The correction delay has allowed switching costs to grow.

Latency can therefore convert a temporary mistake into a durable inheritance.

See How The World Works | Path Dependence.

Latency and Friction

Delay is one form of friction.

But not all friction is latency. A process can be immediate but cognitively difficult. A switch can take seconds but cost money. A form can be fast to submit and hard to understand.

Separating the two concepts helps diagnosis.

See How The World Works | Friction.

Human Latency: Attention Has a Queue Too

A manager with forty unread messages is a queueing system.

So is a teacher with two hundred scripts. So is a parent handling work, children, appointments and household tasks. So is a student with six deadlines.

Human attention has finite service capacity.

When incoming demands exceed that capacity, tasks wait, priorities change and low-salience items may never receive service.

This is one reason “I sent the information” is not the same as “the information entered the recipient’s decision process.”

The Meeting as a Latency Device

A weekly meeting can create an average delay of several days for issues that are only discussed there.

This may be acceptable because batching decisions reduces coordination cost.

But urgent issues need an exception path.

Good organisational design therefore separates routine cadence from escalation.

Everything should not interrupt everyone immediately. Everything should not wait until Friday either.

The system needs multiple latency classes.

Priority Queues: Not All Waiting Is Equal

Hospitals, networks, maintenance teams and operating systems all face versions of the same problem: which item should be served next?

First-in-first-out is simple and often fair.

It is not always appropriate.

An urgent safety issue may need to jump a queue. A nearly expired request may be worthless to process. A low-priority bulk task can yield to interactive work.

Queue discipline therefore embeds values: urgency, fairness, efficiency, deadlines and consequence.

Scheduling is not separate from latency. It allocates latency.

See How Scheduling Works.

The Deadline Propagation Principle

In a multi-stage system, downstream components should know how much time remains.

Google SRE guidance discusses deadline propagation because a backend should not spend expensive work on a request whose caller has already given up.

The human analogue is obvious.

If a decision must be made by noon, the analyst should know the decision deadline. If an examination begins next week, feedback designed to change examination behaviour must arrive before then. If a shipment must catch a vessel, upstream processing needs the departure window.

Without deadline propagation, stages optimise locally while the end-to-end task expires.

Latency and Reliability

Slow response can become failure even when the system is technically available.

A website that returns every request after two minutes is “up” in a narrow sense and unusable in a practical one.

This is why reliable systems treat latency as a service-level indicator alongside errors, traffic and saturation.

Reliability is receiver-centred. The system has to work within the time frame in which the receiver can still use it.

Latency and Safety

Safety-critical systems often have hard response-time requirements.

A protective device that detects danger accurately but acts after damage occurs has failed its job.

This is why safety design identifies time-to-harm as well as probability and severity.

But safety can also justify added latency through verification.

The design problem is a race between two clocks:

time needed to become sufficiently sure versus time remaining before action loses value

Latency and Irreversibility

Delay becomes more consequential when an action cannot easily be reversed.

Sometimes waiting is valuable because new information may arrive before commitment.

Sometimes waiting is dangerous because the world may cross a threshold after which correction is impossible or much more expensive.

This creates one of the deepest decision tensions in the world:

wait to learn more versus act before the window closes

The fourth article in this batch develops that problem directly.

Latency in Learning

Learning needs time, but learning also suffers from badly placed delay.

A student attempts a problem and receives immediate confirmation. That can help calibrate a simple skill.

For deeper tasks, immediate correction is not always ideal; the student may need productive struggle, reflection or self-checking before receiving the answer.

But feedback that arrives so late that the student no longer remembers the reasoning path loses diagnostic value.

The educational question is therefore not “instant feedback or delayed feedback?”

It is:

When should feedback arrive so the learner still has enough memory and agency to use it?

Latency in Examinations

An examination is a hard deadline system.

The answer in the student’s mind has no value to the marker until it becomes visible on the paper or screen. Recognition must happen before time expires. Retrieval must happen. Reasoning must happen. Execution must happen. Checking must happen.

A student can therefore possess knowledge and still suffer excessive retrieval or execution latency.

Repair should distinguish “does not know” from “knows but cannot access quickly enough under these conditions.”

That is a much higher-resolution diagnosis.

Latency in Organisations

Organisations can survive with imperfect information if they correct quickly.

They can also fail with excellent information if correction is too slow.

A frontline worker notices a recurring problem. The report enters a system. It is summarised monthly. A committee reviews it quarterly. Funding is approved in the next cycle. Procurement takes months. Training follows.

By the time repair arrives, the problem may have changed, spread or become normalised.

Organisational intelligence therefore depends not only on sensing quality but on time-to-authorised-action.

Latency in Government

Public institutions face a difficult timing problem because legitimacy often requires process.

Consultation, evidence, legal review, budgeting, parliamentary procedure, procurement and implementation all take time.

That delay can protect against arbitrary action.

It can also make institutions slow relative to fast-moving threats, technologies or markets.

The solution is not simply “government should move faster.” Different decisions deserve different clocks.

Emergency action, ordinary policy, rights-sensitive decisions and long-horizon infrastructure should not share one latency architecture.

Latency in Science

Scientific knowledge has latency too.

Research is conducted, analysed, written, reviewed, published, replicated, synthesised, taught and eventually translated into practice.

Some delay is quality control. Some is administrative. Some comes from the genuine time needed for evidence to accumulate.

During emergencies, the pressure to reduce latency rises. Preprints, rapid reviews and living evidence syntheses can move information faster, but speed also raises the risk that weak findings travel before correction.

Again: timely correctness, not speed alone.

The Freshness Problem

Every dataset should answer a quiet question:

Fresh enough for what?

A population census can be useful for years. A traffic-control signal may need second-level freshness. A payment balance may need stronger recency than an archive catalogue. A learning diagnosis may remain useful for a term or become obsolete after intensive intervention.

Freshness requirements depend on the velocity of the state being measured and the cost of acting on stale information.

The Latency–Accuracy Frontier

In many systems, additional accuracy requires additional time.

More samples. More checks. More computation. More consultation. More verification.

At first, extra time may buy large improvements in confidence. Later, it may buy tiny improvements while the world continues changing.

A mature decision process therefore asks for a stopping rule.

When is the answer good enough to act?

That is not anti-evidence. It is evidence disciplined by a deadline.

The Latency–Cost Frontier

Low latency usually costs something.

Extra servers sit idle to absorb bursts. More staff are available. Inventory is positioned closer to demand. Faster transport is used. Data is replicated. Teams maintain on-call coverage. Emergency lanes remain open.

A perfectly responsive system can be extremely expensive.

This is why latency targets should be tied to value rather than prestige.

Do we need 10 milliseconds, 100 milliseconds, one hour or one week?

“As fast as possible” is not a specification.

Precomputation: Pay Before the Request Arrives

One way systems reduce response latency is by doing work in advance.

Indexes are built before searches. Inventories are stocked before orders. Emergency plans are written before disasters. Students practise foundational skills before examinations. Organisations pre-authorise certain responses before crises.

Precomputation trades flexibility and storage for speed.

The risk is staleness. Precomputed answers are fast only while their assumptions remain valid.

Caching: Remember the Recent Answer

Computing systems often store recently used results so they do not have to recompute or retrieve them from a slower source.

Human organisations do the same thing through templates, precedents, checklists and standard operating procedures.

Caching lowers latency.

It also creates a freshness problem.

The old answer is fast. Is it still true?

This tension appears everywhere: speed through reuse versus accuracy through revalidation.

Parallelism: Do More at Once

A task can sometimes be split so independent parts proceed simultaneously.

This can reduce total latency.

But parallel work creates coordination cost, duplicated effort and merge problems. If one branch depends on another, the apparent parallelism may be false.

Human teams discover the same thing. Four people do not automatically make a sequential four-hour task finish in one hour.

Dependencies determine which time can actually be compressed.

Speculation: Start Before You Know You Need It

Some systems reduce latency by predicting likely future requests and beginning work early.

This can be powerful when prediction is good and the cost of wasted work is low.

It can also waste resources, reinforce mistaken assumptions or pre-commit a system too early.

Speculation therefore connects latency with forecasting and irreversibility.

The earlier you begin, the faster you can respond—but the less certain you may be that the predicted future will arrive.

The Latency Audit

Define the trigger. What event starts the clock?
Define the receiver. Who needs the response?
Define the deadline. When does the response lose value?
Decompose the path. Detection, transmission, queue, processing, decision, action and effect.
Measure the distribution. Median, high percentiles and extreme tails—not just average time.
Measure arrival rate. How much demand enters the system?
Measure capacity and saturation. Where do queues form?
Check freshness. How fast does the underlying state change?
Check retries and feedback. Does delay create more demand or overcorrection?
Check handoffs. How much time is spent waiting between owners?
Check batching. Is efficiency being bought with excessive delay?
Check priority. Do urgent items have a safe faster route?
Check safety. Which deliberate delays must remain?
Check irreversibility. Is waiting preserving option value or closing a window?
Measure end-to-end return. Did the response arrive while the receiver could still use it?

When the Latency Lens Fails

Not every slow outcome is a latency problem.

A task may genuinely require a long physical process. A student may need time to develop conceptual understanding. A court may require procedure to protect rights. A scientific question may need years of observation.

Calling all duration “latency” encourages reckless compression.

Use the concept when delay separates a trigger from a response and that gap affects system performance.

The Most Important Timestamp Is the Receiver’s

Teams like to measure when they completed their work.

The receiver cares when the result became usable.

A report finished on Monday but unread until Friday did not enter the decision on Monday. A parcel reached a depot but not the customer. A teacher marked a script but the student did not receive the feedback. A sensor recorded the event but the alert never reached the operator.

Latency should be measured to receipt.

Otherwise systems congratulate themselves for finishing upstream while the world waits downstream.

How Latency Connects to the Rest of the World

Capacity: saturation creates queues and rising delay.
Scheduling: priority rules allocate waiting time across tasks.
Feedback: delayed return can cause oscillation and overcorrection.
Monitoring: stale observations can make dashboards describe yesterday.
Control systems: response time helps determine stability.
Emergence: local delays can generate collective waves and cascades.
Information asymmetry: one party can know the same fact earlier than another.
Friction: waiting is one cost of crossing an interface.
Path dependence: slow correction gives mistakes time to accumulate dependencies.
Risk: the value of speed rises as time-to-harm shrinks.
Irreversibility: waiting can either preserve options or destroy the chance to act.
Verification: checking reduces error but consumes latency budget.

Questions a Reader Can Now Ask

When does the clock actually start?
When does the receiver stop caring?
Where is the largest delay?
How much time is spent working versus waiting?
Is the average hiding a slow tail?
Does load increase waiting nonlinearly?
Does delay make information stale?
Does delay trigger retries or overreaction?
Can urgent work bypass routine queues?
Which delays are safety controls?
Can work be precomputed, cached or parallelised?
Would doing so create staleness or waste?
Are we measuring completion at the sender or receipt at the receiver?

Frequently Asked Questions

Is latency just another word for slowness?

No. Latency is specifically the delay between an initiating event and a corresponding response or effect. A long process can be slow without being poorly latent if that duration is intrinsic to the job.

Why do queues increase latency?

Because requests spend time waiting before service begins. Near or beyond capacity, waiting time can dominate actual processing time.

Why are percentiles useful?

Because averages can hide a minority of very slow experiences. High percentiles reveal the tail of the latency distribution, which may matter greatly in multi-stage or high-stakes systems.

Is lower latency always better?

No. Verification, deliberation and batching can create valuable delay. The correct target is appropriate latency for the receiver’s deadline and the risk of error.

What is the most useful latency question?

Ask when the response stops being useful. That gives the system a real temporal requirement instead of the vague instruction to “be faster.”

Research Basis and Further Reading

Google Site Reliability Engineering, “Monitoring Distributed Systems”, for latency as one of the four golden signals.
Google Site Reliability Engineering, “Service Level Objectives”, for latency distributions, percentiles and tail behaviour.
Google Site Reliability Engineering, “Addressing Cascading Failures”, for queues, overload, retries, deadlines and graceful degradation.
MIT, Queueing Models lecture notes, for Little’s Law and the relation among arrival rate, work in system and time in system.
Google Site Reliability Engineering, “Embracing Risk”, for the trade-off between low-latency and throughput-oriented workloads.

The Larger Idea

We often judge systems by whether they know the right thing.

Did the sensor detect the problem?

Did the analyst identify the cause?

Did the teacher recognise the misconception?

Did the organisation choose the correct response?

Those questions matter.

But the world keeps moving while we answer them.

A system does not only need truth.

It needs truth in time.

Sometimes one second is generous. Sometimes a month is fast. Sometimes waiting is wisdom because more evidence will arrive. Sometimes waiting is the decision that closes the door.

Latency teaches us to put a clock beside causality.

The right response is only fully right if it reaches the world while the world can still use it.

How The World Works | Latency — Why Delay Changes What a System Can See and Do