Find your next learning step
A replica can exist in an inventory before it can complete useful work. Scheduling, image start, application warm-up, readiness checks and routing split declared capacity from effective service. Queue analysis must follow the capacity that actually completes jobs.
Open the chapter contents · Queueing Theory root · Gold Standard of Education · How X Works
Chapter contents
Choose the route closest to the question, or read from Chapter 1 for the complete argument. The chapters remain open on the page; these grouped routes are an index.
Route 1 · Separate capacity states
Route 2 · Work the cold-start ledger
Route 3 · Test readiness and buffering
Route 4 · Check evidence and limits
Route 5 · Ask and continue
CHAPTER 1 OF 12 · SEPARATE CAPACITY STATES
1. Capacity has several timestamps
A queueing model should distinguish requested, scheduled, running, ready, routed and completing capacity. A worker may have a process ID yet still be downloading data, warming a cache or failing its readiness check. Only the stage that can accept and complete the defined jobs belongs in the service ledger.
The earlier guides on delayed feedback and replica caps explain decisions and constraints. This continuation follows the actuator from declaration to useful work.
| State | What it proves | What it does not prove |
|---|---|---|
| scheduled | a placement was assigned | the application has started |
| running | a process is executing | it can accept traffic |
| ready | its gate currently passes | it is receiving work |
| completing | useful jobs finish | every job class is healthy |
CHAPTER 2 OF 12 · SEPARATE CAPACITY STATES
2. Cold start is a service interruption for new capacity
Cold-start delay is the time from a scale decision to the first useful completion attributable to the new worker. Break it into scheduling, image or code start, initialization, readiness stability, routing and first-service time. The pieces have different remedies.
A mean startup time is not enough when rare slow starts shape peak backlog. Keep percentiles and the conditions behind them, such as image cache state, node availability and job type.
CHAPTER 3 OF 12 · WORK THE COLD-START LEDGER
3. Work a readiness-gated queue
Use one-minute intervals. Existing capacity completes four jobs per minute. A burst contributes eight jobs per minute for four minutes. At the start of minute 2, three extra replicas are created, but they become ready only at the start of minute 4. Begin with no queue.
Completions are limited by one ready replica in minutes 1–3, so the ending queues are 4, 8 and 12. In minute 4, four replicas can complete up to 16 jobs; with eight new arrivals, the queue falls to 4. The inventory showed four replicas from minute 2, but useful capacity remained four jobs per minute until minute 4.
| Minute | Arrivals | Created replicas | Ready replicas | Ending queue |
|---|---|---|---|---|
| 1 | 8 | 1 | 1 | 4 |
| 2 | 8 | 4 | 1 | 8 |
| 3 | 8 | 4 | 1 | 12 |
| 4 | 8 | 4 | 4 | 4 |
CHAPTER 4 OF 12 · WORK THE COLD-START LEDGER
4. Flow balance validates the capacity story
For each interval, ending queue equals starting queue plus admitted arrivals minus completions, bounded below by zero. Do not substitute nominal capacity for completions. A ready worker may still be idle because routing, concurrency or a downstream dependency blocks it.
Reconcile unique jobs, retries, rejections and abandonments. A cold start can raise timeouts, and timeout retries can make attempt arrivals rise even when original demand is unchanged.
CHAPTER 5 OF 12 · WORK THE COLD-START LEDGER
5. Changed context: one minute faster readiness
Keep the same burst but make the new replicas ready at the start of minute 3. The minute-3 completion ceiling becomes 16 rather than 4, so the queue can clear during that interval instead of reaching 12. The controller decision and replica count did not change; actuation delay did.
This transfer check prevents threshold tuning from receiving credit for a startup improvement. It also shows why a created-replica graph alone cannot reproduce the queue path.
CHAPTER 6 OF 12 · TEST READINESS AND BUFFERING
6. Readiness gates protect routing
A readiness gate keeps traffic away until a stated condition passes. A permissive gate may admit work before dependencies or caches are usable, turning “capacity” into errors and slow service. An overly conservative gate may withhold healthy capacity and lengthen the queue.
Kubernetes documents readiness and startup behaviour, and the Horizontal Pod Autoscaler treats startup and readiness transitions specially when considering metrics. Those implementation rules do not replace an application-level service test.
CHAPTER 7 OF 12 · TEST READINESS AND BUFFERING
7. Scale to zero changes the first request
With zero ready workers, the first event must be observed by an external metric or buffer, trigger capacity, wait through startup and then reach service. Kubernetes notes that scale-to-zero suits work that can wait in a durable queue; a Service itself does not buffer request traffic.
Define where work waits and what happens before readiness. Dropped HTTP requests and durably queued jobs are different outcomes even if both show zero application completions during startup.
CHAPTER 8 OF 12 · TEST READINESS AND BUFFERING
8. A ready flag can still overstate service
Readiness is a gate, not a throughput measurement. A worker may pass a shallow health check while a specialist job class, database path or external dependency remains unavailable. Segment completions and errors by route or class.
Compare readiness time with first accepted job and first successful completion. Persistent gaps identify routing or application bottlenecks after the gate.
CHAPTER 9 OF 12 · CHECK EVIDENCE AND LIMITS
9. Limits and misconceptions
Cold start is not always the dominant delay. Metric windows, controller intervals, caps and downstream service can dominate. Nor does a faster start prove safe initialization.
The worked example assumes identical jobs, immediate routing after readiness and deterministic capacity. Real service rates vary. Use the table as an accounting model, not a universal autoscaling prescription.
CHAPTER 10 OF 12 · CHECK EVIDENCE AND LIMITS
10. A practical evidence check
Join decision, placement, container start, startup-probe, readiness, route-registration, first-accept and first-completion timestamps. Reconcile the queue from event records before interpreting dashboards.
Replay idle-to-burst and warm-to-burst cases. Record peak queue, clear time, failed attempts, startup distribution and replica-time. Change readiness logic and image cache state separately.
CHAPTER 11 OF 12 · ASK AND CONTINUE
11. Questions readers should ask
Is a running replica service capacity?
Not automatically. It must be ready for the defined work, routed and able to complete it successfully.
Does scale to zero always save resources without cost?
No. It trades idle capacity against cold-start delay and requires a suitable trigger and waiting boundary.
Why record first completion?
It is direct evidence that new capacity became useful to the queue, not merely visible to orchestration.
CHAPTER 12 OF 12 · ASK AND CONTINUE
12. Continue through verified routes
Start with the Queueing Theory root, Gold Standard of Education and How X Works. Read Time-Varying Arrivals and Capacity That Arrives Too Late, then the verified delay and anti-windup guides linked above.
For implementation facts, use the official Kubernetes HPA documentation and Scale to Zero guide. The interval ledger is an original teaching example.
Top · Contents · Previous · How X Works