Find your next learning step
Waiting for N jobs before starting can reduce repeated setup, yet it deliberately transfers time from the machine ledger to early customers. The policy is understandable only when shutdown, off-time, setup and service are kept separate.
Open the chapter contents · Queueing Theory root · Gold Standard of Education · How X Works
Chapter contents
Choose the route closest to the question, or read from Chapter 1 for the complete argument. The chapters remain open on the page; these grouped routes are an index.
Route 1 · Define the policy states
Route 2 · Work the threshold ledger
Route 3 · Test cost and transfer
Route 4 · Check limits and evidence
Route 5 · Ask and continue
CHAPTER 1 OF 12 · DEFINE THE POLICY STATES
1. Define the N-policy clock
An N-policy keeps a server off after the system becomes empty and starts it only when the queue reaches a threshold of N jobs. The policy is not simply “batching”. It is a state rule with at least four timestamps: empty-system entry, shutdown completion, threshold crossing and service readiness.
The attraction is setup amortisation. Fewer starts can mean less warm-up energy, operator effort or connection establishment per completion. The cost is deliberate waiting: the first job in a new cycle may sit while the remaining N − 1 jobs arrive.
CHAPTER 2 OF 12 · DEFINE THE POLICY STATES
2. Keep shutdown, off-time and setup separate
A delayed shutdown leaves the server ready for a grace interval after the queue empties. If a new job arrives inside that interval, service can resume without a fresh setup. If the grace expires, shutdown may itself take time, followed by an off state and then a setup after the threshold is met.
These intervals have different meanings. Shutdown is a transition cost, off-time is a policy state, and setup is a return-to-service transition. Combining them into one “idle” duration hides which control parameter changed.
| Interval | Trigger | What a job can experience |
|---|---|---|
| grace period | queue becomes empty | immediate service if the server remains ready |
| shutdown | grace expires | delay or a cancelled shutdown, depending on the rule |
| off state | shutdown completes | wait for the N threshold |
| setup | threshold is crossed | wait until useful service is ready |
CHAPTER 3 OF 12 · WORK THE THRESHOLD LEDGER
3. Work a threshold ledger
Consider a fictional server with N = 3, deterministic interarrival gaps of four minutes, a six-minute setup and two-minute services. It has already shut down. Jobs arrive at minutes 0, 4 and 8. The threshold is reached at minute 8; setup ends at 14; completions occur at 16, 18 and 20.
The three arrival-to-completion times are 16, 14 and 12 minutes, for a mean of 14. The first arrival waits longest even though all three share one setup. Setup effort is 6/3 = 2 server-minutes per completed job, but that quotient is not any particular customer’s waiting time.
| Job | Arrival | Threshold wait | Completion | Sojourn |
|---|---|---|---|---|
| A | 0 | 8 min | 16 | 16 min |
| B | 4 | 4 min | 18 | 14 min |
| C | 8 | 0 min | 20 | 12 min |
CHAPTER 4 OF 12 · WORK THE THRESHOLD LEDGER
4. Compare immediate and threshold starts
With N = 1, each arrival to an off server triggers setup immediately. For the same three arrivals, whether the system empties between jobs depends on the setup and service durations. That dependence is exactly why a fair comparison must replay one arrival trace through both policies rather than compare unrelated averages.
The threshold policy usually reduces start frequency only by accepting more accumulation delay. It can still reduce total delay when repeated setups under N = 1 are long enough, arrivals are close enough, and the threshold batch clears efficiently. No direction is guaranteed without the full cycle ledger.
CHAPTER 5 OF 12 · WORK THE THRESHOLD LEDGER
5. Add delayed shutdown as hysteresis
Suppose the server stays ready for five minutes after emptying. A job arriving four minutes later avoids both shutdown and setup. A job arriving six minutes later may meet a server that has just crossed the shutdown boundary. The one-minute difference can create a discontinuous delay jump.
This is hysteresis: the start decision depends on the current queue, while the stop decision uses a different boundary or timer. Hysteresis can prevent rapid on–off cycling, but it must be represented explicitly. The warm-pools guide supplies the corresponding ledger for readiness held in reserve.
CHAPTER 6 OF 12 · TEST COST AND TRANSFER
6. Changed context: move one arrival
Return to N = 3, but move Job C from minute 8 to minute 20. Jobs A and B now wait 20 and 16 minutes merely for the threshold. Setup finishes at 26, and completions occur at 28, 30 and 32. Their sojourns become 28, 26 and 12 minutes, with a mean of 22.
The setup effort per job is still two minutes, yet mean sojourn rises by eight minutes. This transfer check separates the engineering saving from the arrival-dependent waiting penalty. A threshold that looks efficient under a burst can be harsh under sparse demand.
CHAPTER 7 OF 12 · TEST COST AND TRANSFER
7. Optimise a cost, not a slogan
A common analysis combines holding cost, setup cost, operating cost and shutdown cost per unit time. Changing N shifts all four. A higher threshold may reduce starts while increasing queue area. A longer grace period may reduce setups while consuming more ready-state resource time.
State the cost units before optimising. Dollars per start cannot be added to customer-minutes without a conversion weight. If the weights are policy choices, show sensitivity rather than presenting one optimum as a physical constant.
CHAPTER 8 OF 12 · TEST COST AND TRANSFER
8. Do not confuse policy thresholds with capacity
An N-policy does not create service capacity. Once ready, the server must still clear work faster than it arrives on the relevant long-run scale. The threshold changes cycle structure and delay; it does not rescue a system whose usable service capacity is below offered load.
Readiness can also fail. Scheduled, running and service-ready states differ, as the cold-start guide explains. Count failed starts and cancelled shutdowns, not only successful cycles.
CHAPTER 9 OF 12 · CHECK LIMITS AND EVIDENCE
9. Limits and misconceptions
The worked example is deterministic, single-server and non-abandoning. Random arrivals, batch arrivals, priorities, multiple servers, vacations, failures and setup interruption change the result. Some policies begin setup before N and complete it at N; others wait until N before beginning setup.
“Larger N is more efficient” is incomplete. It may amortise setup while worsening first-arrival delay, abandonment or deadlines. “Delayed shutdown is always better” is also false when ready-state cost dominates or when shutdown cannot be cancelled safely.
CHAPTER 10 OF 12 · CHECK LIMITS AND EVIDENCE
10. Build an auditable policy experiment
Log queue length and policy state at every transition. Record empty time, grace expiry, shutdown start and end, threshold crossing, setup start and end, service readiness, job starts, completions, abandonments and failed transitions. Preserve one cycle identifier across the whole path.
Replay a common arrival trace for several N values and grace periods. Report setup count, cancelled shutdowns, ready-idle time, queue area, first-job delay, later-job delay, percentiles, abandonment and completed work. The setup-times guide explains why the first service must remain identifiable.
CHAPTER 11 OF 12 · ASK AND CONTINUE
11. Questions readers should ask
Is N the batch size?
Not necessarily. N is the activation threshold. More arrivals can join during setup and service, so the busy period may contain more than N jobs.
Does a grace period remove the need for N?
No. It handles short idle gaps. After shutdown, the threshold can still govern reactivation.
What should be compared first?
Use the same arrivals and service requirements, then compare customer delay, queue area, transition counts and resource-time by state.
CHAPTER 12 OF 12 · ASK AND CONTINUE
12. Continue through verified routes
Begin with the Queueing Theory root, Gold Standard of Education and How X Works. Continue with the warm-pools, cold-start and setup-times guides linked above, and Delayed Feedback, Oscillation and Autoscaling Stability when the threshold is embedded in a controller.
Primary research routes include Ke, Wu and Zhang’s bulk queue with vacations, setup, N-policy and closedown, Wang, Cao and Li’s M/G/1 N-policy with server setup10088-4), and Ke and Pearn’s optimal control with vacations, startup and breakdowns00235-8). The threshold and changed-arrival ledgers are original teaching examples.
Top · Contents · Previous · How X Works