To the Bedrock
Flow & Queuing

Flow and Waiting

Queueing theory is the formal study of waiting: how demand arrives at a processing step, how capacity at that step determines throughput, and how the relationship between arrival rate and service rate governs the time work spends in the system. Its central result—that waiting grows non-linearly as utilization approaches service capacity—has consequences for any system where variability exists and capacity is shared.

Product development is a queueing system. Features, fixes, reviews, and decisions arrive at constraint steps whose capacity determines flow. Utilization—the ratio of arrival rate to service rate—can appear stable on average while backlog, lead time, and interruption grow disproportionately at the constraint. This divergence between average load and experienced waiting is the utilization trap.

This module applies queueing theory to knowledge work. It covers the core quantities, Little's Law, the M/M/1 baseline model, Kingman's variability approximation, and the connection to Theory of Constraints.

Queueing theory in plain English

Queueing theory is the study of waiting. It asks a simple question with uncomfortable consequences: when work arrives at some rate and a system can only finish work at some other rate, what happens in the space between? That space is the queue.

The word "queue" does not only mean people standing in line. It can be a backlog of tickets, pull requests waiting for review, decisions waiting for a meeting, features waiting for a specialist, incidents waiting for diagnosis, or partially finished work waiting for the next step.

The trap is that averages hide timing. If one item arrives every hour and one item can be finished every hour, the average rates match. But if four items arrive at once, three of them wait even though the average still looks balanced. Add variable item sizes, interruptions, rework, reviews, and handoffs, and waiting becomes the dominant part of elapsed time.

Core quantities

  • Arrival rate (λ): how fast work enters the system.
  • Service rate (μ): how fast the constrained step can finish work.
  • Utilization (ρ) = λ / μ: average load at the constraint.
  • Variability (Ca, Cs): coefficients of variation for arrivals and service times. C = 0 is deterministic, C = 1 is exponential (Poisson arrivals), C > 1 is bursty or fat-tailed. Ca measures arrival unevenness; Cs measures service-time unevenness.
  • WIP (L): average number of items in the system at any moment.
  • Lead time (W): average time an item spends in the system, from entry to exit.

Worked example: if λ = 4 items/hour and μ = 5 items/hour, then ρ = 0.80. Under M/M/1, W = 1 / (μ − λ) = 1 hour. Little's Law then gives L = λ × W = 4 items in the system on average.

Little's Law

Little's Law is the accounting identity of flow. If WIP goes up and throughput does not, where does the pain go? Into time. In any stable system observed over a meaningful interval, average work in the system equals average throughput multiplied by average time in the system.

L is average work in system, λ is average arrival or throughput rate, and W is average time in system.
Variable Mathematical meaning In high-software-content work Order-of-magnitude notes
L Average number of items in the system. Work in process: ideas, features, fixes, approvals, reviews, blocked work, and partially done delivery items. Count with one consistent unit and system boundary.
λ Average arrival rate, equal to throughput rate in a stable system. Rate at which work enters, or exits as completed value once the system is stable. Use the completed throughput rate across the same boundary used for WIP and lead time.
W Average time an item spends in the system. Lead time from accepted work to delivered or validated outcome. Use percentiles alongside averages because product work is usually skewed.

Little's Law does not say why waiting happens. It says the quantities must balance. If throughput is capped and WIP rises, lead time rises with it.

Why high utilization bites

M/M/1 is not a model of software development. It is a stripped-down warning light: one server, random arrivals, random service times, and stable arrival rate below service capacity. Even in that small world, waiting does not rise gently as utilization increases. It bends upward, then turns nearly vertical as arrival rate approaches service capacity.

ρ is utilization, λ is arrival rate, and μ is service rate.
W is average time in system (lead time), μ is service rate, and λ is arrival rate. In M/M/1, W grows without bound as λ approaches μ.

The formula looks simple, but it is a hyperbola, not a line. Rewrite the denominator: μ − λ = μ(1 − ρ), so W = (1/μ) ÷ (1 − ρ) — average service time divided by the spare capacity fraction. When utilization is 50%, spare capacity is 50% and wait is two service times. At 90%, spare capacity is 10% and wait is ten service times. At 99%, wait reaches a hundred service times. The denominator collapsing toward zero is the nonlinear part.

The Poisson arrivals and exponential service times are not in the formula because they are used to derive it. Those distributional assumptions make the queueing math tractable (via a birth-death Markov chain), and the result of all that work happens to be a clean rational expression. The distributions are baked in; the formula is the simplified answer. See the M/M/1 appendix for the full argument.

Where variability enters

Kingman is where the warning starts to look like product work. Real teams do not just have average load; they have uneven arrivals and uneven item sizes. Rework appears, reviews block, priorities change, and dependencies turn one card into six.

Wq is average waiting time in queue, Ca is interarrival variation, Cs is service-time variation, and E[S] is average service time.

The formula has three multiplicative factors: the utilization term ρ / (1 − ρ), which grows steeply near saturation; the variability term (Ca2 + Cs2) / 2, which scales with the combined unevenness of arrivals and service; and E[S], the average service time, which sets the baseline wait. Variability amplifies waiting at any utilization, but near saturation there is no slack to absorb it.

Connection to Theory of Constraints

Queueing theory describes the waiting mechanism. Theory of Constraints offers the complementary management question: where is the constraint, and what would protect its flow?

TOC's defining claim is that the throughput of any system is governed by its constraint. Subordinating all other decisions to the constraint—protecting its input queue from starvation, preventing non-constraints from overfeeding it—turns a diagnosis of high utilization into a tractable management action. Queueing gives the formula for why things are slow; TOC asks what to change first.

From utilization to flow

Henrik Kniberg's resource utilization trap demonstration makes the practical lesson visible. Two lanes process the same bursty arrivals and variable task sizes, but with different release policies: one pushes work aggressively, one caps WIP and protects flow.

  1. Bursty arrivals: the same average load produces very different queues depending on whether work arrives steadily or in waves.
  2. Variable item sizes: fat-tailed effort distributions compound the arrival variability penalty at each stage.
  3. Push policy: starting work early to keep everyone visibly busy raises WIP, which raises lead time by Little's Law.
  4. Pull policy: capping WIP and accepting local slack absorbs variation before it becomes persistent backlog.
  5. Flow measurement: throughput, lead time, and WIP as a system—not resource utilization in isolation.

See Kniberg's resource utilization trap video for the original visual demonstration.

Continue exploring

Taxonomy Classify flow regimes across four dimensions: load, variability, release discipline, and constraint behavior Explore Test assumptions interactively with arrival, capacity, and variability controls Concept Map Trace assumptions, observations, and surprises across connected frameworks M/M/1 Derivation Trace the formula from Poisson arrivals and exponential service times through balance equations to the steady-state result

References

  • Little, J. D. C. (1961). A Proof for the Queuing Formula: L = λW. Operations Research, 9(3), 383–387.
  • Kingman, J. F. C. (1961). The single server queue in heavy traffic. Mathematical Proceedings of the Cambridge Philosophical Society, 57(4), 902–904.
  • Kleinrock, L. (1975). Queueing Systems, Volume 1: Theory. Wiley.
  • Goldratt, E. M., & Cox, J. (1984). The Goal: A Process of Ongoing Improvement. North River Press.
  • Goldratt, E. M. (1990). What Is This Thing Called Theory of Constraints and How Should It Be Implemented? North River Press.
  • Reinertsen, D. G. (2009). The Principles of Product Development Flow: Second Generation Lean Product Development. Celeritas Publishing.
  • Kniberg, H. (c. 2012). Resource Utilization Trap [video]. Resource Utilization Trap on YouTube.