Leaf-spine oversubscription ratio: a 32 × 25G worked example

A leaf with 32 host-facing 25 Gbps ports and four fabric-facing 100 Gbps uplinks is oversubscribed 2:1 when the ratio is stated as host-facing capacity : fabric-facing capacity. The arithmetic is (32 × 25 Gbps) : (4 × 100 Gbps) = 800 : 400 = 2:1. If one uplink is unavailable, the same configured host edge faces only 300 Gbps of fabric capacity, so the ratio becomes 800:300 = 2.67:1 until capacity is restored.

Scope: a capacity worksheet, not a throughput guarantee

Oversubscription compares potential demand at a boundary with the capacity available to carry it. In this example, the boundary is one leaf switch. “Host-facing” means the aggregate nominal line rate of the ports toward servers. “Fabric-facing” means the aggregate nominal line rate of the active uplinks toward the spine layer.

Cisco's two-tier design description connects every leaf to every spine and uses Equal-Cost Multi-Path routing across those Layer 3 links. Its current Massively Scalable Data Center white paper also publishes a close worked example: 48 downlinks at 25 Gbps and eight uplinks at 100 Gbps produce 1,200/800 = 1.5 at each leaf. Juniper's spine-and-leaf sizing paper likewise calculates oversubscription by comparing workload-facing and fabric-facing bandwidth, then warns that traffic concentration can still produce queueing, drops, delay, and jitter.

Those sources do not make a bare ratio self-explanatory. Some documents print fabric : workload, while others print workload : fabric or only a decimal. This article therefore uses one explicit convention throughout:

oversubscription = aggregate host-facing capacity
                   --------------------------------
                   aggregate fabric-facing capacity

A value of 1.0 is balanced by this capacity test. A value above 1.0 means the configured host edge could demand more than the fabric-facing links can carry at once. Do not write only “2:1” in a design record; write “2:1 host-facing : fabric-facing” so a reviewer does not invert it.

Inputs and assumptions

The calculation assumes all links operate at their stated speed and that every configured host port could offer traffic simultaneously. It does not subtract Ethernet framing, forward-error-correction, routing, encapsulation, or application overhead. It also does not assume that actual traffic is evenly distributed across destinations or ECMP paths. Those effects belong in a traffic model and validation plan, not in a port-count shortcut.

The host farms in the figure are aggregates, not literal single cables. Each “32 × 25G hosts” glyph represents 32 distinct host-facing links on that leaf. The four lines from each leaf to the spine row are the four distinct 100G fabric uplinks.

Calculate the normal per-leaf ratio

First total the capacity in each direction without mixing port counts and rates:

host-facing capacity = 32 ports × 25 Gbps/port
                     = 800 Gbps

fabric-facing capacity = 4 uplinks × 100 Gbps/uplink
                       = 400 Gbps

host : fabric = 800 Gbps : 400 Gbps
              = 2 : 1

The units cancel only in the final division. Keep them visible in the intermediate lines: 32:4 = 8:1 is a port-count ratio, not a bandwidth ratio, because the ports run at different speeds.

At full simultaneous offered load, the configured host edge can present 800 - 400 = 400 Gbps more than the leaf's uplinks can carry toward other leaves. That is a capacity envelope, not a prediction that 400 Gbps will be dropped. Traffic between two hosts on the same leaf need not cross the spine layer, and real workloads rarely drive every port at line rate in the same direction at the same instant. Conversely, one hot receiver or a small set of hashed flows can encounter congestion even when average fabric utilization looks comfortable.

Across the four identical leaves shown, the arithmetic scales to 4 × 800 = 3,200 Gbps host-facing and 4 × 400 = 1,600 Gbps leaf-to-spine link capacity. The ratio remains 2:1 because both sides scale by four. Do not use those fabric-wide sums to approve a particular traffic matrix: every inter-leaf flow consumes capacity at both its source leaf and its destination leaf, and the destination edge can be the limiting point.

Recompute after one uplink is unavailable

Suppose Leaf 1 loses its 100G path to one spine while all 32 host ports remain configured:

remaining fabric capacity = (4 - 1) uplinks × 100 Gbps/uplink
                          = 300 Gbps

degraded host : fabric = 800 Gbps : 300 Gbps
                       = 2.666... : 1
                       ≈ 2.67 : 1

capacity reduction = (400 - 300) / 400
                   = 0.25
                   = 25%

The 25% figure is a reduction in that leaf's nominal fabric-facing capacity. It is not a claim that application throughput falls exactly 25%, that convergence is lossless, or that the design meets an availability target. Routing convergence, ECMP rehashing, flow distribution, buffer behavior, and the location of demand decide the observed result.

This degraded-state calculation is a useful invariant: a design review that quotes only the all-links-up ratio has not yet stated its capacity boundary during maintenance or failure.

Reproducible Schematex source

network "Leaf-spine oversubscription: 2:1"
  layout: spine-leaf
  l3switch sp1 "Spine 1"
  l3switch sp2 "Spine 2"
  l3switch sp3 "Spine 3"
  l3switch sp4 "Spine 4"
  switch lf1 "Leaf 1"
  switch lf2 "Leaf 2"
  switch lf3 "Leaf 3"
  switch lf4 "Leaf 4"
  spines: sp1 sp2 sp3 sp4
  leaves: lf1 lf2 lf3 lf4
  serverfarm hosts1 "32 x 25G hosts" count: 32
  serverfarm hosts2 "32 x 25G hosts" count: 32
  serverfarm hosts3 "32 x 25G hosts" count: 32
  serverfarm hosts4 "32 x 25G hosts" count: 32
  lf1 -- hosts1 : "32 x 25G downlinks"
  lf2 -- hosts2 : "32 x 25G downlinks"
  lf3 -- hosts3 : "32 x 25G downlinks"
  lf4 -- hosts4 : "32 x 25G downlinks"

Schematex 1.0.13 strictly parsed and rendered this source with no diagnostics. The SVG reports 12 devices and 20 links: 16 generated leaf-to-spine links from the 4 × 4 full mesh, plus four explicit leaf-to-host-farm links. The network topology syntax reference documents that layout: spine-leaf places two fabric rows and automatically connects every declared leaf to every declared spine. The renderer proves the stated topology is present; it does not infer link speed for auto-generated links or calculate oversubscription.

Charles Clos's 1953 paper is the historical source for the multistage non-blocking switching-network concept behind the modern folded-Clos label. A contemporary Ethernet fabric is not proven non-blocking merely because its drawing has leaf and spine rows. Port allocation, link rate, switch capacity, forwarding behavior, failures, and traffic demand still determine whether the implemented network meets that objective.

Review invariants before approval

1. Name the ratio direction. Record host-facing : fabric-facing, not an unlabeled 2:1.
2. Calculate per leaf first. Each leaf has 32 × 25G = 800 Gbps toward hosts and 4 × 100G = 400 Gbps toward the fabric.
3. Check physical completeness. With four spines and one link from each leaf to each spine, every leaf must have four distinct fabric links; the full four-leaf figure must contain 4 × 4 = 16 such links.
4. Separate configured from populated capacity. If only 20 host ports are populated, current attached line-rate capacity is 20 × 25 = 500 Gbps, or 1.25:1, while the fully configured design remains 2:1. State which number the document controls.
5. Recompute degraded states. One 100G uplink down leaves 300 Gbps and changes the ratio to approximately 2.67:1.
6. Keep switch fabric limits separate. Port arithmetic does not verify forwarding capacity, packet-rate limits, overspeed, breakout constraints, or line-card placement.
7. Test the traffic matrix. Include east–west concentration, same-leaf traffic, storage or backup hot spots, incast, north–south paths, and asymmetric demand.
8. Validate operation. Confirm routing adjacency, ECMP membership, hashing, convergence, telemetry, queue thresholds, optics, cabling, and maintenance behavior on the actual platform and software release.

Failure modes a clean topology can hide

Inverted notation. A worksheet copies 1:2 from a source that writes fabric : workload, while an operator reads it as host : fabric. Both people think they approved the same design. Labels on both sides of the ratio prevent that silent disagreement.

Counting ports instead of bandwidth. Four 100G uplinks and 32 25G downlinks are not 8:1. The rates convert the comparison to 800:400 = 2:1.

Averages masking hot spots. Juniper explicitly distinguishes oversubscription from contention and notes that concentrated traffic can increase queues, loss, delay, and jitter. A low average across all leaves cannot prove one leaf or receiver is safe.

All-links-up math treated as resilience. The topology still looks symmetric after a planned design review, but a failed or administratively disabled uplink changes both capacity and path distribution. Record normal and degraded envelopes separately.

Diagram semantics overstated. Auto-meshing guarantees that the rendered model contains every leaf-to-spine connection. It does not inspect live cabling, negotiate 100G, configure routing, or confirm that ECMP installed all paths.

Paste the source into the Schematex playground and verify that the SVG contains 16 leaf-to-spine links. Then replace the four sizing inputs with one real leaf's bill of materials, write the ratio as labeled arithmetic, and repeat the calculation for every maintenance and failure state the design must tolerate.