Faster inference,
starting with prefill

Measured 2026-10-24 on 4×H100 (HF forwards, bf16/FP8, p50 of 2 reps). Solid-border bars = measured transfer components; dashed border = projected (prefill quadratic, map/move linear, first-token affine fits). The projected 64K total (4,010 ms) matches an independently measured 64K run (4,013 ms). Black tick = native 128B TTFT (measured through 64K). At 1M projected: native ≈2h, transfer ≈11min.

Measured 2026-10-24 on 4×H100 (HF forwards, bf16, p50 of 2 reps), largest receiver shown (1.7B → 32B): 5.7–8.6× faster. Solid-border bars = measured; dashed border = projected past 32K (model context limit 40K). Black tick = native 32B TTFT. Smaller receivers: 14B 3.0–4.0×, 8B 1.9–2.6×, 4B 1.4–2.0×.

Measured 2026-10-22 on 8×H100 (HF forwards, bf16, long-context campaign; one rep per long point). Solid-border bars = measured transfer components through 524K; dashed border = projected to 1M. Native 72B OOMs past 127K on this rig, so its tick is projected (dashed) beyond that. The cache move is folded into the map phase in this campaign’s instrumentation. At 1M projected: native ≈76min, transfer ≈9min.

Illustrative Gemma-4-E2B → 31B timing projection anchored to the measured Qwen3 1.7B→32B H100 campaign. Source prefill is scaled by the 2B/1.7B parameter ratio; KV map, cache move, first-target-token, and native 31B series reuse the similarly sized 32B receiver campaign. All dashed-border bars and native ticks are projections pending a dedicated Gemma timing run.

Projected Qwen3.5-2B → Qwen3.8-27B timing from the measured 4B→27B campaign: source prefill is scaled by the 2B/4B parameter ratio; KV map, cache move, first-target-token, and native 27B series reuse the measured campaign values. Dashed-border bars indicate that the 2B transfer totals are projections pending a dedicated timing run.

Illustrative Gemma-4-E4B → 31B timing projection anchored to the measured Qwen3 1.7B→32B H100 campaign. Source prefill is scaled by the 4B/1.7B parameter ratio; KV map, cache move, first-target-token, and native 31B series reuse the similarly sized 32B receiver campaign. All dashed-border bars and native ticks are projections pending a dedicated Gemma timing run.

Measured 2026-10-24 on H100s (HF forwards, bf16, p50 of 2 reps): ≈2–2.5× faster than native 27B prefill (32K native on 2×H100). Solid-border bars = measured through 64K; dashed border = projected (prefill quadratic, map/move linear, first-token affine). Black tick = native 27B TTFT.

Illustrative GLM-4.7-Flash → GLM-5.3 timing projection anchored to the measured Qwen3.5-4B → Qwen3.8-27B H100 campaign (similar flash-class source and large receiver; prefill quadratic, map/move linear, first-token affine fits). All dashed-border bars and native ticks are projections pending a dedicated GLM timing run. GLM-5.3's 198K context window caps directly servable contexts at the 198K column; 512K and 1M are extrapolations past the window, shown for scale.

Measured 2026-10-24 on 4×H100 (HF forwards, bf16, p50 of 2 reps): 1.9–2.6× faster than native 8B prefill. Solid-border bars = measured; dashed border = projected past the 32K measurement (model context limit 40K; prefill quadratic, map/move linear, first-token affine). Black tick = native 8B TTFT.