How to read these benchmarks
We take the KV cache that a small model built while reading a long context, translate it with a
learned map, and hand it to a bigger model. The bigger model then answers as if it had read the
context itself, without paying for its own prefill. These pages measure how much quality survives
that handoff, and how much time it saves.
The three bars
Every benchmark shows three numbers. The white bar is the small model on its own: the floor.
The dark bar is the big model reading the context natively: the ceiling, and the thing we are
trying to avoid paying for. The colored bar in the middle is the transfer. The closer the colored
bar gets to the dark one, the less quality the translation loses. When the colored bar beats the
white one, the big model is adding reasoning on top of the small model's reading.
The thin gray line is chance. The dashed line is the big model answering with no context at all.
If a bar doesn't clear those, the context isn't doing anything.
Quality vs context length
Same idea, stretched over input length. Solid points are the transfer, with error bars and the
number of items in each bucket. Dashed is the big model native, dotted is the small model native.
The vertical line marks the longest context the map was trained on: past it, you're watching the
map generalize to lengths it has never seen. Quality holding up past that line is the interesting
part. Exact-recall tasks like MRCR are the hardest case: small reconstruction noise that barely
moves a multiple-choice answer will destroy a verbatim copy. We show those anyway.
TTFT
Time to first token. The stacked bar is everything the transfer pays: small-model prefill, the
map itself, moving the cache, and the first token out of the big model. The black tick is what
the big model would have paid to prefill natively. The gap between them is the point of all this.
Bars with dashed borders are projections from fitted curves, not measurements; we label which is
which because the difference matters.
