The 30B result
The cache survived the scale jump, and two small adapters improved it in different ways.
We transferred a 1,024-token KV cache from Qwen3-30B-A3B-Instruct-2507 into its exact-architecture Thinking sibling, then measured the target distribution for 32 oracle-forced steps. Direct reuse began at 0.020145 mean KL32 and 93.08% top-1 agreement.
A static ridge map produced the lowest mean divergence, 0.017956 KL32, a 10.86% point reduction from direct reuse. A rollout-trained residual applied directly to the source cache reached 0.018833 and supplied the strongest paired evidence: 20 of 28 windows improved, with a 6.51% mean reduction and a 95% interval for the absolute improvement from 0.000318 to 0.002307.
This is evidence that useful cache structure and correctable trajectory error persist at 30B scale. It is not yet a claim about open-ended response quality, serving latency, or other model pairs.
A controlled sibling pair
Matching architecture removed the usual shape mismatch.
The ordered pair shares the same 48-layer mixture-of-experts architecture, tokenizer vocabulary, attention geometry, RoPE configuration, and 262,144-token context limit. Chat templates differ, so the experiment used raw token sequences instead of chat formatting.
- Source
- Qwen3-30B-A3B Instruct
- Target
- Qwen3-30B-A3B Thinking
- Ridge fit
- 32 FineWeb-Edu samples
- Checkpoint selection
- 4 frozen validation samples
- Final evaluation
- 28 frozen WikiText windows
- Evaluation path
- 1,024-token prefix · 32 forced steps
- Cache geometry
- 48 layers · 4 KV heads · 128 dimensions
- Hardware
- 2 × NVIDIA H100 80GB
Every final row was paired: the four cache paths saw the same input and the same target-forced continuation. An independent recomputation verified 28 unique sample IDs, cache-shard checksums, mapper provenance, and both selected residual checkpoints.
The coordinate map
One corresponding source layer was enough for every target layer.
The ridge mapper used k=1: each target layer could select one source layer. All 48 selected their same-index sibling. Across those chosen pairs, mean K/V reconstruction R² was 0.953 and mean attention-output cosine was 0.977.
That clean diagonal is the mechanistic reason the simple map remained competitive. The two checkpoints organize their cache in closely related coordinates, so a per-head linear translation can recover much of the remaining representation difference without searching across depth.
The rollout correction
Behavior-aware training concentrated on the hardest direct transfers.
The second adapter was a rank-8 linear residual for each layer and KV head. Instead of minimizing cache reconstruction error, it trained through the frozen Thinking model to reduce divergence over the same 32-step rollout metric used for evaluation.
Its gain over direct reuse was not only a lower average. The paired improvement interval stayed above zero, the two-sided paired test gave p = 0.0116, and improvement correlated with the error left by direct reuse at r = 0.72. The larger the initial mismatch, the more correction the residual tended to provide.
Placement mattered. After ridge had already removed most of the structured coordinate error, the rank-8 residual reached 0.018333 KL32, 2.1% worse than ridge alone. Its improvement correlation fell to r = 0.13. At this scale, alignment and rollout correction were both useful, but stacking them was not automatically additive.
Small-data discipline
The useful checkpoint appeared early, so selection stayed frozen.
The direct-base residual reached its best four-sample validation KL32 at step 5 and degraded afterward. The ridge-base run moved within a much narrower, noisier band and selected step 70. Both choices were fixed before any successful final-set row was produced.
The follow-up rank and learning-rate sweep reinforced the capacity lesson on this same validation set. Rank 4 at learning rate 1e-3 selected the lowest direct-base validation KL32, while larger ranks did not improve monotonically. Because that comparison used only four validation samples, it guides the next experiment rather than replacing the 28-window result.
What this establishes
Scale preserved the signal, not the right to generalize.
The 30B sibling result establishes three useful facts. A foreign sibling cache can retain more than 93% next-token agreement without translation. A matching-layer linear map can produce the strongest average trajectory match. And a compact rollout-trained residual can deliver a positive paired correction when it is placed on the harder direct path.
The uncertainty is equally important. Ridge beat direct reuse on the point estimate, but its paired 95% improvement interval crossed zero. The final set covers 28 WikiText windows under oracle forcing, not free generation. No 30B end-to-end latency was measured. The next gate is therefore a larger, multi-corpus replay with functional generation and synchronized serving measurements.
At 30B, cache transfer remained measurable, correctable, and simple enough to diagnose.