The result

One parallel draft pass replaced a sequence of serial guesses.

On four disjoint prompts, native Qwen3-8B took a median 804 to 845 milliseconds to generate 64 greedy tokens. DFlash completed the same requests in 318 to 432 milliseconds. Every prompt won, with paired speedups from 1.876× to 2.526×.

Across three timed repetitions per prompt, the geometric-mean speedup was 2.154× and the paired median was 2.136×. DFlash produced exactly the same token IDs as the native vLLM baseline in all 12 comparisons.

Grouped horizontal bars comparing native Qwen3-8B and DFlash median latency on four prompts. DFlash is faster on each prompt by 1.88 to 2.53 times.
Disjoint confirmation set, 64 greedy tokens, one warmup and three timed runs per prompt. Prompt 4 had a 2,723-token input. Bars show per-prompt medians, not pooled samples.

How DFlash works

Draft in parallel. Verify with the model that owns the answer.

Ordinary autoregressive decoding predicts one token, updates state, then predicts the next. Speculative decoding makes that loop cheaper by asking a smaller model for several guesses before the target verifies them. The drafter still produces those guesses serially.

DFlash changes the proposal step. Its five-layer draft model conditions on hidden features from five points in the Qwen3-8B target and predicts a 16-token block in one forward pass. The full 36-layer target checks the block, accepts matching positions, and corrects the first mismatch. The target remains the authority over every committed token.

System diagram separating the confirmed exact DFlash path from a future cross-model KV-transfer path. The confirmed path builds Qwen3-8B state, drafts 16 positions in parallel, then verifies with the target.
Solid orange arrows mark the confirmed same-target path. The dashed lower path is an experiment plan, not a measured result.

What we tested

The campaign narrowed a broad design space to one repeatable winner.

We started with public Qwen3-8B and Qwen3-4B DFlash checkpoints, then evaluated the pieces needed for a production path: native serving, fixed and adaptive block widths, exact shared-trunk verification, 8B-to-32B verification, a DFlash/native cascade, conventional speculative decoding, prefix-cache connectors, and long-prompt cache hits.

The strongest observation was consistent across the fixed-width screen: wider parallel blocks amortized target calls better. Four, eight, twelve, and sixteen-token blocks all preserved the native token sequence on the screen, while geometric-mean speedup rose from 1.705× to 2.162×. We therefore carried the full 16-token block into the repeated, disjoint confirmation.

Line chart showing geometric-mean speedup increasing from 1.705 times at a four-token block to 2.162 times at a sixteen-token block.
Selection screen only: one timed run on each of four prompts after warmup. The 16-token point was subsequently tested with three repetitions on a disjoint prompt set.
What we did not promote

Several more complex branches taught us where the system boundary is, but did not clear the same speed-and-exactness gate. Adaptive schedules added correction overhead. Cross-model verification depended heavily on the prompt. A generic prefix-cache connector restored target state without reconstructing DFlash's five draft layers. We do not treat those branches as production evidence.

The KV-transfer connection

DFlash speeds the answer. KV transfer moves the read.

The two technologies attack different parts of inference. KV-cache transfer avoids asking every model to reread a long input. DFlash reduces the serial work required after the cache exists. That makes them complementary, but not interchangeable.

Our separate Qwen3-4B→32B KV-transfer system already demonstrated the prefill side: a 4B model built the long-context cache, a 32B model repaired the latest 32 prompt tokens, and adaptive speculation delivered 1.177× lower total latency with 2.636× faster time to first token. It scored 96/128 on the strict task set, but it was an approximate operating point rather than native-32B-exact inference.

The clean hybrid is to let a larger model build a rich prompt representation, translate or share that state with a compatible parallel drafter, and keep the large model as the final verifier. If draft agreement is high enough, KV transfer removes duplicate prefill while DFlash removes serial proposal work. The next benchmark must measure both savings together rather than adding speedup ratios from separate experiments.

Next experiments

Move from a confirmed component to an end-to-end serving path.

01 · Native DFlash under load1, 8, 32, 128 requestsP50 · P95 · TPS
02 · Method-aware prefix cachingTarget + five draft statesEXACT CACHE HIT
03 · Mapped prefill to exact speculation70B → 8B draft → 70B verifyNO DUPLICATE READ
04 · Confidence-aware truncation4 / 8 / 16-token blocksFIXED-WIDTH CONTROL
05 · Larger-model checkpointTrain DFlash for the target32B OR 70B

Each experiment keeps the same promotion rule: matched hardware and prompts, disjoint confirmation data, repeated timing, and explicit output-quality checks. The 2.154× result shows the potential of parallel drafting. The next job is to preserve that advantage when cache reuse, concurrency, and a genuinely large verifier enter the system.