DFlash 2 Doubles Qwen 3.8 27B Again: The VRAM Cost

Does DFlash 2 Actually Double Qwen 3.8 27B?

Yes, against a plain baseline. One A100 test went from 28.9 to 59.1 tok/s, and one RTX 4090 test went from 60 tok/s on MTP to about 90 on DFlash 2. The word “again” matters. DFlash 2 is not a faster MTP. It is a second, independent speed lever, a separate 3.85 GB draft model that proposes whole blocks of tokens in one pass, and it costs VRAM rather than a flag. Two testers, two cards, every number attributed below. This is the honest version of a claim that started travelling before its caveats did.

What DFlash 2 Is

DFlash is “Block Diffusion for Flash Speculative Decoding”, from the z-lab/dflash project on GitHub, 5,715 stars. Speculative decoding works like a rough first draft: a small draft model proposes tokens, and the full target model checks them, keeping only what it would have written itself. That is why the output is identical by construction. A bad draft just wastes time; it never changes a single token.

Standard speculative decoding drafts one token at a time. DFlash 2 drafts a whole block in one pass, keeps the top candidates at every position, and a selector traces one coherent path through them. Two-tap dynamic convolutions keep the draft from decaying toward the end of the block. The target is still Qwen 3.8 27B. The draft is incoai/Qwen3.8-27B-DFlash2, mirrored at z-lab/Qwen3.8-27B-DFlash2, and it is a 3.85 GB model, not a config flag.

MTP vs DFlash 2: Two Different Kinds of Fast

Qwen 3.8 27B already has a built-in speed multiplier, multi-token prediction. Our speed page covers it. DFlash 2 is the second one, and they are not the same mechanism.

The two stack. One tester measured MTP first, then added DFlash 2 on top of it. That is the jump from 60 to 90.

The Measured Numbers

Three independent tests exist so far, all single testers, all attributed. Treat them as three rows, not one average.

Fahd Mirza’s benchmark is in his video, and he states the output is provably identical. @analogalok’s number carries a detail worth reading twice: his headline says 90 tok/s, and his own table, shown below, says 83 to 87. The headline travels. The range is what a card owner should plan around. @ViC305’s row is the cleanest: a plain baseline, 1.98×, and he measured the cost the others skipped. Prefill dropped from 73.38 to 37.62 tok/s because the draft has to prefill the same prompt, at a 54.2 percent acceptance rate. Decode is the win, and for agent work decode is what you wait for.

What It Costs in VRAM

DFlash 2 is not free, but it is cheaper than the headline number. The draft model is 3.85 GB at full precision, and 1.14 GB as the Q4_K_M GGUF the 4090 test actually used. On top of that, draft depth is a dial between VRAM and speed.

@analogalok’s full matrix, on a single RTX 4090 with 24 GB, using Unsloth’s Qwen 3.8 27B UD-Q4_K_XL target and the DFlash 2 draft at Q4_K_M:

The number that matters is the flag --spec-draft-n-max. At 7, the draft states eat too much VRAM on a 24 GB card. At 4, the overhead drops and throughput actually rises. He reports a 5.39 token acceptance rate at that setting. The physical rule from section 5c applies: speculative decoding only helps while the draft accepts enough tokens to pay for its own memory and compute, and on a card where memory is the constraint, a shallower draft wins.

That 110k row, 83.35 tok/s decode sitting at 23.96 GB, is the ceiling on a 24 GB card. Nothing more fits, and the fact that it holds at all is the point.

How to Run It Today

DFlash 2 is landing through open pull requests, not shipped releases. The README links vLLM PR #52816, SGLang PR #35371, and llama.cpp PR #27342, plus an oMLX fork at 0.6.2-dflash2.

On llama.cpp, the PR is patchable now:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git fetch origin pull/27342/head:pr-27342
git switch pr-27342
cmake -B build -DGGML_CUDA=ON && cmake --build build -j

Then serve it with @analogalok’s flags:

./build/bin/llama-server -m Qwen3.8-27B-UD-Q4_K_XL.gguf \
  -md Qwen3.8-27B-DFlash2-Q4_K_M.gguf \
  --spec-type draft-dflash --spec-draft-n-max 4 \
  -c 110000 -ngl 99 -ctv q4_0 -ctk q4_0

Two caveats, stated plainly. First, “already in vLLM, SGLang, and llama.cpp” means open pull requests, and llama.cpp’s DFlash path has open bug reports: corrupted predicted_ms on some Q4 and Metal requests, and an AMD APU regression. It works on the cards people have tested, and it is not a finished, merged feature. Second, we measured none of this. We own no hardware. Every number above is a named tester’s, with their card and their flags.

The target model’s VRAM and quant tiers, for anyone sizing the base model first, are on the VRAM requirements page. The MTP baseline that DFlash 2 builds on top of is on the speed page. The flags and how they change what fits are on the llama.cpp flags guide, and the GPU checker tells you whether your card holds the 20.6 GB base model at all. If it does not, DFlash 2 does not rescue it: the draft needs the target resident first.

FAQ

What is DFlash speculative decoding?
A small draft model proposes blocks of tokens in parallel, and the full target model verifies them, keeping only what it would have written. Output is identical to running the target alone, because the target is still the one deciding every token.

Is DFlash 2 the same as MTP?
No. MTP is a prediction head trained into Qwen 3.8 27B’s own weights and switched on with a flag. DFlash 2 is a separate block-diffusion draft model, 3.85 GB at full precision and 1.14 GB as Q4_K_M, that works alongside the target. They stack.

Does DFlash 2 really double Qwen 3.8 27B?
Against a plain baseline, one A100 test measured 28.9 to 59.1 tok/s. On top of MTP, one RTX 4090 test measured 60 to about 90 tok/s. Both are single testers, so treat them as what they are, two data points, not a spec.

How much VRAM does DFlash 2 add?
The draft model is 1.14 GB as Q4_K_M (3.85 GB at full precision), plus memory for the draft states, which grows with the draft depth. On a 24 GB card, --spec-draft-n-max 4 keeps the base model plus draft under the ceiling while holding around 83 to 87 tok/s decode.

Do I need extra VRAM to run DFlash 2?
Yes. You need the full target model resident first, then the draft model on top. A card that cannot hold the base Qwen 3.8 27B at your chosen quant cannot use DFlash 2 to fix that.

Is DFlash 2 in llama.cpp?
It is available as an open pull request, #27342, patchable today but not yet merged, and its llama.cpp path has open bug reports. It also ships as open pull requests for vLLM and SGLang and an oMLX fork.

Does DFlash 2 change the model’s output?
No. Speculative decoding is lossless by construction. The draft proposes; the target verifies. If the draft is wrong, the target corrects it, and the result is exactly what the target would have produced on its own.