Can You Run Qwen 3.8 Flash Next NVFP4 on One Machine?
Yes, on a single DGX Spark, and the reason is the most useful thing on this page: you never load two fifths of it.
NVIDIA’s own build is 132.68 GB of weights, which is 123.57 GiB. A Spark has a 128 GB unified pool with roughly 112 to 119 GiB actually available to a process. Held entirely in memory the checkpoint misses, by somewhere between 4.6 and 11.6 GiB, before a single token of KV cache.
It runs anyway, because one 53.72 GB file inside it does not need to be resident.
The File NVIDIA Did Not Quantize
model-fp8-mtp-ple.safetensors holds the per-layer embedding n-gram table and the multi-token prediction module, at FP8. It is 40.5% of the download.
NVIDIA did not compress it, and they did not write it either. Their model card states that the MTP module and the PLE n-gram embedding “match Qwen/Qwen3.8-Flash-Next-FP8 byte-for-byte”, copied out of Qwen’s own FP8 release and shipped unchanged. Two fifths of a checkpoint labelled 4-bit is a file lifted from somebody else’s 8-bit one.
That is not sloppiness. An n-gram lookup table is read sparsely, sixteen rows per token, so quantizing it buys little where the model is least tolerant. Our mixed precision quantization piece carries the full byte breakdown and the 50 quantized against 292 excluded tensor groups.
The sparse read pattern is also what makes the next part work.
Does Qwen 3.8 Flash Next NVFP4 Fit One DGX Spark?
Only if you keep that file off the device, and that is exactly what the people running it do.
Tony Deangelo (@Tech2Wild) published a single-Spark deployment on 5 September that serves NVIDIA’s checkpoint byte for byte, with no requantizing and no repacking. His patch leaves the 47.68 GiB PLE table on NVMe and reads the sixteen rows each token needs on demand. About 76 GiB of real weights stay resident, and the rest of the pool becomes KV cache: 995,129 tokens of it at his shipped default.
That figure is not loose. 123.57 GiB of checkpoint minus 47.68 GiB of table is 75.89 GiB, so his number reconstructs against the file listing exactly.
His measurements on the box, 40 prompts across 8 categories, thinking off, no prefix cache:
| Measure | Single Spark |
|---|---|
| Prose | 21.7 tok/s |
| Median across 40 prompts | 32.5 tok/s |
| Time to first token | 0.30 s |
| Cold prefill, 28K to 200K | 1,650 to 1,900 tok/s |
| KV pool at 262K context | 995,129 tokens |
MTP4 acceptance runs at 64%, about 3.56 tokens per step. Without it the floor is 15.4 tok/s on every category, so most of that throughput is speculative decoding rather than raw bandwidth.
Reading from disk does not show up as stutter. At six concurrent streams his p99 step latency sits within 10% of p50. Our own calculator has carried this table as separable since the Flash-Next launch, at about a 6% throughput cost, and his numbers sit comfortably with that.
A Second Publisher Made a Different Call, and It Is Twice as Fast
NVIDIA left the n-gram table at FP8. Mia’s AI Lab did not.
Mia-AiLab/Qwen3.8-Flash-Next-NVFP4 is 105.88 GB against NVIDIA’s 132.68, both read from their own file listings on 6 September 2026. Almost the whole 26.8 GB difference is that one table: their packed version runs about 27 GB where NVIDIA’s is 53.72.
On the same machine that changes the answer. Mia reports 46.3 tok/s on single-stream prose from one Spark under vLLM at TP=1, with the table still memory-mapped off the device, a 992,584 token KV pool, and 108 tok/s across four concurrent streams. That is more than double the 21.7 tok/s @Tech2Wild measured on the official checkpoint, on the same hardware.
Two publishers, one model, one format name, 27 GB and a 2x speed gap between them. Nothing on either file tells you which one you downloaded. The configs differ as well as the checkpoints, so read the gap as the two recipes rather than as a controlled test of the tensor plan alone.
What a Second Spark Actually Buys
Not the model. Context, and a little speed.
@Tech2Wild ran the same 40 prompts across two Sparks at TP2 and published both columns:
| Measure | One Spark | Two, TP2 |
|---|---|---|
| KV pool | 995K tokens | 5.87M tokens |
| Prose | 21.7 tok/s | 24.1 tok/s |
| Median reply | 32.5 tok/s | 35.8 tok/s |
| First token at 6 streams | 690 ms | 410 ms |
| Aggregate at 6 streams | baseline | +5% |
A 5.9x KV pool, and 11% more prose. His own verdict on his own numbers: single Spark stays the default.
@MiaAI_lab published a separate two-Spark vLLM recipe the same day, reporting 54 tok/s on a single stream, 207 tok/s across 8 concurrent streams, prefill around 2,900 tok/s and a 3.6M token KV cache. That is more than double @Tech2Wild’s prose figure on the same hardware count, and neither published a shared harness, so read the two as different serving configurations rather than a contradiction to settle.
What We Have Measured, and What We Have Not
Our speed corpus holds 24 Flash Next runs, every one attributed to whoever produced it. None of them is ours, and none is this checkpoint. The closest is Atlas (@0xBakeer) on 29 August, a single DGX Spark under vLLM at NVFP4, 508.5 tok/s prefill and 33.4 tok/s decode, on an earlier NVFP4 conversion.
Two that bracket the hardware question, both GGUF rather than NVFP4:
- @analogalok, RTX 4090 with 110 GB of DDR4, UD-Q4_K_XL, offloading experts with
-cmoe, measured from 80K to 250K context. - @superalesha, four RTX 3090s under vLLM with FP8 KV and experts held on GPU, 67 tok/s at 262K context.
You can filter all of them by card and quant in the speed checker.
Our Calculator and This Build
The calculator sizes formats that apply one rule to the whole model. This checkpoint applies two rules and an exemption, so a plain bits-per-weight estimate will not reproduce it.
Use 132.68 GB for the download, and 123.57 GiB if you intend to hold all of it in memory. Every published single-Spark run does not, and lands near 76 GiB resident with the PLE table on disk. The calculator gives you the KV cache on top of whichever you pick.
NVFP4 is a Blackwell format. A 3090, a 4090 and an A100 cannot execute it, whatever the file says. NVIDIA’s card lists vLLM as the supported runtime.
What To Do With This
- One DGX Spark: both recipes work with the table off the device, and both are public. Mia’s 105.88 GB build measured roughly twice the prose speed of NVIDIA’s.
- Two Sparks: you are buying a 5.9x KV pool, not the ability to run it.
- A big Blackwell card: plan around 132.68 GB, not the 90 GB that “4-bit” implies.
- Anything older than Blackwell: the format will not run. Take a GGUF, and our Flash Next VRAM page has those sizes. To check a card you already own, use the GPU checker.
FAQ
How big is Qwen 3.8 Flash Next NVFP4?
132.68 GB to download, 123.57 GiB if held entirely in memory. Qwen’s BF16 release is 360.00 GB, so the compression is 2.71x rather than the 4x the name suggests.
Does Qwen 3.8 Flash Next NVFP4 run on one DGX Spark?
Yes. Fully resident it does not fit, but the 47.68 GiB PLE n-gram table can stay on NVMe and be read sixteen rows at a time, which puts about 76 GiB in memory and leaves a 995,129 token KV pool.
What is model-fp8-mtp-ple.safetensors?
The per-layer embedding n-gram table and the MTP module at FP8, 53.72 GB, 40.5% of the download. NVIDIA copied it byte for byte from Qwen’s own FP8 checkpoint rather than quantizing it.
Is a second DGX Spark worth it?
For context, yes: the KV pool goes from 995K to 5.87M tokens. For speed, marginally, at 11% on prose. The author of both benchmarks keeps a single Spark as his default.
What hardware does NVFP4 need?
Blackwell or Blackwell Ultra. Earlier NVIDIA architectures have no FP4 execution units.
Is this NVIDIA’s own build?
Yes, nvidia/Qwen3.8-Flash-Next-NVFP4, published by NVIDIA on Hugging Face with a card dated 31 August 2026. It is safetensors for vLLM, not a GGUF.