How to Run DeepSeek V4 Flash on DGX Spark
One command, and it fits on a single 128 GB node.
curl -sSL https://raw.githubusercontent.com/entrpi/ds4-on-spark/main/install.sh | bash -s -- --start
That is the installer from Entrpi’s ds4-on-spark, an MIT-licensed Blackwell CUDA fork of the DwarfStar engine, built specifically for the GB10. It pulls the model, builds the engine and starts a server.
That command pipes a script straight into your shell, which is how the project publishes it. Read install.sh before you run it, as you should with any install one-liner. The repository is MIT and the script is short.
Before you run it, there is one thing worth understanding, because the number going around today does not mean what most people reading it assume.
What Actually Fits in 128 GB
A DGX Spark has 128 GB of unified LPDDR5X, about 119 GiB usable. There is no second memory pool to spill into.
The official DeepSeek-V4-Flash-0731 checkpoint is 166.9 GB. It does not fit in memory, and while weights can be streamed from disk, nothing makes that run at usable speed.
| Build | Size | Single Spark |
|---|---|---|
| DwarfStar asymmetric quant (what the installer pulls) | ~81 GiB | yes |
| UD-IQ2_XXS | 90.9 GB | yes |
| UD-Q2_K_XL | 96.8 GB | yes |
| REAP-pruned build | 107.8 GB | yes |
| UD-IQ3_S | 116.1 GB | tight |
| UD-Q3_K_M | 129.3 GB | no |
| UD-Q4_K_XL | 155.1 GB | no |
| Official FP8 checkpoint | 166.9 GB | no |
So when you see “the full official model on ONE Spark”, the model being served is a 2-bit class quantization, not the released checkpoint.
The build the installer uses is smarter than a flat 2-bit, which is why it holds up. Per the repository: IQ2_XXS for routed-expert gate and up projections, Q2_K for routed-expert down, and Q8_0 for everything dense. The experts, which are the bulk of the 284B parameters, take the aggressive quantization. The dense layers that matter most for coherence stay at 8-bit. A ~6.5 GiB DSpark drafter ships alongside it for speculative decoding.
You also need at least 110 GiB of free disk for the GGUF files.
What the 59 tok/s Figure Actually Measures
This is the number spreading today, and it is real. It is also being read wrong.
From the repository’s own benchmarks on a GB10:
| Measurement | Figure |
|---|---|
| Prefill | ~960 tok/s at 2K, ~1,010 at 12K, ~933 at 64K |
| Plain decode, single stream | 20.0 tok/s at 2K, 17.7 at 48K |
| Decode with DSpark speculation | 27.7 tok/s mean, 1.38x |
| 59 tok/s | aggregate across 12 concurrent requests |
Mia of MiaAI-Lab, who published the single-node recipe, later posted the full concurrency curve. It is the clearest thing anyone has shared about this model:
| Concurrent sessions | 1 | 2 | 4 | 6 | 8 | 12 |
|---|---|---|---|---|---|---|
| Aggregate tok/s | 26.7 | 32.9 | 46.5 | 54.1 | 58.5 | 58.5 |
| Per stream | 26.7 | 16.5 | 12.1 | 9.5 | 7.7 | 5.2 |
Read the bottom row. The 59 tok/s figure is twelve people each getting 5.2 tok/s. Aggregate throughput rises because you are serving more requests, and it stops rising at eight. What each person experiences falls the whole way.
Mia also reports that this single-node build beats the FP8 version running on two DGX Sparks under vLLM on agentic workflows, which is not the result anyone expected, and credits @bleysg for the work. Her start and stop recipe is at MiaAI-Lab/DeepSeek-v4-Flash-One-DGX-Spark.
The 1,000 tok/s is prefill, which is the model reading your prompt, not writing an answer. The 59 tok/s is aggregate throughput across twelve simultaneous requests, which is a serving metric.
If you sit down and ask it one question, the repository’s own figure is 20 tok/s, or 27.7 with speculative decoding on.
That is not a criticism of the work. Entrpi reports a 2.43x to 3.30x prefill improvement over upstream, though note that is his fork measured against the project it forked, by the person who wrote it. It is a warning about what you will actually experience. Both numbers are correct. They answer different questions, and only one of them is the one most readers are asking.
The Benchmarks and These Speeds Do Not Combine
Worth knowing before you size anything around the published scores. As @BosonJoe points out, DeepSeek ran its benchmark suite at the max reasoning effort level, which their model card states directly.
Max reasoning spends 2 to 10 times the output tokens, at a slightly lower draft acceptance rate. He reports that working out to 5 to 10 times the wall clock on real tasks: 25 to 50 minutes where a lower effort level takes 5.
Put that next to 5.2 tok/s per stream and the arithmetic gets uncomfortable. The scores that make this model interesting were produced in a mode that multiplies the token count, on hardware that generates them slowly. Running it at a lower reasoning effort is entirely reasonable, but you are then not running the configuration the benchmarks describe.
The same pattern showed up on the Kimi K3 local builds, where headline figures travelled much further than the conditions attached to them.
Prerequisites
From the repository, verbatim:
- NVIDIA DGX Spark (GB10, SM121, 128 GB LPDDR5X unified)
- aarch64 Linux (Grace)
- CUDA 13.x, tested on 13.0.88
- 110 GiB or more free disk
Datacenter B200 and B300 are sm_100 and are noted as untested. The Metal backend is unaffected by this fork.
If you are going the two big cards route instead, @light_foundry published a full recipe for 2x RTX PRO 6000 Blackwell and flags the trap: the official FP8 checkpoint fits 2×96 GB with no requantization, but current upstream vLLM and SGLang do not support this model. You need a community build made after 31 July, because that release changed the MTP norm layout and older builds break on it.
Running It
Once installed, the server commands are:
ds4-serve
Defaults to 32,768 context on 127.0.0.1:8000. For more context, and to reach it from another machine on your network:
ds4-serve -c 69632 --host 0.0.0.0
It exposes an OpenAI-compatible endpoint, so anything that speaks that API will connect to it.
The Engine Underneath
ds4-on-spark is a fork of DwarfStar, Salvatore Sanfilippo’s native inference engine written specifically for DeepSeek V4. It is not a llama.cpp wrapper, though it builds on GGML, and it targets this model family first rather than as one of hundreds.
That matters for a practical reason: it also runs on Metal and ROCm, so the same engine covers Mac, DGX Spark and Strix Halo. Be careful reading the table below, though. Only the Spark and Mac rows share this engine. The multi-GPU rows run vLLM, at different quantizations and different concurrency levels, so they are not like-for-like.
make cuda-spark # Linux CUDA, DGX Spark
make # macOS Metal
make strix-halo # Linux ROCm, AMD Strix Halo
How the DGX Spark Compares
Every figure below is attributed to whoever measured it. We do not own any of this hardware and have run none of these tests.
| Hardware | Decode speed | Conditions | Measured by |
|---|---|---|---|
| 2x RTX PRO 6000 Blackwell | 243 tok/s single stream, 403 at 4 concurrent | official FP8, TP 2, DSpark, 524K max length, 74 to 76% draft accept | @light_foundry |
| 2x DGX Spark | 85.9 aggregate over 6 concurrent sessions, 6,925 tokens in 80.65s | OMP with vLLM, TP 2 | @aijoey |
| 2x DGX Spark | ~40 single stream, ~92 aggregate at 8 concurrent | vLLM TP 2, FP8 KV, MTP, 78% draft accept | tonyd2wild |
| 1x DGX Spark | 20 plain, 27.7 with DSpark, 59 aggregate at 12 concurrent | ~81 GiB asymmetric quant | Entrpi |
| MacBook Pro M5 Max 128 GB | 33.7 to 39 tok/s | DwarfStar on Metal, the DS4 author’s official 0731 asymmetric build, ~91 GB | @sm_hn and @ksuniri, independently |
| Mac Studio M3 Ultra | 27 tok/s | q2 to q4 via DwarfStar | @ivanfioravanti |
| MacBook Pro M3 Max 128 GB | 26.68 generation, 58.52 prefill | q2, short prompts | DwarfStar repository |
Two RTX PRO 6000 Blackwell cards are in a different class entirely at 243 tok/s, because they run the official FP8 checkpoint rather than a 2-bit quant, and they have the memory to do it.
The rest of that table mixes engines, quantizations and concurrency levels, so it shows the range rather than a fair fight. For that, there is one better test.
The One Controlled Comparison Anyone Has Published
@stevibe benchmarked the same quantization, Unsloth’s IQ3-XXS, across three setups and posted video of it. He picked that quant because it is the largest that fits a 128 GB machine, which makes it the natural choice for a Spark.
| Setup | Decode | Time to first token |
|---|---|---|
| DGX Spark | 19.09 tok/s | 531.64 ms |
| 6x RTX 4090 | 51.74 tok/s | 194.72 ms |
| 4x RTX 5090 | 62.32 tok/s | 200.40 ms |
His stated caveat, and it is the right one: these are vanilla numbers with no MTP, no DSpark and no speculative decoding of any kind, so he calls it bottom-line worst-case performance for these machines. He also notes a single RTX PRO 6000 at 96 GB just misses the 97 GB this quant needs, so it sits the test out.
Two things follow from it.
First, his 19.09 tok/s on a Spark lands within 1 tok/s of Entrpi’s 20.0, despite a different quantization and a different engine. Two independent measurements converging is much stronger evidence than either alone, and it confirms what a single Spark actually generates.
Second, the Mac is ahead of the Spark on generation, not level with it. Ivan Fioravanti reports 27 tok/s on an M3 Ultra against roughly 19 to 20 on a Spark. The Spark’s real advantage is prefill, near 1,000 tok/s against the Mac’s much lower figure, which is why it feels faster on long prompts and agentic work despite generating fewer tokens per second.
And a laptop has since gone past both. Two people running a MacBook Pro M5 Max with 128 GB, the same memory as a Spark, report 33.7 to 39 tok/s: @sm_hn at 35 to 39, and @ksuniri at 33.7 on code and 36.2 on long context. Both switched to the DS4 author’s official 0731 asymmetric build on Metal after llama.cpp underperformed, which is close to the same quantization the Spark installer pulls. On the nearest like-for-like pair available, 26.7 on a Spark against 33.7 on the Mac, the laptop is ahead.
If none of this is within reach, renting is the cheaper way to find out whether the model earns the hardware, and our full requirements breakdown covers what it needs at every quantization. If you are tuning a llama.cpp build instead, our guide to its offload flags covers the settings that matter on constrained memory.
Should You Buy a Spark For This
Two honest observations, neither of which is advice.
A single Spark runs a 2-bit build. That is a real quality compromise on a model whose whole appeal is frontier-class agentic performance. Nobody has published a benchmark comparison of this specific asymmetric quantization against the FP8 original, so the size of that compromise is currently unmeasured.
Two Sparks change the calculation more than the tok/s suggests. At 2 nodes you clear 155 GB and can run the 4-bit build, which is a different model in quality terms. Most of the published recipes target two nodes for exactly that reason.
For sizing any other configuration, the VRAM calculator covers 69 models across 75 GPUs, and the GPU-first tool answers it from the hardware side.
DSpark Just Got Faster, And It Costs You 10 GB
On 6 August, Unsloth enabled DSpark for the V4-Flash-0731 GGUFs and put the change in their own changelog: 1.5x to 1.9x faster inference, automatically on, with llama.cpp integration landed. That is a real improvement and it applies to the builds people here actually download.
The number travelling with the announcement is 120 tokens per second. Before anyone buys hardware on it, read where it comes from. Unsloth’s documentation:
DSpark enables DeepSeek-V4-Flash to reach 120 tokens/s on a B200 GPU compared to the original 60 tokens/s baseline.
A B200. That is a datacenter card costing more than a car, not the box this page is about. The figures above for a GB10 stand: 20.0 tok/s single stream, 27.7 with speculation. Nothing in the announcement changes them, and the GGUF speedup is quoted as 1.5x to 1.9x rather than 2x.
The part that matters on a 128 GB box
Speed is not what decides whether this runs on your machine. Memory is, and DSpark changes it. Unsloth publishes both columns, and the gap is consistent:
| Quantization | Standard | With DSpark | Fits 128 GB with DSpark? |
|---|---|---|---|
| 1-bit | 92 GB | 102 GB | Yes |
| 2-bit | 102 GB | 112 GB | Yes, tight |
| 3-bit | 110 to 135 GB | 120 to 145 GB | Mostly no |
| 4-bit, near lossless | 162 GB | 172 GB | No |
| Q8_K_XL, lossless | 169 GB | 179 GB | No |
Their own guidance is to plan for roughly 10 GB of additional memory with DSpark on. Read down the 3-bit row and you can see what that does here. Standard 3-bit starts at 110 GB and is already marginal on a machine whose 128 GB is shared with the operating system. Add DSpark and it starts at 120 GB. Usable memory on a Spark is not a fixed number: it moves with how far you tune the operating system, from the roughly 112 GB Simon Willison reported hands-on up to the practical ceiling of about 119 GiB noted earlier. Take the ceiling and be generous about it. 3-bit with DSpark still misses by roughly a gigabyte, and that already assumes you tuned everything.
So on this hardware the speedup is not free, and at 3-bit it is not available. You choose: 3-bit standard without DSpark, or 2-bit with DSpark and a quality step down.
That trade does not exist on a B200 with 192 GB, which is where the 120 tok/s was measured. It is specific to a box this size, which is why it is worth writing down.
FAQ
Can you run DeepSeek V4 Flash on one DGX Spark?
Yes, using a 2-bit class quantization. The official 166.9 GB checkpoint does not fit in 128 GB of unified memory, but Entrpi’s ds4-on-spark installer pulls a roughly 81 GiB asymmetric build that does, with IQ2_XXS and Q2_K on the routed experts and Q8_0 on the dense layers.
How fast is DeepSeek V4 Flash on a DGX Spark?
About 20 tok/s single stream, or 27.7 with DSpark speculative decoding, according to the repository’s published benchmarks. Prefill is roughly 1,000 tok/s. The 59 tok/s figure circulating is aggregate throughput across twelve concurrent requests, not what one person asking one question will see.
How much memory does DeepSeek V4 Flash need on a DGX Spark?
The build the installer uses is about 81 GiB, and the repository states that 128 GB, roughly 119 GiB usable, is enough for the model plus about 250 MB of KV cache at 16K context. You also need at least 110 GiB of free disk for the GGUF files.
Is a DGX Spark faster than a Mac Studio for DeepSeek V4 Flash?
Not for generation. @stevibe measured 19.09 tok/s on a Spark and Entrpi’s repository reports 20.0, while Ivan Fioravanti reports 27 tok/s on an M3 Ultra running DwarfStar. The Spark is substantially faster at prefill, near 1,000 tok/s per Entrpi’s benchmarks, so it handles long prompts and agentic workloads better despite the lower generation rate. None of these figures are ours.
What do I need before installing?
A DGX Spark with GB10, aarch64 Linux, CUDA 13.x, and at least 110 GiB of free disk. The repository notes it was tested on CUDA 13.0.88 and that datacenter B200 and B300 are untested.
Do I need two DGX Sparks?
Not to run it. Two nodes let you clear 155 GB and run the 4-bit build instead of a 2-bit one, which is a meaningful quality difference, and most published recipes target two nodes for that reason.