GLM 5.3 Flash VRAM: 320B With a 1M Window, on Two Sparks

GLM 5.3 Flash VRAM Requirements: The Short Answer

Running Z.ai’s newest frontier MoE takes 191 GiB at Q4_K_M, 141 GiB at Q3_K_M, and still 104 GiB at the bottom of the quant ladder. No gaming card on the market holds any of those numbers. Most measured runs are on two DGX Sparks, some of them in our tokens per second tool, and one RTX 3090 now runs it with the experts in system RAM. The surprise sits in the second column: taking context from 8K tokens to the full 1,048,576 costs 15 GiB, total, on a 320 billion parameter model: 14 GiB of cache and about 1 GiB of llama.cpp’s own overhead.

Here is the whole table, from our calculator, llama.cpp backend, weights plus cache plus overhead. These are capacity figures, not speed promises.

Read the columns, not the rows. A normal model’s memory fan-outs as context grows; this one barely breathes. From 8K to a million tokens, every tier moves by roughly 15 GiB, less than many 24 GB models move crossing 128K. The rest of this piece explains where the 14 GiB of cache inside that comes from, what hardware clears the bar, and why a same-day Qwen release gives the same 128 GB machine a completely different answer.

The Stealth Week, and Our Receipt

For seven days before the announcement, a model called Ox Alpha served on OpenCode under no name and no lab. Mid-session on 21 August, its thinking and output flipped to Chinese characters for a few seconds, then back. The next day we posted the screenshot with the conclusion the behaviour pointed to:

Many are speculating about the origin of Ox Alpha. Mid-session yesterday, the thinking and the output automatically switched to Chinese characters for a few seconds before returning to English. The thinking process and the work are very similar to GLM’s. x.com/VRAMCalculator

Four days later Z.ai confirmed it from the other side, pinning that GLM 5.3 Flash was “previously previewed as Ox Alpha, running entirely on Chinese AI chips“. OpenCode followed, moving the renamed model onto OpenCode Go. Behaviour gave the lab away before anyone official did. We mention it once, because it is ours, and get back to the hardware.

Why a Million Tokens Only Costs 14 GiB

The spec sheet explains the curve. Every figure below is from zai-org/GLM-5.3-Flash and its config.json, read the day the weights landed.

Thirty four of the forty five layers hold a fixed size state, like shelving rented by the unit rather than by the box stacked on it. Rent the shelf once and the bill never changes, however full your warehouse gets. Only the 11 sparse MLA layers keep a cache that grows with tokens, and theirs is compressed to a 512 wide latent per position instead of a full key and value copy. Multiply that compression across eleven layers and a million positions and you get the 14 GiB.

The pattern holds in the table. Q4_K_M goes 190.7, 191.1, 194.4, 205.7 across the context ladder. The jump from 8K to 1M is 15.0 GiB: 14 GiB of cache, plus about 1 GiB of llama.cpp’s own overhead at 1 KiB per token, which we measured on an A100 in September. At Q8_0 it is also 15.0. Q3_K_M spreads 141.2 to 156.2, again 15.0. The context surcharge does not care which quant you pick, because at these settings the cache is not what gets quantized.

Z.ai frames the design the same way in its own card: “for the first time in the GLM series, we introduce a hybrid architecture combining sparse and linear attention, sharply reducing long-context serving costs while preserving precise long-context capabilities.”

What Hardware Actually Clears the Floor

Start at the top of the consumer market. An RTX 5090 carries 32 GB. The cheapest entry in the table needs three times that at minimum context, so the entire gaming stack, 8 GB through 32 GB, is out before the discussion starts.

Unified memory changes the conversation, which is why the next section matters. Between a Spark and a rack sit the large-memory workstations and Mac Studios, and the honest read of the table is:

• Under 120 GiB usable: Q2_K fits on paper at short context and climbs to 119.3 at the full million. Whether 2-bit quality on an MoE this sparse holds up is the thing to check before buying for it.
• Around 191 to 206 GiB usable: Q4_K_M through the full 1M window, with margin at everyday contexts.
• 340 GiB or more: Q8_0 anywhere on the ladder.

Day-one runtime support is real rather than aspirational: the card lists SGLang, vLLM, TokenSpeed and KTransformers.

Same-Day Releases, Opposite Answers on the Same Box

The most useful comparison landed the same day the weights did. Alibaba opened Qwen3.8-Flash-Next, 180B at 3.3% active, and our requirements page for it records Q4 at 256K context at 110 GB, sized for one DGX Spark.

GLM 5.3 Flash does not fit it, and the table says so precisely: Q4_K_M starts at 190.7 GiB against 128 GB of unified memory, and even Q3_K_M at 141.2 GiB is over the line. Two frontier-class MoEs, launched the same day, straddling the exact machine enthusiasts actually bought this year. Capacity math is the only fair referee here, because the same five GPUs face both models and give opposite verdicts. If you sized a purchase around Flash-Next, GLM 5.3 Flash resets that arithmetic.

The Benchmarks, Priced in Hardware

Z.ai published its launch numbers itself, as evaluation results on the model repo: Terminal-Bench 2.1 at 84.3, DeepSWE at 63.4, HLE at 55.3. Around them the card supplies the headline in Z.ai’s own words: “approaching Claude Opus 4.8 on coding and agentic benchmarks”. Vendor figures, all of them.

Follow those numbers to the desk and they stop being abstract. A benchmark tier like that only becomes real if you can host the model at a quant worth running and a context worth filling, and our table prices that hosting at 190.7 GiB for Q4_K_M, rising to 205.7 GiB across the full million-token window. One Spark holds neither figure. A second Spark gets the pool past the number, at doubled complexity, and a large workstation does it in one unit. On paper the model plays in frontier company; on our table, so does the hardware bill.

The same week supplies its own counterweight. Qwen3.8-Flash-Next ranked ahead of Claude Opus 4.6 in Alibaba’s launch material and still hosts Q4 at 256K on one box, 110 GB per our requirements page. So the reader’s decision this week is not which scorecard wins. It is which set of trade-offs fits the memory already on the desk, and owning the weights keeps the usual levers open either way: a lower rung on the ladder, a trimmed cache, tuned flags. The table at the top prices every one of those steps.

Total Parameters Buy the Card. Active Parameters Buy the Speed

Three releases inside one week make the point better than any explainer we could write, and our MoE conversion piece made it first: Qwen3.8-27B Whittle runs 66% of parameters per token, GLM 5.3 Flash runs 5.6%, Flash-Next runs 3.3%. Total size determines whether the model fits your memory at all. Active size drives how fast it generates once resident.

So a reader who hears “320B impossible at home” and “only 18B, trivially light” is being sold two halves of one wrong picture. The card wants the whole file. Your tok/s cares about the slice. Speed on specific home hardware will be somebody else’s measurement before it is ours, and when those numbers surface they land on this page the way every other launch page here handled it.

What Changed on 30 September

Upstream llama.cpp runs GLM 5.3 Flash from build b11279, published on 30 September (release notes). Its multi-token prediction was left out of that first version, so drafting still needs a fork or another engine.

Two DGX Sparks got much faster in a week. @Tech2Wild’s new two-Spark recipe decodes prose at 35.3 tok/s, up from 18.1, and code at 68.9, up from 44.7, with NVIDIA’s NVFP4 weights, a DFlash 2 draft and 262K of context (runbook). On the same pair of Sparks, @jayleaton’s final TensorFold build reaches 51.6 tok/s on chat and 89.6 on code, against 22.8 and 41.9 for vLLM, with identical output (his post).

A correction to our own figures. The sizes on this page were estimates until Unsloth published its files. On 30 September we replaced them with the real file sizes, which run 3 to 13 GB larger below Q8_0, and the table now includes llama.cpp’s 1 KiB per token of overhead. Q4_K_M moved from 185.9 to 190.7 GiB at 8K; the conclusions about which machines hold it do not change.

New Runs, 1 October

Two DGX Sparks, five builds in a week. Each tester used their own prompts, so compare each row with its own baseline rather than across rows:

@MiaAI_lab’s recipe (post) reads prompts at about 1,950 tok/s and keeps a cache pool of 2.7M tokens across the two boxes. @WescheNex1q’s resent 64K prompt returned its first token in 0.27 seconds (post). TensorFold 0.6.0, released on 30 September, also makes the GLM draft model read only its sliding window: 8% faster at 52K and 3.8 GiB lighter on each Spark.

One RTX 3090, with the experts in system RAM. @0xSero runs GLM 5.3 Flash on one 24 GB RTX 3090 through a layer of their own over exllamav3 (repository). The model is a 3.05-bit EXL3 build of 125.3 GB. Every routed expert lives in pinned system RAM, the card caches the busiest, and the CPU computes the coldest misses. The README gives two modes, measured on an AMD EPYC 7443P with eight-channel DDR4:

Both modes hold a 131,072-token cache. Stock exllamav3, with the experts it cannot fit on the card running on its CPU worker, decodes the same model at 7.5 to 8.4 tok/s on the same machine. Two details matter before you plan a build around the headline. The posts name a 3090 with 196 GB of DDR4, while the README’s fast mode lists about 218 GiB of system RAM, which is 234 GB. And the README itself says the fast mode’s decode is not exact yet, with a fix in progress. The 28 tok/s is real and measured; the RAM bill behind it is larger than the post suggests. This is the same move Strata makes for Qwen3.8 Flash Next, and like Strata it sits outside what our calculator models, which sizes the model fully on the card.

Four CMP 170HX mining cards. @Morrowmake’s recipe (repository) serves GLM 5.3 Flash on four NVIDIA CMP 170HX cards through their own fork of vLLM. Upstream vLLM runs the model’s sparse attention only on Hopper; the fork adds Ampere kernels for it, then tunes them. The weights are canada-quant’s W4A16 build, the cache stays at full precision, nothing is offloaded, and the context is 262,144 tokens. Release 1.6.0, out on 1 October, lists these figures at 180 W per card, all with the DFlash 2 drafter:

An owner reproduced the default numbers the same day. @seanphan, on the recipe in a Supermicro 4028 server, measured 396.0, 306.2 and 192.5 tok/s for one user (structured, code, prose) and 798.6 across eight (post).

Read the requirements before pricing the cards. The recipe wants each card to expose 64 GiB and, for the tensor-parallel layout, PCIe Gen2 x16 links. Stock cards expose less memory and run at x4, and the recipe states that getting to 64 GiB and x16 is outside its scope. On x4 links the tensor-parallel layout is bus-bound and much slower; the pipeline layout is the one built for them, and its figures above were measured on x16 links. The default drafter carries a non-commercial licence (CC BY-NC-ND 4.0).

Four Sparks and a Mac Studio, 2 October

Four DGX Sparks. Two recipes now publish full tables for NVIDIA’s NVFP4 checkpoint on four GB10 boxes, both with the DFlash 2 drafter. Kindling AI’s (repository), built on ASUS GX10 boxes, also runs on two, three or six:

The model takes all of each box: about 91.9 GiB of GPU memory per box, with 1.5 to 3 GiB left free, so the boxes run without a desktop session. They talk over ConnectX-7 through a 200G switch, or cabled in a loop with no switch at all, which another user measured at 100.7 / 59.9 / 146.7 tok/s on four. One of its speed-ups, FP8 and NVFP4 copies of the layers the checkpoint leaves in 16 bits, is the only change the repository measures an accuracy cost for: about 1% in NLL on prose.

@Tech2Wild’s four-Spark recipe (repository) moved on 29 September to knapcio’s stack, run unmodified, with a 500,000-token window on a cache pool of 3,453,703 tokens. One stream writes 120.5 tok/s on code and 61.0 on prose; prose across 16 streams went from 158 to 215 tok/s in total, and cold prompts read at 2,607 to 2,786 tok/s from 8K to 64K. The two repositories use different prompts and harnesses, so read each against its own baseline rather than against the other.

One Mac Studio. keys (@u1tra_instinct) measured antirez’s Q4_K GGUF, 177.8 GiB, on a Mac Studio M5 Ultra with 256 GB, served by a development build of mlx-serve through its bundled ds4 engine, thinking on (repository). Decode runs 26 to 30 tok/s from 2K to 33K of context and 19 at 256K, with a headline of 35.8. Prompts read at 370 to 470 tok/s at every length, so a full 256K prompt takes about 11.5 minutes before the first token, and the engine serves one request at a time. One setting matters: left to size the context itself, mlx-serve picked 13,312 tokens for this file, and an explicit --ctx-size 262144 loaded the full window with memory steady at about 178 GB.

On a Mac, the engine changes the answer more than the file does. With TensorFold, @volatilemarkts’ 512 GB M3 Ultra went from 27 tok/s on MLX to 61 on TensorFold 0.6.2, with the first token in 0.20 s instead of 0.46 and eight requests at once at 78 tok/s in total, every answer identical to its solo run (post), and on an M5 Ultra at 32K @cryptosibbers measured oMLX at 56 tok/s and TensorFold at 79, the same model and question.

An Independent Check, and a Third Spark, 3 October

Mia’s two-Spark kit, re-run by someone else. @MiaAI_lab’s TensorFold kit serves GLM 5.3 Flash from her own 4-bit EXL3 build, 175.7 GB, with an FP8 cache pool of about 2.9 million tokens and a 1,048,576-token window (repository). @yume_arasaki ran his own test grid on it, on his own two Sparks with clocks locked, thinking off and temperature 0, and his numbers land on hers (post):

The kit serves four requests at a time by default; in the re-run, eight sent at once queued and completed at 67.5 tok/s in total. Version 1.5 adds an option to serve eight together: 130.8 tok/s on prose in total, 17.0 for each, with the cache pool shrinking to about 1.5 million tokens to make room, which is why two Sparks keep four by default.

The long-context part is the news. In the same re-run, writing slowed only from 54.9 tok/s at 8K to 44.6 at 874K tokens, 81% of the short-context rate with 83% of the window full. Reading prompts held at 1,888 tok/s at 32K, 1,340 at 524K and 1,100 at 874K, and a test fact hidden at six depths was found 18 times out of 18 through 879K tokens. A million-token window that slows this little and still recalls what is in it is what long agent sessions need.

A third Spark. Version 1.4 of the kit adds a three-Spark layout. Her repository gives 77.6 tok/s on prose for one request, 146.2 for four requests in total (37.3 each) and 2,064 tok/s reading a 32K prompt, against 60.4, 108.8 and 1,978.9 on two Sparks (post). Those are her measurements; the independent re-run above covers the two-Spark layout only.

Two workstation cards for scale. @krunkosaurus measured 264 to 299 tok/s across 6K to 480K of context on two RTX PRO 6000 Blackwell cards, 96 GB each, and about 50 tok/s on two Sparks, which he sums up as about five times faster (post). The two runs used different builds, mixed NVFP4 and FP8 on the RTX PRO 6000s and Mia’s 4-bit EXL3 on the Sparks, so read the ratio as a rough one. What the Sparks keep is the window: a million tokens in 256 GB of unified memory, at a fraction of the power.

Two RTX PRO 6000s in One PC, and Updates to the Spark Kit, 5 October

A one-command recipe for two workstation cards. Mia’s lab and Marc Seal of Aevonix Research published a TensorFold recipe that serves this model from one Linux PC with two 96 GB RTX PRO 6000 Blackwell cards, each at 300 W, with four requests at a time and up to 524,288 tokens per request (recipe page). To fit the model, the draft model and the cache on two cards, it converts the dense layers to 4 bits at load time, which the page says costs some fidelity against the four-card recipe’s FP8 dense layers; the experts are Mia’s 4-bit EXL3 build and the cache is FP8. Their measurements, taken on 4 October with greedy decoding and thinking off:

Prompts read at 5,729 tok/s at 8K and 6,908 at 64K, and a 262,165-token prompt took 41 seconds. When a conversation of about 130K tokens continues, the next turn starts in about half a second instead of the 26 seconds a fresh read takes. These sit above the 264 to 299 tok/s @krunkosaurus measured on two of these cards with a different build, above; the method differs, so compare the shapes rather than the exact figures.

The two-Spark kit, updated. Mia’s kit now has an abliterated variant behind a flag, with different weights, gated behind its terms, and about 6.5% slower by her account; the next turn of a chat now starts in about 1 second instead of about 16 (post).

If your two Sparks feel slow, check the link first. Two Sparks connected directly should measure close to 180 to 190 Gbps. @MehdiHacks measured 26 Gbps until he updated Linux and the firmware on both machines (apt, then fwupdmgr) and rebooted them, after which it read 185 (post).

One Card and Its RAM, or One Mac, 7 October

An RTX 3090 with 50 GB of RAM. @0xSero runs the EXL3 build on one RTX 3090 with 50 GB of DDR4, reading the experts that do not fit from 100 GB of NVMe. After a friend’s optimisation work, writing went from 26 to 35 tok/s and prompt reading from 1,018 to 1,136 tok/s, and four requests at once reach 76.5 tok/s combined (his post). He does not name the build. His earlier offload build, in the FAQ below, needed about 218 GiB of system RAM for about 28 tok/s.

A 2017 Tesla V100. @PeasantSmith runs Unsloth’s IQ1_S file, 93 GB, on one Tesla V100 32 GB with 64 GB of DDR5-5200, an i5-12600T and a Gen4 NVMe drive, through his own build of Strata, and reports 25 tok/s at peak (his post). That is a peak rather than a sustained figure, and he expects newer cards or more RAM to do better.

A 128 GB Mac. Beamsters’ Sushi engine, version 1.2.0, ships a 2.4 bits per weight build of 89 GiB that fits a 128 GB Mac, with a 1M context. His launch post gives about 850 tok/s prompt reading and 40 to 50 tok/s writing on an M5 Max with DFlash 2, a KL divergence of about 0.074 and 92 of 100 on tool-eval-bench (launch post); the table in the project’s README lists about 860 and 55 on an M5 Max with 128 GB. Those are the author’s own figures.

The RTX 3090 and the V100 keep the model at its own quant and move the experts to system RAM or the SSD rather than shrinking it. The Mac simply fits: 89 GiB in 128 GB. Our Project Maya page follows the 3090’s idea on two V100s.

Two Four-Spark Builds, and Unsloth’s GGUFs, 12 October

llama.cpp’s official support for the architecture landed with PR 27773, merged on 30 September and shipped in build b11279 the same day. Unsloth’s GGUF files had stopped loading in the meantime, and the repository, BF16 across 14 parts plus quantized tiers, is updated again. @kis says the model runs for him again after a while (his post). Multi-token prediction is still out of the merged tree. The PR that adds it, 27917, is open as of 12 October.

Four Sparks, two engines. @0xSero serves an EXL3 build, 4 bits per weight, across four DGX Sparks. His post puts prose at 104 tok/s and code at 152 with a 6 to 8 million token cache pool, and the monitor table he attaches gives 101.7 tok/s for a single prose request and 288.4 tok/s across 16 concurrent ones, with a 6.21M token fp8 pool (his post).

@Tech2Wild compared TensorFold against his own vLLM TP4 stack on four Sparks, same harness. At a single request his chart gives 127 tok/s on code against 120, 133 on JSON against 120 and 93 on tool calls against 68, with vLLM ahead on math, 142 to 121, and on counting to 100, 182 to 180, and TensorFold reads a 500,000-token prompt in 191 seconds against vLLM’s 202 (his post).

Checking Your Own Machine

Select glm53-flash in the calculator to reproduce every figure in the table at your own context length and see the component split: weights, cache, runtime allowance. Working backwards from a card you own, the GPU-first tool lists everything in the catalogue your hardware holds today, including the two Flash siblings on either side of this one. The gap between them, 110 GB versus 191 GiB at Q4, is currently the widest spread between two same-week releases we have measured. For the offload path, the Maya calculator checks one card, or two, with the experts held in RAM.

FAQ

How much VRAM does GLM 5.3 Flash need?

At least 104.3 GiB at Q2_K with 8K context, 190.7 GiB at the more practical Q4_K_M, and 325.0 GiB if you want near-lossless Q8_0. Those totals include weights, KV cache and a runtime allowance, computed with the llama.cpp profile in our calculator.

Can a DGX Spark run GLM 5.3 Flash?

No, not at 3-bit precision or better. The Spark offers 128 GB of unified memory, about 119 GiB usable, while Q3_K_M alone needs 141.2 GiB. Qwen3.8-Flash-Next, launched the same day, does fit that same box at Q4. Two Sparks do run GLM 5.3 Flash: this week’s builds decode it at 35 to 60 tok/s on prose and 69 to 90 on code.

Can GLM 5.3 Flash run on one RTX 3090?

Not on the card alone, but with the experts in system RAM, yes. @0xSero’s offload build decodes it at about 28 tok/s with about 218 GiB of system RAM, or 12.6 tok/s output-exact with about 111 GiB, at a 131K cache. The card holds only the busiest experts. On 7 October he reported 35 tok/s with 50 GB of RAM and 100 GB of NVMe.

Why does the 1M context only add 14 GiB?

Because 34 of the model’s 45 layers hold a fixed size state that never grows, and only the 11 sparse MLA layers store a per-token cache, compressed to a 512 wide latent. Eleven cheap caches times a million tokens equals roughly 14 GiB, and llama.cpp adds about 1 GiB of its own on top.

Is the context cost the same at every quant level?

In this table, effectively yes: 15.0 GiB at every level, 14 of it cache and 1 of llama.cpp overhead. The cache is kept at full width while the weights carry the quantization.

What was Ox Alpha?

The working name under which GLM 5.3 Flash ran anonymously on OpenCode for about a week. Z.ai’s launch post confirms the lineage, adding that the preview ran entirely on Chinese AI chips.

Which runtimes support GLM 5.3 Flash on day one?

SGLang, vLLM, TokenSpeed and KTransformers, per the official model card, each with published recipes or cookbooks.