We Said Kimi K3 Was Datacenter-Only. That Changed in Three Days.
On July 27 we published what Kimi K3 requires and wrote that it was enterprise hardware regardless of quantization. Our calculator put full residency at 1,796 GB at Q4_K_M, which is five Instinct MI430X cards at 98% of their pool.
Two days later Unsloth published a dynamic 1-bit build at 594 GB, down 62% from the 1.56 TB original, and people started running it.
So the “regardless of quantization” part was wrong. Here is what replaced it, and what the headlines left out.
What Actually Shipped
Unsloth’s quantization ladder for K3, with their own quality figures:
| Build | Size | Quality |
|---|---|---|
| UD-IQ1_S | 594 GB | ~78.9% top-1 accuracy, perplexity 2.5789. |
| UD-IQ1_M | 649 GB | |
| UD-IQ2_XXS | 711 GB | ~90% accuracy. |
| UD-Q2_K_L | 861 GB | |
| UD-Q4_K_XL | 1.51 TB | |
| UD-Q8_K_XL | 1.56 TB | lossless, perplexity 1.4581. |
The 1-bit build is the one that travelled. It keeps roughly four fifths of the model’s top-1 accuracy at 38% of the size, which is a genuinely impressive piece of engineering and not something to be sniffy about.
There is public proof it runs. One developer posted video of K3 in the UD-IQ1_S build generating code on four B200 cards, and it works.
The Number That Did Not Travel
Here is where it gets interesting, and it is not a case of anyone hiding anything.
Unsloth published a speed figure. Their documentation states you get “~20 tokens/s generation when using B200s and >120 tokens/s throughput.”
But the HuggingFace model card carries no speed figures at all. It has reasoning, coding, agentic and vision benchmarks, and zero inference speed.
And the post that reached a million views carries no speed either. It gives the size, the accuracy, and the hardware, which are the three facts that make it sound accessible, and stops there.
A separate figure of 36 tokens per second on four B200s is circulating, attributed to a demo where several models were given the same prompt. We have not been able to confirm that number on any Unsloth page we can read directly, so we are reporting it as a claim rather than a fact, and we are still looking. If we find it at source we will update this page and say so.
There is one more measured result, and it took searching the right place to find it. VESSL AI published throughput on 16 H200 cards: 327.5 tokens per second, up 17.9% from 277.7, with time to first token falling from 0.924s to 0.611s, using their own W4AFP8 quantization rather than the 1-bit GGUF.
Notice what all three numbers have in common. Four B200s is roughly 768 GB of HBM. Sixteen H200s is roughly 2,256 GB. Both are servers costing well into six figures. Every published speed for Kimi K3 describes hardware nobody reading this owns.
Why we had to search X to find that
The VESSL figure does not appear in general web search results. Neither does most measured local-inference data any more.
People who spend real money on hardware now publish their results almost exclusively on X, because that is where the monetization is. That has a side effect worth understanding: the measurements exist, but they live where engagement decides what surfaces, and a throughput table is not what gets amplified.
So when we say a number has not been published, we now check there first. It is where this material actually lives.
The Mac Studio Claim, Against the Arithmetic
The headline said the model runs on “a Mac Studio + 128 GB RAM device.” That phrasing is doing a lot of work.
Unsloth’s own documentation states the rule plainly:
RAM+VRAM ≈ the quant size; otherwise it’ll still work, just much slower due to disk offloading
And it puts the requirement for the 594 GB build at at least 610 GB of RAM.
Run the arithmetic on the machine in the headline. A 128 GB Mac Studio holds about 21% of the model. The other 79% has to come off an SSD, on every token, forever.
That is not a hypothetical penalty. It is the same wall we documented when nobody would publish a tokens-per-second figure for 405B on 8 GB: once your storage is inside the inference loop, throughput stops being a function of your GPU and starts being a function of your disk.
Nobody has published a Mac Studio throughput figure for K3. Not the announcement, not the model card, not the docs, and not on X where the hardware testers actually post.
What is on X instead is the claim itself, spreading. Searching as this was written, the same sentence had been reposted across four languages in under half an hour, always with the size, the accuracy and the Mac Studio, never with a speed.
The interesting part is that readers are catching it unaided. One French post quoted the 594 GB against the 128 GB machine and simply wrote “faites le calcul”, do the math. A Japanese one worked out that the realistic requirement is a 512 GB Mac Studio at roughly 2.5 million yen, and concluded it is currently impractical. Another put it more bluntly: at just under 600 GB in IQ1_S, the model is “still hopelessly large.”
None of them needed our calculator to see it. They needed one division.
Why The Speed Goes Missing
This is worth naming, because it is not specific to AI and it is not dishonesty.
“594 GB, 78.9% accuracy, runs on a Mac Studio” is a shareable sentence. “594 GB, 78.9% accuracy, and here is what it does per second when four fifths of it lives on an SSD” is not. On a platform that pays in reach, the first sentence wins and the second one stays in the documentation, where it was published honestly and where almost nobody reads it.
The result is a claim that is technically true at every step and misleading as a whole. It loads is not the same as it runs, and the gap between those two words is where the entire user experience lives.
We have made the same category of error ourselves. Our own “regardless of quantization” line was true when we wrote it and wrong within 48 hours, which is why this page exists.
What It Actually Takes
Our calculator computes full residency, everything on the GPU, nothing offloaded. That is the honest baseline, and for K3 at Q4_K_M it is 1,796 GB, or five MI430X cards with almost nothing to spare.
For the 1-bit build, the arithmetic is simpler. The weights are 594 GB and K3’s cache is unusually small for its size, 0.7 GB at 8K context and 29.5 GB at a full 1M, because 69 of its 93 layers use Kimi Delta Attention and hold a fixed state instead of a growing cache. So your target is roughly 600 GB of combined RAM and VRAM, and the context is nearly free once you are there.
Three honest paths:
- A multi-GPU server. Four B200s, or the 16 H200s in the VESSL result. These are the only configurations with a published speed.
- Rent it. For a model this size, hourly rental is how most people will ever touch it.
- Accept the disk penalty knowingly. It will load on far less. Just do not expect it to be interactive, and do not budget for it until someone publishes a number. If you go this way, the flags that matter on constrained hardware are worth reading first, because they decide how much of the model stays resident between tokens.
To check a specific machine rather than read a table, see what your GPU can actually run.
What We Are Still Checking
We are actively looking for a measured tokens-per-second figure on consumer or prosumer hardware, specifically Apple silicon and single-GPU-plus-RAM setups. If you have run the 594 GB build and measured throughput, that is exactly the data this question needs.
Partly answered on 7 August: one tester published 2.42 tok/s across three linked DGX Sparks, covered further down this page. That is a single report on a pre-release quant, so the request stands for Apple silicon and single-GPU setups, and we will keep updating this page in either direction.
If 1,796 GB is out of reach, and for almost everyone it is, DeepSeek V4 Flash 0731 is the same question with a very different answer: 162 to 178 GB at 4-bit, with measured throughput figures from people running it on two linked DGX Sparks.
Then Somebody Ran It In 8 GB
On 1 August a repository called kimi-k3-in-c appeared, Apache-2.0, and collected 2,700 stars in five days. The claim that travelled off it was simple: a 2.78 trillion parameter model, one CPU, 8.24 GB of RAM.
The measurement is real. The author published it himself, along with everything needed to check it.
It was never run on a laptop.
His README says so in its own words, in a section titled “The machine, and what it assumes”:
Every measurement here comes from one workstation: a two-socket AMD EPYC 7763 with 124 cores and no SMT, 228 GB of RAM, and 3.2 TB of NVMe. It also has four NVIDIA L40 GPUs, which sat completely idle for the entire campaign, because this engine has no GPU path.
--preset laptop is a memory budget flag. It caps how much the process may hold, on a machine with 228 GB available. The 8.24 GB is a genuine peak RSS under that cap. It is not a laptop result, and the repo never said it was.
Why this does not contradict the 610 GB above
Both numbers are right because they describe different things. The 610 GB figure is residency: every weight in memory at once, which is what llama.cpp and every runtime in our calculator assume. The 8.24 GB is streaming: only 16 of the model’s 896 experts fire for any given token, so the sleeping 93% stays on disk and arrives on demand.
Different mode, different bill. And the bill moves from RAM to disk traffic.
What 8 GB of RAM actually costs
The author puts the price in one line in his quickstart, and it is the most useful sentence in the repository:
Storage matters more than you would expect, the engine moves ~135 GB per token at small budgets.
135 GB, per token, off NVMe. That is what buys the 8.24 GB. His logged run reads 8 tokens in 261.5 s, 32.69 s/token average, which is close to 1.8 tokens per minute. A sentence takes about a quarter of an hour.
Give it more room and it does improve. A second logged run at 127.92 GB of RSS reaches 10.69 s/token. Worth knowing before anyone reads that as a clean scaling law: the runs generated 8 and 28 tokens, so start-up cost is spread differently across them, and the author never presents them as a controlled pair. Neither do we.
The number that matches ours exactly
One detail in the repo matters more to this site than the speed does. Its verification step walks all 96 shards of the checkpoint and reports bytes: 1560936091448, then RESULT: EXACT MATCH.
That is 1,560.9 GB. It is the same figure our calculator carries for Kimi K3, taken from the safetensors index when the weights landed, and it was reached here by a completely independent implementation in C. Two routes, one number.
We keep pointing out that several tools publish 4-bit totals smaller than the file you have to download. This is the other side of that: when a figure is measured rather than estimated, independent people land on it to the byte.
The other path, measured on rented hardware
Streaming is one way to fit K3 on hardware you can reach. The other is to rent enough machine to hold it, and somebody has now published a real number for that.
ComputingForGeeks measured the 594 GB UD-IQ1_S build at 0.10 tokens per second, on four A100-SXM4-40GB cards with dual EPYC 7542 processors and 2 TB of system memory, running llama.cpp b10245 on Ubuntu 24.04 with CUDA 12.6. They put the rental at $3.87 an hour and the entire exercise, download and build and benchmark included, at $8.29.
And a third engine, which is faster and not running the same bits
Four days before kimi-k3-in-c, a project called WASTE did the same trick with more room. It reached 1,800 stars and 339 points on Hacker News, and its published measurement is a 64 GB MacBook Pro with an M5 Pro: 29.06 GB of RAM at roughly 0.6 tokens per second. That is about twenty times faster than the 8 GB budget above.
Read the container size before drawing conclusions from that. WASTE ships 982 GB, not 1.56 TB, because it re-quantizes the experts to 3-bit residual vector quantization and keeps only the more sensitive shared weights at 4 or 8 bits. Its README says “this is the full model, not a distilled or pruned version”, and on parameter count that is accurate. On precision it is a different artifact from the one kimi-k3-in-c streams, which reads Moonshot’s original MXFP4 weights untouched.
So the two are not a like-for-like race. One is faster partly because it is carrying less.
Its storage note deserves repeating for anyone about to buy a drive: the internal SSD sustained 12.78 GB/s, while a tested USB enclosure managed 0.94 GB/s. Thirteen times slower, on the one resource that decides everything here. External storage is not an option for either engine.
| Route | What it loads | Memory to run it | Speed | Measured by |
|---|---|---|---|---|
| Streaming, WASTE | 982 GB, experts re-quantized to 3-bit | 29.06 GB RAM | ~0.6 tok/s | WASTE authors |
| Residency, 1-bit on 4x A100 | 594 GB UD-IQ1_S | 160 GB VRAM plus 2 TB RAM | 0.10 tok/s | ComputingForGeeks |
| Streaming, kimi-k3-in-c | 1.56 TB, original MXFP4 | 8.24 GB RAM, 1.7 TB disk | 0.031 tok/s | repository author |
The column that matters is the second one. Every figure in the speed column is bought by giving something up, and the three engines give up different things: precision, money, or time.
Some skepticism is fair here and it is worth being straight about where it lands. The largest Hacker News thread on WASTE spent most of its 166 comments arguing about whether its README was written by a language model rather than about whether the engine works, which tells you little either way. The substantive caution is simpler: none of these three numbers was produced by someone independent of the project that published it, except the A100 result. Two are self-reported, and self-reported numbers on new engines have a way of moving once other people try them.
That is the honest state of running Kimi K3 locally in August 2026. Three routes work. All three are demonstrations, not tools.
What nobody has done yet
Reproduced it. As of 6 August the repository has 454 forks and no third-party speed report we could find, on GitHub, Reddit or X. The author has already merged a pull request re-measuring the laptop preset at 8.78 GB against the published 8.24, a 6.5% gap, on his own hardware.
Storage is the real entry ticket, not memory. The checkpoint is 1.56 TB and the dense trunk is another 108.81 GB, so roughly 1.7 TB of local NVMe, and the author warns that network volumes run several times slower. An 8 GB laptop with a 512 GB SSD cannot hold the file, never mind stream it.
None of which is a criticism of the engine. It does what it says, the documentation is unusually honest, and reading MXFP4 weights directly with a 176 kilobyte binary is a genuinely impressive piece of work. The claim simply lost its conditions somewhere between the README and the timeline.
A Measured Speed Finally Arrived, On Three Sparks
The section above has been asking for a measured tokens-per-second figure on hardware a person could actually buy. On 7 August one appeared, and it is the fastest K3 figure published so far by a wide margin.
An independent tester benchmarked a pre-release quantization from @ViC305 across three linked DGX Sparks using llama.cpp with RPC, and reported:
- 308 GiB total footprint
- 2.42 tok/s median, single stream
- ~2.7 seconds time to first token
Put that against the speed column earlier on this page and the gap is the story. The best previously published figure was 0.6 tok/s. This is four times faster than anything measured before it, and roughly 78 times the 0.031 tok/s of the streaming build.
The size is the other half. At 308 GiB, or about 331 GB, this build is 44% smaller than the smallest published Unsloth build at 594 GB, and about 82% below the 1,796 GB full model. The tester’s own summary was that they expected compression that aggressive to destroy the model and it did not: their acceptance run covered factual recall, arithmetic, JSON, multi-turn, coding with execution, reasoning format and repetition, and reported passes across all of them.
What the quantization actually did
The published inspection figures explain why it survived, and they are internally consistent, which is worth saying because plenty of numbers on this topic have not been:
| Property | Value |
|---|---|
| Experts retained per layer | 896 |
| Active per token | 16 |
| MoE layers | 92 |
| Routed expert instances | 82,432 |
| FFN width | 3072 to 1536 |
896 experts across 92 layers is exactly 82,432, and 3072 to 1536 is exactly half. Every expert route stays alive and each one is half as wide. That is a different strategy from dropping experts entirely, and it is a reasonable explanation for why the routing behaviour survived when the bit-width did not.
The part that will not travel: it fits by about 5 GB
Three DGX Sparks advertise 384 GB of unified memory between them. That is not what you get. Usable memory on a Spark runs roughly 112 to 119 GB depending on how far you tune the operating system, a range covered in detail on our DGX Spark page.
| Basis | Total across 3 | Headroom on 331 GB |
|---|---|---|
| Advertised, 128 GB each | 384 GB | 53 GB |
| Tuned ceiling, about 119 GiB each | 357 GB | 26 GB |
| Hands-on figure, about 112 GB each | 336 GB | about 5 GB |
On an untuned three-Spark setup this fits with roughly 5 GB to spare, and that is before the RPC layer’s own overhead for coordinating three machines. “It fits on three Sparks” is true. It is not the same sentence as “you have room.”
What this does not establish
This is one person’s single run of a quantization that is not publicly released, and nobody has reproduced it. The 2.42 tok/s is credible and it is not yet a fact about the model. Three specific gaps:
- The quant is pre-release. Until the weights are downloadable, the size and the quality both rest on one report.
- No perplexity or benchmark score was published, only a pass or fail acceptance run. “Genuinely coherent” is an impression, and a careful one, but it is not a measurement of quality loss.
- Nothing here was measured by us. We own no hardware. Every figure in this section belongs to the tester or to @ViC305.
Our position on this page moves accordingly, and only this far: the usability claim now has one credible measurement behind it instead of none. At 2.42 tok/s K3 is slow enough that you would not sit and wait for it, and fast enough that a batch job finishes overnight rather than never. That is a real change from 0.031 tok/s, and it took three machines to get there.
FAQ
Can you run Kimi K3 locally?
Yes, since July 29, 2026. Unsloth published a dynamic 1-bit build at 594 GB, down from 1.56 TB, retaining roughly 78.9% top-1 accuracy. There is video of it running on four B200 cards. It needs roughly 600 GB of combined RAM and VRAM to run without heavy disk offloading.
How much RAM does the 1-bit Kimi K3 need?
Unsloth’s documentation puts it at least 610 GB for the 594 GB UD-IQ1_S build, and states the general rule that RAM plus VRAM should roughly equal the quant size, otherwise it still works but is much slower due to disk offloading.
Does Kimi K3 run on a Mac Studio?
It will load, but no measured throughput figure has been published for Apple silicon. A 128 GB Mac Studio holds about 21% of the 594 GB build, so roughly four fifths streams from disk on every token. The only published speed for K3 is on B200 hardware.
How fast is Kimi K3 locally?
Unsloth’s documentation states about 20 tokens per second generation on B200s with over 120 tokens per second throughput. A figure of 36 tokens per second on four B200s is also circulating, which we have not yet confirmed at source. No figure has been published for consumer hardware.
Why is Kimi K3’s KV cache so small?
Because 69 of its 93 layers use Kimi Delta Attention, which holds a fixed recurrent state rather than a per-token cache. Only 24 layers use MLA. Its cache is about 0.7 GB at 8K context and 29.5 GB at a full 1M-token context, so long context costs very little once the weights fit.
Can Kimi K3 really run in 8 GB of RAM?
Yes, using kimi-k3-in-c, which streams weights from disk instead of holding them in memory. The 8.24 GB peak RSS is real and measured, but on a 228 GB EPYC workstation with 3.2 TB of NVMe, not on a laptop. It costs roughly 135 GB of disk reads per token and runs at 32.69 seconds per token, about 1.8 tokens a minute.
How much disk space does streaming Kimi K3 need?
Around 1.7 TB of local NVMe: the 1.56 TB checkpoint plus a 108.81 GB dense trunk. The author warns that network-attached storage runs several times slower, so an external drive or NAS is not a substitute.
Is streaming Kimi K3 from disk faster than CPU offload?
No. Streaming trades speed for memory, and it trades a lot of it. At 32.69 seconds per token you wait about a quarter of an hour for a short answer. It is worth doing to see the model run on hardware you own, not to use it for work.