How Much VRAM Does Inkling Need?
680 GB at Q4_K_M with an 8K context for the 975B flagship, or 193 GB for Inkling-Small, which fits a single card. Those are the two numbers most people want, because Q4_K_M is the quantization almost every local deployment reaches for first.
Inkling is Thinking Machines Lab's flagship open-weights model, released July 15, 2026: 975 billion total parameters with 41 billion active per token, a 1M-token context, Apache 2.0, and native multimodal input across text, images and audio. It is the largest open-weight model published by a US lab. On July 30 the lab added Inkling-Small at 276B total and 12B active, covered further down.
None of that changes the arithmetic. Here is every quantization tier for the flagship, computed on our calculator at 8K context with an F16 KV cache under llama.cpp:
| Quantization | Weights | Total VRAM | Smallest pool that fits |
|---|---|---|---|
| BF16 / FP16 | 1,950 GB | 2,243.2 GB | 7x Instinct MI430X (74% used) |
| Q8_0 | 1,036 GB | 1,192.0 GB | 4x Instinct MI430X (69% used) |
| Q6_K | 801 GB | 921.5 GB | 3x Instinct MI430X (71% used) |
| Q5_K_M | 691 GB | 795.4 GB | 3x Instinct MI430X (61% used) |
| Q4_K_M | 591 GB | 680.4 GB | 2x Instinct MI430X (79% used) |
| Q3_K_M | 408 GB | 470.2 GB | 2x B300 288GB (82% used) |
| Q2_K | 320 GB | 368.6 GB | 1x Instinct MI430X (85% used) |
The bottom row is the interesting one. At Q2_K, Inkling drops onto a single 432GB card. That is the only configuration on this table that does not require a multi-GPU pool, and it is also the tier where quality degradation stops being theoretical.
Why Published Inkling Figures Disagree So Badly
Search for Inkling's memory requirement and you will find numbers ranging from roughly 280 GB to 2,076 GB for what appears to be the same model. That spread is not a rounding dispute. It is a factor of seven.
The spread comes from three things nobody labels: which quantization the figure assumes, whether it includes KV cache and runtime overhead or only raw weights, and whether it describes a single-GPU requirement or an aggregated pool across a server.
The useful cross-check is Thinking Machines' own deployment guidance, which is Tier 1 by definition. The lab's model card states the BF16 checkpoint needs at least 2 TB of aggregated VRAM, deployable as 8 B300s or 16 H200s, and that the quantized NVFP4 checkpoint needs at least 600 GB, deployable as 4 B300s in W4A4 mode or 8 H200s in W4A16.
Both of those are deployment totals, not weight sizes, so they line up against our total column rather than our weights column. Compared correctly: the lab says at least 2 TB at BF16 and we compute 2,243 GB. The lab says at least 600 GB at 4-bit and we compute 680.4 GB at Q4_K_M. Ours sits slightly higher in both cases, which is what you would expect, because Q4_K_M averages 4.85 bits per weight against NVFP4's 4, and because we report full residency with nothing offloaded.
Two independent methods, one answer. That reconciliation is what the other pages are missing, and getting it right means keeping weights and deployment totals in separate columns instead of quoting whichever is smaller.
The Architecture Behind the Numbers
Inkling's config.json explains why its cache behaves the way it does. The model has 66 layers, of which 55 see only a 512-token sliding window and 11 attend globally. There is a detail in there that is easy to get backwards: the sliding-window layers carry more key-value heads than the global ones, 16 against 8. Assume they match and you understate the cache on 55 of 66 layers.
The expert routing is conventional for a model this size: 256 routed experts with 6 active per token, plus 2 shared experts. All of them must be resident in memory, which is why the weights dominate every row of the table above. Sparse activation saves compute. It does not save memory.
Context Is Almost Free, and That Is the Sliding Window Working
Because only 11 layers attend globally, Inkling's cache grows far more slowly than its size suggests:
| Context | KV cache | Total at Q4_K_M |
|---|---|---|
| 8K | 0.6 GB | 680.4 GB |
| 1M | 47.5 GB | 734.4 GB |
Going from 8K to the full 1M-token context, a 128-fold increase, adds 54 GB, about 8% more memory. The 55 windowed layers stop growing at 512 tokens no matter how long the prompt runs, so only the 11 global layers scale.
This is the same structural trick, arrived at differently, that lets Kimi K3 carry a 1M context for a 1.7% memory increase. Sliding windows and recurrent states are converging on the same outcome: attention that stops tracking context length.
Inkling-Small: A Quarter the Size, and It Fits One Card
On July 30, 2026, Thinking Machines released Inkling-Small: 276 billion total parameters with 12 billion active, full weights, the same 1M context and the same text, image and audio input.
For anyone sizing hardware it is the more important of the two releases. The same table, for the smaller model:
| Quantization | Weights | Total VRAM | Smallest pool that fits |
|---|---|---|---|
| BF16 / FP16 | 552 GB | 635.2 GB | 2x Instinct MI430X (74% used) |
| Q8_0 | 293 GB | 337.6 GB | 1x Instinct MI430X (78% used) |
| Q6_K | 227 GB | 261.0 GB | 1x B300 288GB (91% used) |
| Q5_K_M | 196 GB | 225.3 GB | 1x B300 288GB (78% used) |
| Q4_K_M | 167 GB | 192.8 GB | 1x B300 288GB (67% used) |
| Q3_K_M | 116 GB | 133.3 GB | 1x H200 141GB (95% used) |
| Q2_K | 90.6 GB | 104.5 GB | 1x H200 141GB (74% used) |
The flagship needs 680.4 GB at Q4_K_M and a two-card pool. Inkling-Small needs 192.8 GB and fits a single B300 at 67% utilization. That is 3.5 times less memory, and it is the difference between specifying a server and specifying a card. At Q3_K_M it lands on one H200.
Its cache behaves the same way, for the same reason. Of 42 layers, 35 see only a 512-token window and 7 attend globally, so the cache is 0.3 GB at 8K and 30.1 GB at a full 1M. Total at 1M is 227.1 GB, up about 18% for a 128-fold increase in context.
One architectural note, because it is easy to get wrong and we checked rather than assumed: Inkling-Small uses 8 key-value heads on both its global and its windowed layers. The 975B uses 16 on the windowed ones against 8 global. Copying the larger model's shape would double the computed cache across 35 of 42 layers.
It Beats the Flagship on Reasoning. It Collapses on Knowledge.
The obvious objection is that a lab shipping a 975B model and then a 276B one that beats it makes the first model pointless. It does not, and the reason sits in the half of the comparison that rarely gets quoted.
Where Inkling-Small wins, from Thinking Machines' published figures:
| Benchmark | Inkling-Small 276B | Inkling 975B |
|---|---|---|
| SWE-Bench Verified | 80.2% | 77.6% |
| Terminal Bench 2.1 | 64.7% | 63.8% |
| GPQA Diamond | 89.5% | 87.2% |
| ARC-AGI-1 | 84.0% | 79.5% |
| GDPval-AA v2 (Elo) | 1269 | 1238 |
| IFBench | 82.2% | 79.8% |
And where it loses, from the same page:
| Benchmark | Inkling-Small 276B | Inkling 975B |
|---|---|---|
| SimpleQA Verified | 20.6% | 43.9% |
| AA Omniscience | -9.0 | 2.1 |
| Global-MMLU-Lite | 86.7% | 88.7% |
| AIME 2026 | 95.5% | 97.1% |
| Audio MC | 54.9% | 56.6% |
The flagship is more than twice as accurate on SimpleQA. And Inkling-Small scores negative on AA Omniscience, an index built so that confidently asserting things you do not know pushes you below zero. For anyone planning to deploy the smaller model, that is the most important number on this page.
The shape is a well-understood trade. Reasoning tracks the training recipe and the active parameter count. Factual recall tracks total parameters, and 276 billion cannot store what 975 billion store.
The small one is built from the big one
This is what resolves the apparent paradox. Inkling-Small is not a competing design. The lab states it "began training after its larger counterpart, which let us improve its training process," and that they "post-trained an earlier checkpoint, Inkling-Small (preview), in part using on-policy distillation with Inkling as the teacher."
Two details matter there. The 276B model exists because the 975B exists, since you cannot distil from a teacher you have not trained. And on-policy distillation is not the ordinary kind: the student generates its own attempts and the teacher grades them, which can exceed the teacher, where copying fixed outputs cannot.
Independently verified, where anyone has checked
Vendor benchmarks deserve suspicion, and a distilled student beating its teacher across most of a suite is not the normal result. So the useful question is whether a third party reproduced any of it.
One has. ARC Prize independently evaluated Inkling-Small and published 84% on ARC-AGI-1 at $0.11 per task and 40.1% on ARC-AGI-2 at $0.23 per task, calling it the highest-scoring open-weight model they have evaluated on both and a new cost-performance frontier. Those figures match Thinking Machines' own claims exactly, and ARC Prize publishes both its testing policy and its reproduction code.
That does not verify the other fourteen benchmarks. It does mean the two numbers anyone has independently checked came back identical, which is the opposite of what an inflated claim looks like.
Our position: verified where independently tested, unverified elsewhere, and carrying a documented knowledge weakness that matters more than the headline. We will update this page as third-party results appear.
Which one should you actually run
- Agentic coding, tool use, structured reasoning: the 276B, at 192.8 GB on a single card. Better at this, and a third of the memory.
- Anything leaning on world knowledge, RAG groundedness, factual answers: the 975B, at 680.4 GB. A SimpleQA gap of 43.9 against 20.6 is not something a system prompt fixes.
- A base to fine-tune: the 975B, which is what the lab positioned it as from the start.
Pricing follows the same logic rather than contradicting it: $1.20 per million output tokens against $4.05. The cheaper model is cheaper because it is smaller, not because it made the other obsolete.
What It Costs Against What It Delivers
This is where a requirements page has to be honest, because memory is only half of a hardware decision.
On the independent Artificial Analysis Intelligence Index, Inkling debuts at 41, ahead of Nemotron 3 Ultra at 38, Gemma 4 31B at 29 and gpt-oss-120b at 24. That places it 11th out of 98 models on their board, and the 41 is measured at Inkling's highest thinking-effort setting.
Artificial Analysis is careful with its own framing, describing Inkling as the leading open-weights release from a US lab rather than the leading open-weights model outright. That qualifier is theirs, not ours, and it is the most accurate one-line summary of where this model sits.
Set that against what each model costs to hold in memory, all at Q4_K_M and 8K on our calculator:
| Model | Total VRAM | Notes |
|---|---|---|
| Qwen3 235B-A22B | 165.7 GB | fits 1x B200 |
| Inkling-Small 276B / 12B | 192.8 GB | fits 1x B300 |
| DeepSeek V4 Flash 284B / 13B | 198.0 GB | fits 1x B300 |
| GLM-5.2 753B | 525.9 GB | |
| Inkling 975B / 41B | 680.4 GB | |
| Kimi K3 2.8T | 1,952.9 GB |
When this page first published, the comparison that mattered was the flagship against DeepSeek V4 Flash. Artificial Analysis scores Inkling at an Elo of 1238 on GDPval-AA v2 against V4 Flash max at 1189. Inkling won that pairing by roughly 4% and spent 3.4 times the memory doing it, which is a poor trade however you frame it.
Inkling-Small answered that within a day. Thinking Machines reports it at 1269 on the same benchmark, and our calculator puts it at 192.8 GB against V4 Flash's 198.0 GB. That is 80 Elo higher on 2.6% less memory. Not a trade, just a better result for less.
One caveat on sourcing. The 1238 and 1189 are published by Artificial Analysis; the 1269 is reported by Thinking Machines. They share a scale, because the 1238 the lab quotes for its own flagship matches what Artificial Analysis publishes independently, but that particular figure has not been reproduced by a third party. On Artificial Analysis' AA-Briefcase benchmark the older ordering still stands, placing the 975B above DeepSeek V4 Flash but below GLM-5.2 and Nemotron 3 Ultra, which need 525.9 GB and 384.0 GB respectively on our calculator. Neither competes in Inkling-Small's weight class.
Inkling does have one capability most of the models above do not: it accepts audio directly rather than a transcript. If your workload needs that, the memory arithmetic changes, because the alternative is running a second model.
One further caveat on the benchmarks themselves. Inkling was evaluated in mid-July, and its comparison set includes Kimi K2.6 rather than Kimi K3, which was published twelve days later at 2.8 trillion parameters. Those scores describe a field that has since moved.
Can You Run Inkling Locally?
The flagship, not on one consumer card, and not on any single GPU below the 432GB tier. Even the most aggressive quantization on its table needs 368.6 GB. Inkling-Small changes that materially: at Q3_K_M it fits a single H200, and at Q4_K_M a single B300.
Realistically there are three paths:
- A multi-GPU server. Two MI430X cards or four B300s for the flagship at Q4_K_M. This is datacenter hardware with datacenter power and cooling. Inkling-Small needs one card at the same quantization.
- Rent instead of buy. For a model this size, an hourly rental is how most people will ever touch it, and it costs far less than being wrong about a purchase.
- CPU offloading. Because only 41B parameters activate per token, most experts can sit in system RAM. This works, and it is slow. Our flags guide covers the trade-offs honestly.
Our calculator deliberately reports the full-residency number: everything on the GPU, no offloading, maximum speed. That is the honest baseline, and it is the figure that tells you whether you need any of the workarounds at all.
If you want to check a specific card rather than read a table, see what your GPU can actually run.
FAQ
How much VRAM does Inkling need?
Inkling needs about 680 GB at Q4_K_M quantization with an 8K context, including KV cache and runtime overhead. At BF16 it needs roughly 2,243 GB, and at Q2_K it drops to about 369 GB. Thinking Machines' own guidance says the BF16 checkpoint requires at least 2 TB of aggregated VRAM and the NVFP4 checkpoint at least 600 GB, which matches our independently computed figures.
How much VRAM does Inkling-Small need?
Inkling-Small needs about 192.8 GB at Q4_K_M with an 8K context, which fits a single B300 288GB card at 67% utilization. At Q3_K_M it drops to 133.3 GB and fits one H200. At BF16 it needs 635.2 GB. It has 276 billion total parameters with 12 billion active, against 975B and 41B for the full Inkling.
Is Inkling-Small better than Inkling?
On reasoning and agentic work, yes. Thinking Machines reports it ahead on SWE-Bench Verified, Terminal Bench, GPQA Diamond, ARC-AGI and GDPval-AA v2, and ARC Prize independently verified 84% on ARC-AGI-1 and 40.1% on ARC-AGI-2. On knowledge it is far behind: 20.6% against 43.9% on SimpleQA Verified, and a negative AA Omniscience score. Inkling-Small was distilled from Inkling, so the larger model is also the reason the smaller one exists.
Can Inkling run on a single GPU?
The 975B only at Q2_K, and only on a 432GB card such as the Instinct MI430X, which lands at about 85% utilization. Inkling-Small fits a single B300 at Q4_K_M and a single H200 at Q3_K_M. No consumer graphics card comes close to either at any quantization.
Why does Inkling's KV cache stay small at long context?
Because 55 of its 66 layers use a 512-token sliding window and only 11 attend globally. The windowed layers stop growing once the prompt passes 512 tokens, so moving from 8K to a full 1M context raises total memory only from 680.4 GB to 734.4 GB, roughly 8%, despite a 128-fold increase in context length.
Is Inkling better than Chinese open-weight models?
It depends on the benchmark. Artificial Analysis places Inkling at 41 on its Intelligence Index and describes it as the leading open-weights release from a US lab, while noting on the AA-Briefcase agentic benchmark that it ranks below GLM-5.2. Inkling-Small lands at 192.8 GB against DeepSeek V4 Flash's 198.0 GB, so the American entry now has a variant in the same memory class as the Chinese mid-tier. Our breakdown of the Chinese open-weight models has the full tier comparison.