Qwen 3.8 27B Quantization: What Quality Costs at Every Rung

Which GGUF Quant Should You Download?

Take the largest four-bit build that leaves room for your context. On Qwen 3.8 27B that is roughly 14 to 19 GB to download depending on who built it, and every campaign that measured a real task this month agrees it performs like the full-precision model. Below that the answer stops being a number and starts being a question about your work: two-bit builds hold up on one-shot questions and fall apart in agent loops. One-bit builds are not a compressed model, they are a different one.

Two things make that answer harder than it should be. The quality you keep is not proportional to the bits you pay for, and the file name does not tell you the size: two publishers ship a file called Qwen3.8-27B-Q4_K_M.gguf and they are 2.16 gigabytes apart.

What Each Rung Actually Costs, FP16 Down to 1-Bit

This is the table the question deserves. Download size is from the live Unsloth listing on 2 September 2026. The pass rate is @superalesha’s 300 frozen tasks at maximum thinking effort, paired against a BF16 reference. The agentic score is his separate 240-task ladder out of 720, half of it agent work, with a stated 3-point noise floor.

Read the first four rows together and the whole top of the ladder collapses into one decision. BF16, FP8, Q6_K and Q4_K_XL are separated by 1.7 points across 300 tasks, and every gap between them has a p-value over 0.49. Q6_K at 23.1 GB and Q4_K_XL at 17.6 GB are statistically the same model. Five and a half gigabytes buys nothing.

Then read the last two rows. A file that scores 79.4% and a file that scores 43.0% are three gigabytes apart.

The Middle of the Ladder Is Where the Data Runs Out, and Where People Live

Three of the eleven rows above have no pass rate at all. UD-Q4_K_M, UD-IQ4_XS and UD-IQ2_S were never put through the 300-task set, and IQ4_XS in particular is the file we and everyone else recommend to 16 GB owners. What exists for them is the agentic ladder, which is the harder suite and the one that separates them: 660, 652, 650 for the three four-bit builds, against a 3-point noise floor.

Ten points across four-bit is small. It is also, by the campaign’s own criterion, real.

The Thinking Budget Moves Quality Seven Times More Than the Quant Does

This is the finding that reframes everything above, and almost nobody who publishes a quant comparison mentions it. Same weights, same tasks, one knob:

Twenty points. No quantization in the entire study moved the number by more than three, except the one-bit build. So a reader agonising over Q4 against Q5 is optimising the smaller variable by a factor of seven.

And a Short Budget Exposes Damage a Long One Hides

The same nine artifacts, at each effort level, against the reference:

Follow the Q2_K_XL row across. At maximum effort it is 0.9 points under the reference and indistinguishable from it. At low effort it drops 8.0 points, p=0.003. The compression damage did not appear or disappear. A long reasoning chain was correcting it, and shortening the chain stopped paying for the repair.

That is the single most practical thing in this article. If you run a two-bit build with thinking turned down, you are not getting the two-bit results everyone quotes.

One honest caution about reading that grid. Q4_K_XL at “off” effort scores 68.0% against the reference’s 61.3%, which is 6.7 points ahead at p=0.004. A four-bit file cannot carry more information than the weights it came from, so that cell is not a result. The off and low columns are noisier than their p-values suggest, and the max column is the one to trust.

One-Bit Does Not Degrade Evenly, and What Survives Is Surprising

IQ1_M against the reference, broken out by what the task asked for:

Three abilities fall off a cliff and one climbs down. Picking the right function out of a schema survives compression that destroys everything else, which @superalesha attributes to how heavily these models are trained on exactly that.

It is still a 17-point loss. But it means a one-bit build is not uniformly useless: it is useless at thinking and merely bad at routing.

Marie’s version of the same verdict is the one to remember, because it is a comparison rather than a number. A one-bit 27B scores below a four-bit Qwen3 4B on LiveCodeBench. If your memory budget only reaches one-bit on this model, a smaller model at four-bit is the better machine.

The Label Is a Request, Not a Description

Two people quantized Qwen 3.8 27B in the same week. Unsloth and AtomicChat both published a full ladder, both used the standard llama.cpp format names, and both uploaded to Hugging Face where the size sits next to the filename.

We read both repositories on 2 September 2026. These are the download sizes as the listings report them.

Read the top row first, because there is a control available and it is not the one you would guess. Unsloth and lmstudio-community both ship a plainly named Q8_0 for this model and their files are 1,792 bytes apart out of 29 gigabytes. That is the agreement two people get running the same tool on the same weights.

AtomicChat’s Q8_0 is 160 megabytes smaller than both. ggml-org’s is 450 megabytes smaller. So even the rung nobody thinks of as a recipe is one, and the 0.6% in the table above is already a choice rather than a rounding difference. Every row under it is a bigger choice, reaching a third of the file at IQ2_S.

The Consequence Nobody Prints

Once two ladders diverge that far, they cross.

• AtomicChat’s IQ2_S is 11.14 GB. Unsloth’s IQ3_XXS is 10.93 GB. A two-bit file, larger than a three-bit file.
• AtomicChat’s IQ1_M is 8.50 GB. Unsloth’s IQ2_XXS is 7.27 GB. The one-bit build is the bigger download.
• AtomicChat’s IQ4_XS is 16.51 GB against Unsloth’s Q4_K_M at 16.46 GB. The economy option costs more than the standard one, though only just.

If you picked a quant by name, believing a lower number means a smaller file, three of those choices go the wrong way.

d4nnyy, who built the AtomicChat ladder, measured the same effect on quality rather than size. Two publishers ship a file with the exact name Q4_K_M and a third ships its own AD-prefixed version. He ran all three through one harness against one reference:

His words: a quant name tells you which recipe was requested, not what you are getting.

What Is Actually Inside a Dynamic Quant

The publishers are not hiding this. Unsloth’s UD prefix means Unsloth Dynamic, and the whole point of the method is to keep sensitive layers at higher precision instead of squeezing everything equally. But almost nobody has looked at what that produces.

A GGUF file carries a tensor table in its header, and the header can be read over a range request without downloading the weights. We read Unsloth’s Qwen 3.8 27B files that way. The file labelled UD-Q2_K_XL contains fifteen different quantization formats, and here is how the bytes divide:

The format on the label is sixth by size. The largest single component is a three-bit format.

UD-IQ1_M is the same story: 6.7% of it is IQ1_M, and 34.6% is IQ1_S with another 25.1% at IQ2_XXS.

This matters for reading every benchmark below. When a campaign reports that “Q2_K_XL held up”, the artifact it tested was a fifteen-format mixture whose largest part is three-bit. That is a real and useful result about a real and downloadable file. It is not a result about two-bit quantization, and the two get conflated constantly.

Five Campaigns on One Model, and They Do Not All Agree

Qwen 3.8 27B is the most heavily measured quantization subject in local AI right now. Between 15 August and 2 September five independent efforts published:

• @superalesha ran two campaigns on 4x RTX 3090. One is pre-registered, with the analysis plan written before the results were read: 300 frozen tasks, paired against a BF16 reference, permutation tests at 20,000 resamples. The other is a seven-rung ladder scored across 240 tasks, half of them agentic.
• Piotr Migdał at Quesma ran three separate benchmarks with about $3,000 of rented compute, reporting Wilson confidence intervals.
• d4nnyy at AtomicChat measured 36 files, his own and other publishers’, on top-1 agreement and KL divergence against a full-precision reference.
• Benjamin Marie benchmarked 15 GGUFs against a 95% recovery threshold using more than 150 million generated tokens.
• @witcheer scored seven rungs on a single RTX 5090, which we already cover on our Qwen 3.8 27B speed page.

Every one of these is one person on one rig, which is exactly why the useful thing is not any single campaign. It is what they agree on when they turned different knobs.

Where All Five Agree

Four-bit lands on full precision, and the four-bit builds do not all land on each other. Every campaign that scored a real task found the rung indistinguishable from the reference: @superalesha’s Q4_K_XL came in 0.7 points under a BF16 reference at p=0.58, and Migdał replicates the full-precision result on agentic coding. That much needs no hedge.

What the agentic ladder adds is that inside four-bit, Q4_K_XL scored 660 out of 720 and IQ4_XS scored 650, against his own stated 3-point noise floor. Ten points is not the 31 that separates two-bit, and it is not nothing either. Four-bit is the rung to be on. Which four-bit file is still a choice.

One-bit is a different model. @superalesha’s IQ1_M scored 43.0% against the reference’s 80.3%, a drop of 37.3 points at p below 0.0001. Migdał puts one-bit at or below random guessing on graduate science questions. d4nnyy, who built one himself, says of it: it exists, we would not run it. Marie’s conclusion is the practical one, that a smaller model at a healthier quantization is the better trade.

The Question Was Never Which Quant. It Was Which Job.

Ask whether two-bit costs you anything and you get two answers. Both are correct.

On one-shot questions with a long thinking budget, it costs nothing you can measure. @superalesha found 0.9 points against the reference across 300 tasks, well inside his noise floor. Migdał found no change at all on instruction following.

Give the same file an agent loop, or take its thinking budget away, and the cost appears. @superalesha’s agentic ladder puts Q2_K_XL 31 points below the top of the ladder out of 720, against a stated noise floor of 3. The same file at low reasoning effort loses 8.0 points at p=0.003. Migdał reports a noticeable fall on 89 agentic coding tasks.

Same file. Same weights. The split runs clean along one axis: every suite that says two-bit is free is single-turn or instruction-shaped, and every suite that says it costs you is agentic. Two independent parties found it separately, and Migdał found it inside one campaign across his own three benchmarks.

The thinking-budget axis is less settled. @superalesha’s own file loses 8.0 points at low effort with p=0.003, while Marie ran at low thinking mode and still found a 9.83 GB two-bit build above 95% recovery. Those point different ways, and nobody has run the two setups against each other.

Nobody in that set is wrong. The single-number quality ladder is.

We Published Half of This Ourselves, and It Needs the Other Half

Our Qwen 3.8 27B speed page carries @witcheer’s seven-rung ladder and reports that it spans 2.9 points with no cliff at any step. That page already scopes the claim to short prompts, and it already shows his depth test flipping it: Q6_K retrieved 45 of 45 buried facts, UD-IQ3_XXS managed 33 of 45.

He measured something real and it was not wrong. A seven-rung ladder on one composite metric cannot resolve what a 150-task agentic suite resolves. What the new campaigns add is that context length was never the only knob. Thinking budget does it. Task type does it. Three people turned three different dials and the ladder tilted every time.

The Other Way to Measure This, and Why It Does Not Convert

Everything above is a task pass rate: did the model finish the job. d4nnyy measured something different across 36 files, which is how often each quantized file picks the same next token as the original weights.

That method ranks 36 files for the price of one benchmark run, which is why it can cover the whole ladder when nobody else can afford to. It does not convert into a pass rate and must never be read as one. A one-bit file keeps roughly three quarters of its next-token choices and still scores at random chance on graduate science questions, because the quarter it gets wrong are not distributed evenly across a sentence. One token can end an answer or invert a conclusion.

Use fidelity to choose between two files of the same size. Use pass rates to decide whether that size is enough.

Where It Actually Stops Being Worth It

Three of the five campaigns stop before the interesting part. @superalesha’s pre-registered set has nothing between 2.7 and 1.8 bits and he says so. His ladder never ran IQ1_M at all. Migdał jumps from a 10.7 GB two-bit file to a 6.2 GB one-bit file with nothing in between.

Marie went into that range and came out with a number. In his newsletter of 29 August, which is free to read:

Below roughly 9-9.5 GB, you’re probably giving up too much accuracy.

And on what still works above it: the quantizations from Unsloth, LM Studio, AtomicChat, llama.cpp and bartowski are all very good down to UD-Q2_K_XL at 9.83 GB, retaining more than 95% of BF16 accuracy in his tests, including at context lengths up to 128K. Below about 9.5 GB his advice is to take a smaller model at a higher precision instead.

He also published per-file verdicts inside the window, on a separate set of builds from ISTA-DASLab: IQ2_S and IQ3_XXS are fine, and avoid IQ2_XXS, whose LiveCodeBench accuracy he puts slightly below Qwen3-4B-Thinking-2507. A 27B losing to a 4B is the most concrete version of this whole article anyone has published.

The Bytes Predicted the Same File

We read the tensor tables of all five rungs across that range before Marie’s verdicts were in front of us, and counted what share of each file sits in formats below 2.5 bits.

The file that survived is a fifth low-bit. The file that collapsed is nearly three quarters low-bit. UD-IQ2_XXS sits 7.5 points from the one that collapsed and 44 points from the one that held. On composition it belongs with the wrong neighbour.

Two other people named the same rung, from two other directions. @superalesha, in his thread: the drop most likely already happens at Q2_XXS, but it was not in this benchmark set. He inferred it from where his benchmark stopped. Marie then measured an IQ2_XXS and told people to avoid it. We got there from bytes in a file header.

Ours is an inference, not a measurement, and it is not proof of the other two. “Share under 2.5 bits” is our own metric rather than a standard one, the per-format bit widths behind it are published approximations, and composition is not quality. Marie’s verdict is also on ISTA-DASLab’s build while our read is on Unsloth’s, which is exactly the publisher difference this article is about. Three methods pointing at one rung is worth saying. It is not the same as one benchmark run, which would settle it in an afternoon.

Why This Question Is About to Get Much Bigger

On 2 September Qwen released Qwen 3.8 Max 0902, and Arena has it debuting at number one overall on Code Arena WebDev with 1691 points, three above Claude Opus 5 and 22 above the previous Max. It is 2.4 trillion parameters, a million tokens of context, and it is API only at $2 and $6 per million tokens. No weights.

That sounds like somebody else’s problem until you notice what is already published. Qwen3.8-2.4T-A95B has open weights, and Unsloth has shipped GGUFs of it since 12 August. @mudler_it has already run one on a single DGX Spark: UD-Q1_0, 62 GiB resident out of a 370 GiB checkpoint, at 11.05 seconds per token on CPU.

Eleven seconds per token is not a usable machine. It is also exactly the number physics predicts when most of a 370 GB checkpoint streams from system memory every token, which is why it is a figure worth trusting. The same pattern ran through Kimi K3, where the 1-bit build is what put a 2.8 trillion parameter model on hardware people own.

So the compression question stops being about saving disk space. It is the thing standing between a frontier-scale model existing and it running in your house, and every point of quality lost on the way is the price of admission.

Which makes the open question sharper rather than softer. Everything above says one-bit is a different model, measured on a 27B. Whether that holds at 2.4 trillion parameters is unknown: the common expectation is that larger models tolerate aggressive quantization better, and nobody has published a quality measurement on a 1-bit 2.4T at all. The person who ran it reported speed and memory, not accuracy.

So What Do You Download

Sizes below are downloads. A file occupies slightly less once it is resident on the card, and your context window is on top of it. Our calculator does that arithmetic for your specific card and context length.

The rule underneath the table: pick the quant for the job, then check the file size on the listing rather than trusting the name. On this model that name can be wrong by a third.

For a model too large to fit at any quant, CPU offload changes the question entirely, and the full memory breakdown for this model sits on our Qwen 3.8 VRAM requirements page.

FAQ

Is Q4_K_M good enough for real work?

Yes, on this model, and it is the best-supported claim in the subject. Five independent campaigns across five hardware sets and six task suites found four-bit builds performing like the full-precision reference. @superalesha measured Q4_K_XL at 0.7 points below a BF16 reference with p=0.58, inside his stated noise floor.

Why are two files with the same quant name different sizes?

Because the name is the recipe that was requested, not the result. Publishers apply their own layer-by-layer schemes: Unsloth’s UD prefix and AtomicChat’s AD prefix both mean the publisher chose which tensors to keep at higher precision. On Qwen 3.8 27B those choices put their IQ2_S files 33% apart. The only rung where they agree closely is Q8_0, at 0.6%, because neither applies a custom recipe there.

Does a Q2_K file actually contain Q2_K weights?

Mostly not. Reading the tensor table of Unsloth’s UD-Q2_K_XL for Qwen 3.8 27B shows fifteen quantization formats inside one file, with Q2_K sixth by size at 7.2% and the largest component being IQ3_XXS at 25.3%. Benchmarks of that file are results about that file, not about two-bit quantization generally.

At what point does quantization actually hurt?

Benjamin Marie puts it at roughly 9 to 9.5 GB on this model. His tests hold above 95% of BF16 accuracy down to UD-Q2_K_XL at 9.83 GB, including at context lengths up to 128K, and below about 9.5 GB his advice is to take a smaller model at higher precision. He also singles out IQ2_XXS as one to avoid, putting its LiveCodeBench accuracy slightly below a 4B model.

Why do some benchmarks say 2-bit is fine and others say it breaks?

Because they measured different jobs. Suites that are single-turn or instruction-shaped find two-bit costless. Suites that are agentic, or that cut the model’s thinking budget, find it costly: 31 points out of 720 on @superalesha’s agentic ladder, and 8 points at low reasoning effort with p=0.003. The file is the same. The workload is not.

Can I run Qwen 3.8 Max 0902 locally?

Not Max 0902 itself. Qwen 3.8 Max 0902, which Arena ranked number one overall on Code Arena WebDev on 2 September 2026 with 1691 points, is API only and has no published weights. Its open-weight sibling Qwen3.8-2.4T-A95B does have weights and community GGUFs, and @mudler_it ran a 1-bit build on a single DGX Spark at 62 GiB resident from a 370 GiB checkpoint. That was 11.05 seconds per token on CPU, so it runs rather than works. No quality measurement on a 1-bit 2.4T has been published by anyone.

Is one-bit worth trying?

Only as a curiosity. @superalesha measured IQ1_M at 43.0% against a reference at 80.3%, a 37.3 point drop. Migdał puts one-bit at or below random guessing on graduate science questions. d4nnyy, who built one, says it exists and he would not run it. A smaller model at four-bit is the better use of the same memory.