Qwen3.8 27B MoE: A Real One Exists Now, and We Sized It

Qwen3.8 27B MoE: A Real One Exists Now

Somebody carved a mixture of experts out of dense Qwen3.8 27B without training a new model. It is called Whittle MoE 27B-A17.8B, it is Apache 2.0, the GGUFs are already published, and it runs on a 24 GB card quantized.

The headline everyone is repeating is “17.8B active of 27B total”. That number is real, and it is not a hardware number: the published files are within two and a half percent of the dense model at every precision, so it needs the same card it always did. Active parameters and total parameters do not pay for the same thing, and this release is the cleanest demonstration of that difference anyone has published.

Below: the file sizes side by side, what the conversion actually did to the weights, the decode speed it can buy, and the thing the release has no evidence for yet.

The Conversion, and the Detail That Proves It

The author’s own description is “a partition, not a rebuild”, and the arithmetic backs that up exactly.

Each layer’s dense feed-forward network in the parent is 17,408 wide. It was cut into 64 routed expert slivers of 192 each, plus one always-on shared expert of width 5,120. Multiply it out: 64 times 192 is 12,288, plus 5,120 is 17,408. The partition reconstructs the parent’s FFN to the neuron, and the author performs the same arithmetic on the card, calling it the original width to the neuron. No FFN weights were invented, and none were thrown away. The only new parameters in the whole model are the 64 routers, one per layer, and those had to be trained.

A new router picks 16 of the 64 slivers per token. All 64 layers keep the parent’s attention side untouched, including its 3-to-1 hybrid of gated deltanet layers and full attention, 16 attention layers in total.

That is why the active count lands at 17.8B rather than the 3B you see on a designed MoE. Per token you get 16 slivers at 192 wide plus the 5,120 shared expert, which is 8,192 of the parent’s 17,408 FFN width. Count the whole model rather than the feed-forward alone and it comes out at 17.8B of 27B. This is a mild MoE, and that matters for what it can pay you.

The File Sizes Settle the Memory Question

Published GGUFs, byte counts read from the repositories on 2026-08-24 and reported in decimal GB, not calculated from parameter counts:

Dense BF16 and Q8_0 are unsloth’s. Dense Q4_K_M is bartowski’s, because unsloth publishes no plain Q4_K_M for this model, only a UD dynamic build at 16.46 GB. Those are a different recipe rather than a different measurement, since a dynamic quant assigns bit widths per tensor, so putting one next to a flat Q4_K_M would compare two things that are not the same format. The two dense repositories agree on BF16 to within two kilobytes, which is what makes them safe to mix.

Within two and a half percent at every tier, and the MoE is the smaller file every time. The conversion did not shrink the model in any way that matters, because it was never going to: the same parameters are still there, just sorted into groups with a router deciding which groups to read.

This is the total-versus-active rule that governs every MoE, stated as plainly as it will ever get. Total parameters set your memory, which is why our page on the dense model sizes it at 27B and always will. Active parameters set your speed. A model with 17.8B active and 27B total needs the room for 27B, every time, on every card.

The author’s own hardware line is “runs on 24GB of VRAM quantized, splits across two 12GB cards”, and that checks out on its own file rather than on any comparison. Q4_K_M weights are 17.37 GB, which is 16.2 GiB as a card sees it, leaving most of a 24 GB card free for cache and overhead. It fits comfortably. So does the dense model it came from, at the same context length.

What You Actually Trade

What you give up is not context on paper. It is any evidence that long context still works. The config file is untouched at 262,144 tokens, the same as the parent, and nothing in the release reduces it. But the repair was trained on documents of 3,000 to 8,000 tokens, the author’s own serve command caps the window at 8,192, and every published loop and quality number was measured inside that window. Rewiring every feed-forward block in a model and then validating it only at 8K is not a claim that it holds at 262K. Nobody has published a number either way.

That matters more here than it would for a fine-tune, because looping is the exact failure this model shipped with, and looping gets worse with length. The card measures it: the late-turn loop rate, from the fifth turn onward, was 56% at the first release and 8% at v2.1, all inside an 8K window.

The practical read: treat 8K as the tested envelope, run longer at your own risk, and do not assume the parent’s 262K carries over just because the field in the config does.

What you get back is decode speed, and the size of the prize is arithmetic anyone can check. Decode on a single stream is bandwidth-bound: the machine reads the weights it needs, once per token. Dense, that is the whole 17.77 GB. Whittled, it is roughly 66% of 17.37 GB, about 11.5 GB.

On an RTX 3090 at 936 GB/s, that is a ceiling of about 53 tok/s dense against about 82 whittled. Call it 1.55x, and call it a ceiling rather than a measurement, because it is our arithmetic and nobody has published a measured figure for this model yet. It is also a long way short of what a designed MoE buys: Qwen3.6 35B-A3B activates 8.6% of itself per token, not 66%.

The Quality Cost, Which the Author Published Honestly

This is the part worth respecting. The release states what the model looked like before it was repaired.

Freshly carved, the model was gibberish: 4 of 39 on the author’s knowledge battery. Training only the 64 routers, with every expert frozen, took it back to 27 of 39, which is the finding the release is really built on: the knowledge never left the slivers, and the routers only had to learn which ones to reach for. At v2.1, after router distillation from the parent’s own logits and multi-turn supervised fine-tuning, the knowledge battery reads 28 of 39, single-turn looping is 8% against 69% at the previous release, structured output loops 22% against about 75%, and there are no silent or truncated answers.

The author also lists what is still broken: structured output still fails often, arithmetic is weak and inherited from the parent, and there are occasional fabricated words and number errors. The evaluation is one person’s, on a small harness, and the card says so.

The number that is missing is the one that would settle it: the dense parent’s score on that same battery. 28 of 39 could be a small loss or a large one, and without the baseline nobody can tell. We asked the same question of the release and could not find it.

Where This Gets Interesting

A dense model gets nothing from expert-offload engines, because there are no experts to place. That is the main limitation of FreeToken, released the same week, and it is why the most-run local model on this site cannot use it.

Whittle the dense model, and the same weights become eligible. A 27B with 64 routed experts per layer is exactly the shape those engines are built for. Nobody has tried that combination yet, and it is the first thing we would run if we had the hardware sitting idle.

Keep the scale in view, though. In the last thirty days the dense Qwen3.8 27B GGUFs were downloaded more than seven million times. Whittle’s were downloaded a few thousand. This is a research preview by one self-funded person whose compute budget ran out, published with its failures attached, and that is a different thing from a model you should move your workflow to.

What To Do With This

• If you want the memory to drop, this is not it. Use a smaller model or a lower quantization tier. Our GPU checker will show you what clears your card.
• If you want speed on a card that already fits the dense model, this is worth an evening, with the tested 8K window in mind and no measured numbers to rely on yet. For what the dense model actually does on real hardware, we keep a running set of measured Qwen3.8 27B speeds.
• If you were about to repeat “17.8B active” as a hardware claim, size it as 27B. That is the whole lesson, and the calculator will do it for you at any quantization tier.

Sources

• logic65/Qwen3.8-Whittle-MoE-27B-A17.8B and its GGUF repository, Apache 2.0, v2.1 dated 2026-08-22. Architecture, evaluation figures, limitations and the hardware line are the author’s own
• unsloth/Qwen3.8-27B-GGUF for the dense parent’s BF16 and Q8_0, and bartowski/Qwen3.8-27B-GGUF for its flat Q4_K_M, which unsloth no longer publishes
• Every size in the comparison table is a real published file, read from the repositories on 2026-08-24 and reported in decimal GB. The decode ceiling is our own arithmetic and is labelled as such throughout. We have not run this model.

FAQ

Does the Qwen3.8 27B MoE need less VRAM than the dense model?

No. The published files are within two and a half percent of the dense model at every precision, and the MoE is the smaller file every time, because the conversion partitions the existing weights rather than removing any. Total parameters decide memory, and the total is unchanged at 27B.

What does 17.8B active actually change?

How much of the model is read per token, which sets decode speed. Roughly two thirds of the weights move per token instead of all of them, so the bandwidth ceiling rises by about half. That is arithmetic rather than a measured result, and no measured figure has been published yet.

What is the context length?

The config file says 262,144, unchanged from the dense parent, and the release does not reduce it. What the release does not have is evidence that it works that far. The repair training used documents of 3,000 to 8,000 tokens, the author’s own launch command sets an 8,192 window, and every published number was measured there. Treat 8K as tested and anything beyond it as unproven.

Can it run on 24GB?

Yes, quantized. Q4_K_M weights are 17.37 GB, which is about 16.2 GiB as a card measures it, leaving room for cache and runtime overhead. The author also reports splitting it across two 12 GB cards. The dense parent fits the same card at the same context length.

How much quality was lost?

Nobody can say, and that is a fact about the evaluation rather than a dodge. The author reports 28 of 39 on his own knowledge battery, up from 4 of 39 when the model was freshly carved and 27 after router-only training. The battery is a private 39-item harness, so the dense parent’s score on it has never been published and no third party could publish one. Structured output still loops about one time in five.

Is this the same as Qwen releasing a small MoE?

No. Qwen did not ship a small MoE in the 3.8 generation. This is a third-party conversion by one person, published as a research preview with its failure modes documented, and its download counts are a rounding error next to the dense model’s.

Does it work with expert-offload engines like FreeToken?

Untested, and it is the interesting question. Those engines need experts to place, which a dense model does not have and this conversion creates. Nobody has published a result.