Expert guides on VRAM requirements, GPU benchmarks, LLM hardware, and local AI inference.
Qwen3.8 27B MoE: A Real One Exists Now Somebody carved a mixture of experts out of dense Qwen3.8 27B without training a new model. It is called Whittle MoE 27B-A17.8B, it is Apache 2.0, the GGUFs are already published, and it runs on a 24 GB card quantized. The headline everyone is repeating is “17.8B … Read more
FreeToken VRAM Requirements, and Why VRAM Is the Wrong Number Less than you think, and that is the wrong question. FreeToken runs a 35B model on an 8 GB laptop GPU, a 284B model on a 32 GB desktop card and GLM-5.2 at 753B on a single 96 GB workstation GPU. The number that decides … Read more
Are the Intel Arc Pro B70 and B65 Good for Local AI? Yes, if your bottleneck is VRAM capacity rather than raw speed. The Arc Pro B70 puts 32 GB of GDDR6 behind a full 32-core Battlemage die for $949, and the B65 pairs the same 32 GB with a smaller 20-core die for $910 … Read more
VRAMCalculator.com ALPHA build 0.1 What LLM Can My Mac Run? Usable by the GPU 0 GB Chip generation M1 M2 M3 M4 M5 Your Mac Number of machines GPU memory limit 75% of RAM macOS reserves the rest for the system. The default is about 75%, and some machines report less. Run ggml_metal_init in llama.cpp … Read more
Ornith 1.5 9B Runs at 16.3 tok/s on a Card From 2017 We put Ornith 1.5 9B on a Radeon RX 580, a nine year old 8 GB card, and it generated at 16.3 tokens per second at Q4_K_M with a 32,000 token context. On a 2024 Intel laptop it managed 8.6. Then we handed … Read more
Does DFlash 2 Actually Double Qwen 3.8 27B? Yes, against a plain baseline. One A100 test went from 28.9 to 59.1 tok/s, and one RTX 4090 test went from 60 tok/s on MTP to about 90 on DFlash 2. The word “again” matters. DFlash 2 is not a faster MTP. It is a second, independent … Read more
How Does a 27B Model Tie GPT-5.6? Qwen 3.8 27B scores 52 on the Artificial Analysis Intelligence Index, the same score as GPT-5.6 Luna (max), and one point behind GLM-5.2 at 753B and DeepSeek V4 Pro at 1.6T. The numbers come from Artificial Analysis, measured on their nine-eval index, and reported on 17 August 2026 … Read more
What Is an Abliterated Model? An abliterated model is an open-weight LLM whose refusal behavior has been surgically removed by deleting a single direction from its weights, rather than by retraining. The word blends “ablation”, the neuroscience term for cutting out a function to see what it does, with “obliterated.” The result answers prompts the … Read more
Qwen3.8 27B Speed on Real Hardware, Measured by the People Who Ran It Qwen3.8 27B runs at roughly 5 to 144 tokens per second depending on the hardware, and the biggest single factor is not the card: it is whether multi-token prediction is switched on. An RTX 5090 goes from 66 to 144 with one … Read more
How Much VRAM Does Nemotron 3.5 Lightning Need? About 20.1 GB at the smallest published builds, and about 26.6 GB at Q4_K_M, including an 8K cache and llama.cpp overhead. Nemotron 3.5 Lightning is NVIDIA’s 30B-A3B Mamba hybrid, published on August 11, 2026 with roughly 3.6B parameters active per token. The number most people will want … Read more
