Expert guides on VRAM requirements, GPU benchmarks, LLM hardware, and local AI inference.
QLoRA VRAM Requirements: Why Every Number You Find Is Different How Much VRAM Does QLoRA Need? For a 7B model, published answers range from 5 GB to 20 GB, and all of them are defensible. Unsloth says 5 GB. Modal says 5 GB. Spheron says about 8 GB. RunPod says 8 to 10 GB. Introl … Read more
How Much VRAM Does Ling 3.0 Flash Need? 81.1 GB at the official FP4 build, or 86.2 GB at Q4_K_M, both including runtime overhead at an 8K context. Ling 3.0 Flash, published as inclusionAI/Ling-3.0-flash, is a 124B mixture-of-experts model with 5.1B parameters active per token, released on August 4, 2026 under an MIT license. The … Read more
LFM2.5 2.6B VRAM Requirements: What 2.69B Hybrid Actually Needs How Much VRAM Does LFM2.5 2.6B Need? 2.08 GB at Q4_K_M with an 8K context, rising to 4.40 GB at the full 128K context. Weights are 1.67 GB and do not move. What moves is the KV cache: 0.13 GB at 8K, 2.15 GB at 128K. … Read more
How to Run DeepSeek V4 Flash on DGX Spark One command, and it fits on a single 128 GB node. curl -sSL https://raw.githubusercontent.com/entrpi/ds4-on-spark/main/install.sh | bash -s — –start That is the installer from Entrpi’s ds4-on-spark, an MIT-licensed Blackwell CUDA fork of the DwarfStar engine, built specifically for the GB10. It pulls the model, builds the … Read more
How Much VRAM Does DeepSeek V4 Flash 0731 Need? DeepSeek V4 Flash 0731 VRAM requirements land between 162 and 178 GB at 4-bit, and the spread is the interesting part. DeepSeek released the official DeepSeek-V4-Flash-0731 weights on 31 July 2026 under MIT, and Unsloth published GGUF conversions the same afternoon. The 4-bit build measures 155.1 … Read more
We Said Kimi K3 Was Datacenter-Only. That Changed in Three Days. On July 27 we published what Kimi K3 requires and wrote that it was enterprise hardware regardless of quantization. Our calculator put full residency at 1,796 GB at Q4_K_M, which is five Instinct MI430X cards at 98% of their pool. Two days later Unsloth … Read more
How Much VRAM Does Inkling Need? 680 GB at Q4_K_M with an 8K context for the 975B flagship, or 193 GB for Inkling-Small, which fits a single card. Those are the two numbers most people want, because Q4_K_M is the quantization almost every local deployment reaches for first. Inkling is Thinking Machines Lab's flagship open-weights … Read more
Chinese Labs Own the Open-Weight Leaderboards. That Doesn’t Mean You Can Run Them. Open-weight model rankings in mid-2026 read like a list of Chinese labs: Z.ai’s GLM, Alibaba’s Qwen, DeepSeek, Moonshot’s Kimi. And the biggest release of the year has now landed. Moonshot published Kimi K3’s full weights on July 27, 2026, making it the … Read more
Can You Really Run a 26B Model on an RTX 4060? Yes: and there’s video to prove it. On July 24, 2026, a tester running under the handle @analogalok posted a single RTX 4060 (the 8GB card, the one that sells for around $250) running Gemma 4 26B A4B, a 26-billion-parameter Mixture-of-Experts model, with vision, … Read more
How Does a 118B Model Run on a 12GB GPU? It doesn’t: not the way you’d assume. Nobody is holding a 118-billion-parameter model entirely in 12GB of VRAM; that’s not physically possible at any reasonable quantization. What’s actually happening is expert offloading: most of the model’s weights sit in ordinary system RAM, and only a … Read more
