The Calculator

TensorFold vs llama.cpp: Where the Extra Speed Comes From

by
TensorFold Mac, TensorFold DGX Spark, TensorFold vs vLLM, Qwen3.8 27B Mac speed, speculative decoding Apple Silicon, exact speculative decoding, TensorFold memory VRAMCALCULATOR

What TensorFold is TensorFold is an open-source inference server for Apple Silicon and NVIDIA GPUs that speeds up decoding with draft tokens, and promises that the drafts never change a single token of the reply. It comes from @ashxhart, it is licensed under Apache 2.0 since 0.6.0 (releases up to 0.5.0 stay MIT), and it … Read more

We Ran a 27B on a Tesla T4: Three Things It Settled

by
Tesla T4 LLM, T4 27B model, q4_0 KV cache speed, Bonsai 2 27B T4, llama.cpp KV cache quantization, T4 16GB VRAM, ECC VRAM

What a Tesla T4 says about a 27B model We ran Ternary Bonsai 2 27B on a Tesla T4 and measured what it holds and what it does. Both files, four context lengths, two cache types, flash attention on, three repetitions per speed figure. Every number below is ours, taken on 19 September 2026. Three … Read more

Inco Splash: Which Macs Run It, and How Fast on Flash Next

by
Splash inference engine, Inco Splash Mac requirements, Qwen3.8 27B Mac speed, Splash LM Studio, Splash vs oMLX, Splash vs Ollama, DFlash 2 Mac

What Inco Splash is, and which Macs can run it Inco Splash is an open-source inference engine for Apple silicon that runs three model families in its 1.4.0 release, Qwen3.8 27B, Qwen3.6 35B-A3B and Qwen3.8 Flash Next, each with a DFlash 2 speculative decoding draft matched to it. It comes from Inco, the team behind … Read more

Bonsai 2 27B: What It Really Runs on an 8 to 16 GB GPU

by
Bonsai 2 27B VRAM, Ternary Bonsai 2 27B, Bonsai 2 27B 8GB GPU, Bonsai 2 27B GGUF, PTQ1_0, PQ2_0, PrismML llama.cpp fork, Bonsai 2 27B context length

What Bonsai 2 27B needs on your GPU 5.9 GB is the weights. It is not what your card needs. PrismML's Bonsai 2 27B, released on 17 September, is Qwen3.8 27B compressed to ternary weights, and the launch announcement leads with the same footprint. That figure is real: the smaller of the two ternary files … Read more

DeepSeek V4.1 Flash: Long Context Stopped Costing VRAM

by
DeepSeek V4.1 Flash VRAM requirements, DeepSeek V4.1 Flash KV cache, DeepSeek V4.1 Flash vs V4 Flash, DeepSeek V4.1 Flash size, causal encoder decoder, CSA2 sparse attention

What DeepSeek V4.1 Flash actually costs to run A million tokens of context costs 0.87 GiB. That is the whole story, and it is the first time we have run these numbers and found the context column boring. DeepSeek publishes the figure directly in the model card: the global KV cache is 890 bytes per … Read more