Ornith 1.5 9B Runs at 16.3 tok/s on a Card From 2017
We put Ornith 1.5 9B on a Radeon RX 580, a nine year old 8 GB card, and it generated at 16.3 tokens per second at Q4_K_M with a 32,000 token context. On a 2024 Intel laptop it managed 8.6. Then we handed the model the correct specifications and asked it to write this article about itself.
The article below the line is its output, unedited. It got two things wrong. We have marked both rather than fixing them, because the errors are the point. Further down is a third run of the same model with our style rules deleted, which finished faster, looked better and got more wrong.
The Machines
| Desktop | Laptop | |
|---|---|---|
| GPU | Sapphire Nitro RX 580, 8 GB GDDR5 | Intel Arc Xe integrated, 9 GB shared |
| CPU | Core i5-6600 | Core Ultra 7 155H |
| RAM | 16 GB DDR4 2133 | 16 GB LPDDR5X soldered |
| OS | Ubuntu 26.04 LTS | Windows 11 IoT Enterprise LTSC 2024 |
Both ran llama.cpp with the Vulkan backend, the same Q4_K_M weights, and a 32,000 token context. One run each, measured as it ran.
The Numbers
| RX 580, rules | Ultra 7, rules | Ultra 7, no rules | |
|---|---|---|---|
| Prompt processing | 101.4 | 115.9 | 124.8 |
| Generation | 16.3 | 8.6 | 8.4 |
| Article delivered | yes | no | yes |
All figures in tokens per second. The third run is the same laptop with our style rules deleted, covered further down.
The laptop reads faster and writes at half the speed. It processes a prompt 14% quicker than the RX 580 and then generates at 8.6 against 16.3, a ratio of 1.9 to 1.
Those two results are the same fact seen twice. Prompt processing is compute bound, and the laptop's Arc iGPU is a 2024 part with matrix hardware the RX 580 has never had. Generation is bandwidth bound, because every token requires reading the whole active weight set out of memory, and there the RX 580 has 256 GB/s of dedicated GDDR5 against roughly 120 GB/s of LPDDR5X that the laptop shares with its CPU.
Generation speed is what people quote and prompt speed is what you actually wait for, so which of these matters depends entirely on whether you paste in long documents or type short questions.
What We Asked It To Do
We gave the model the correct specifications for the Ornith 1.5 family, taken from the model cards and our own catalogue entries, plus Ornith's published benchmark table for the 397B and a list of eleven graphics cards with their real memory sizes. Then our house rules: no em dashes, a list of banned phrases, no invented benchmark scores or prices, the answer in the first two sentences.
We supplied the facts on purpose. Ornith 1.5 was released days before this test, so the model cannot know anything about itself, and a test of whether it can recall unknowable facts proves nothing. The question worth asking is different: handed correct information, can a 9 billion parameter model on a cheap card produce publishable technical writing?
The Laptop Never Finished
The RX 580 delivered the article in one pass. The laptop, running the same weights and the same prompt, never produced one. It reasoned for 15,890 words and was still revising its draft when llama.cpp stopped it:
request (32784 tokens) exceeds the available context size (32768 tokens)
It talked itself out of the entire 32,000 token window without writing a publishable line. The RX 580 spent 12,906 words thinking before it wrote, which is nine words of deliberation per word of article, so it was close to the same wall and cleared it.
The cause was our prompt, not the hardware. We asked for 1,200 to 1,500 words. A model cannot count its own words without writing them, so a length target becomes an audit loop: draft, count, add forty words, recount. The laptop transcript says "count" 45 times. We have removed the word target from the prompt we use for these runs. The article's shape, the headings, the two tables and the five FAQ answers, sets the length on its own.
We then ran it a third time on the same laptop with every style rule stripped out. It finished. That section is below the model's output.
The unedited output starts here
Everything between these two lines was written by Ornith 1.5 9B at Q4_K_M on the RX 580. It is the complete output, all 1,464 words of it. Nothing is cut, reordered or reworded. The only change is that its heading levels are demoted so they sit under ours, and our corrections are inserted as indented notes after the sentences they correct.
Ornith 1.5 and Your Graphics Card
The 9B model fits an 8 GB card. The 35B fits a 24 GB card with very little spare, and the 397B fits no consumer card.
The Model Family
Ornith 1.5 is a family of three open source models. They released on 18 August 2026. Each model is MIT licensed. Each has a 262,144 token context window and a vision encoder. The vision encoder lets the models take images alongside text. That is true for all three sizes.
The three sizes are a 9B dense model, a 35B mixture-of-experts model with 3B active parameters per token, and a 397B mixture-of-experts model. The 35B and 397B are mixture-of-experts models. They carry many experts. Only a few experts work on each token. The 35B uses 256 experts and activates 8 per token. The 397B uses 512 experts and activates 10 per token.
A mixture-of-experts model splits its work among many small specialists. Each token routes to a small group of experts. The dense 9B does not have experts. Every parameter in the 9B works on every token. This is a different design. The 9B is smaller but dense.
The shared architecture puts full attention on one layer in four. The other three layers hold a fixed-size state. That state does not grow with context. This design is what makes long context cheap on memory. Full attention reads the entire prompt before answering. The state layers return a fixed summary regardless of how long the prompt gets. The 262,144 token window is the addressable span for all three models. The 7.2 GB, 22.1 GB and 230 GB figures are measured at 32,000 tokens. Running the full 262,144 token window costs more memory than those base figures, but the cost stays low because context lives in just one layer in four.
Why Context Is Cheap
Full attention looks at every token. The state layers do not. Only one layer in four stores context, so a longer prompt adds memory at a low rate. The 35B keeps 22.1 GB at 4-bit quantization with a 32,000 token context. Extending the context from 8,000 tokens to 128,000 tokens costs about 2.6 GB extra. That is a small price for a large context window.
A smaller quantization buys headroom or a longer context on a card you already own. It never moves the 397B onto a consumer card, though. The full-size models stay where they are regardless of precision. Mixed precision buys similar headroom. It changes the numbers on a card you can already run.
What Fits Where
The 9B model is 7.2 GB at 4-bit quantization with a 32,000 token context. That fits an 8 GB card. An RX 580 8GB and an RTX 4060 both cover it. Any card with 8 GB or more holds the 9B. There is room left over on that memory for other work.
The 35B model is 22.1 GB at 4-bit quantization with a 32,000 token context. It fits a 24 GB card with very little spare. An RTX 3090 and an RTX 4090 can hold it. Both of those cards have 24 GB. The 35B also fits an RTX 5090, which has 32 GB, with more room to spare. The 24 GB cards are nearly full, so be careful with what else you run at the same time.
The 397B model is 230 GB at 4-bit quantization with a 32,000 token context. It fits no consumer card. The biggest consumer card on the list is the RTX 5090 at 32 GB. That is far too small for 230 GB. A single B300 holds it, at 288 GB. A pair of H200s holds it too, at 282 GB combined. Those are the only options that reach the size the 397B needs.
Comparison
The table below compares the three models side by side.
| Model | Parameters | Type | Full-attention layers | Total layers | 4-bit size |
|---|---|---|---|---|---|
| 9B | 9B | Dense | 8 | 32 | 7.2 GB |
| 35B | 35B | Mixture-of-experts | 10 | 40 | 22.1 GB |
| 397B | 397B | Mixture-of-experts | 15 | 60 | 230 GB |
The 35B and 397B both activate a small number of experts per token. The 35B runs 8 of 256 experts. The 397B runs 10 of 512 experts. The 9B is dense, so every parameter works on every token. The 397B carries the most parameters but runs the fewest active parameters per token. All three share the same 262,144 token context and the same one-in-four full attention layout.
ERROR 1. The 397B does not run the fewest active parameters per token. It runs the most of the two mixture-of-experts models, by roughly three times. Ornith has never published the figure, which is why our own catalogue leaves the field empty, and why we did not supply it.
What is actually true. The 35B holds that title at 3B active. For the 397B, ten routed experts across 60 layers put at least 9.5B to work on every token. That is a floor and not a figure, because the linear attention projection shapes are not documented, and we are not going to invent the last digit.
It knew. Its own reasoning, five minutes earlier: "the active params would be much larger than 3B… more parameters work per token than the 35B." Then it wrote the opposite.
How Fast It Runs
Speed depends on the model and the card. The 9B is small. It runs fast on an 8 GB consumer card. The 35B runs on a 24 GB card. It is slower than the 9B but still usable on a strong consumer GPU. The 397B is a large model. It runs on big data center cards. It is not something a consumer card runs locally. The 397B sits on a B300 or a pair of H200s.
The mixture-of-experts design changes the speed picture. Fewer active parameters per token means the 35B and 397B process each token with less compute than a dense model of similar size. That affects how many tokens move per second. The 9B is dense, so it does full work per token but stays light on memory. More spare memory on the card also helps speed, because the model can keep more in memory at once. A 32 GB card gives the 35B more headroom than a 24 GB card. The 9B runs comfortably on the smallest cards on the list.
ERROR 2. We supplied no speed data and the prompt bans inventing speeds. It wrote this section anyway, and the load-bearing sentence is wrong: spare video memory does not make a model generate faster. Once the weights and the cache fit, the surplus does nothing. Generation speed is set by how fast memory can be read.
Our own measurement is the counterexample. The RX 580 finished this article with about 0.8 GB to spare and generated at 16.3 tokens per second. The laptop had more headroom and managed 8.6, because 256 GB/s of dedicated GDDR5 beats roughly 120 GB/s shared with a CPU. Headroom lets you raise the context. It does not move tokens.
It knew here too: "I should be careful not to invent speed numbers."
Benchmarks
Ornith published benchmark figures for the 397B. These are the maker's own published numbers, not independent testing. The models compared against it are DeepSeek-V4-Flash-0731 at 284B parameters, GLM-5.2 at 753B parameters, and Claude Opus 4.8. GLM-5.2 has nearly twice the parameters of the 397B. Claude Opus 4.8 is a closed frontier model. Its parameter count has never been published.
| Task | Ornith 1.5 | Ornith 1.0 | DeepSeek-V4-Flash | GLM-5.2 | Claude Opus 4.8 |
|---|---|---|---|---|---|
| SWE-bench Verified | 86 | 82.4 | 81.6 | 83 | 85.8 |
| DeepSWE | 56 | 8 | 54.4 | 46.2 | 59 |
| Toolathlon-Verified | 71.2 | 43.2 | 70.3 | 48.2 | 76.2 |
| GPQA Diamond | 92.8 | 88.1 | 91.4 | 91.2 | 93.6 |
| Humanity's Last Exam (no tools) | 44.6 | 30.2 | 35 | 40.5 | 49.8 |
| BrowseComp | 86.6 | 79.7 | 84.8 | 85.6 | 84.3 |
| Frontier-Bench v0.1 | 13.5 | 2.7 | 6.1 | 5.1 | 21.1 |
On SWE-bench Verified, the closed model Claude Opus 4.8 scores 85.8, just below Ornith 1.5. On Toolathlon-Verified, Claude Opus 4.8 scores 76.2, above Ornith 1.5. On BrowseComp, the lead is close. On GPQA Diamond, Claude Opus 4.8 edges Ornith 1.5 by a small margin. On every task, Ornith 1.5 scores higher than the older Ornith 1.0. The jump is largest on DeepSWE. Ornith 1.0 scored 8 there. Ornith 1.5 scores 56.
These are the maker's published figures. Treat them as the model's own report, not a third party audit. The 397B is the only model with published numbers. The 9B and 35B do not have published benchmark scores in these figures.
FAQ
Does the Ornith 1.5 35B fit on an RTX 4090? Yes. The 35B is 22.1 GB at 4-bit. An RTX 4090 has 24 GB. It fits with very little spare. Context extension adds only about 2.6 GB from 8,000 to 128,000 tokens, so longer prompts still fit.
Can I run the Ornith 1.5 397B on a single RTX 5090? No. The 397B is 230 GB at 4-bit. An RTX 5090 has 32 GB. That is far too small. A smaller quantization buys headroom, but it never moves the model onto a consumer card. The 397B needs a B300 or a pair of H200s.
How much VRAM does the Ornith 1.5 9B need? It is 7.2 GB at 4-bit with a 32,000 token context. An 8 GB card covers it. An RX 580 8GB or an RTX 4060 works. You have room left over for other tasks.
Will the Ornith 1.5 35B run on a 24 GB RTX 3090? Yes. The 35B is 22.1 GB at 4-bit. An RTX 3090 has 24 GB. It fits with little spare. Longer prompts cost only a couple of gigabytes, so the card stays usable.
Does the Ornith 1.5 397B fit on two H200 GPUs? Yes. Two H200s give 282 GB combined. The 397B needs 230 GB at 4-bit. Both GPUs are enough. That leaves about 52 GB of headroom to spare.
The unedited output ends here
What It Got Right
Everything we handed it, it transcribed accurately. Parameter counts, layer counts, expert counts, the licence, the context window, the architecture, and all 35 numbers in the benchmark table.
It also handled the benchmark rules exactly. It said Ornith published the figures, it never once attached a 397B score to the 9B or the 35B, and it closed that section by stating outright that the two smaller models have no published scores. That is the failure mode we most expected, and it did not happen.
The one number it worked out for itself is correct: two H200s hold 282 GB, which leaves 52 GB spare against the 397B's 230.
The best thing in this run is not in the article at all. We gave it "35B with 3B active per token" in one line and "256 experts, 8 active" in another, and it tried to reconcile them. It divided 35B by 256 experts, got 137 MB per expert, multiplied by the 8 active and arrived at 1.1 billion active parameters. Then it compared that to the 3 billion we had given it, decided its own maths must be wrong, and wrote: "Let me not compute this. I'll just report the given facts."
Its arithmetic was right. The expert contribution really is about 1.05 billion parameters per token. What it missed is that a mixture-of-experts model also runs an always-on part on every token: the embeddings, the ten full attention layers, the thirty linear state layers and the routers, roughly 2 billion parameters that are never switched off. 1.05 plus 2 is the 3 billion on the model card. It was one step away and stopped.
Stopping was the right call. A model that publishes 1.1 billion because its own division said so is worse than one that reports what it was told, and the second machine did exactly that: the laptop transcript contains "8 × 256 = 2048B = 2B active", which multiplies a count of experts by a count of activations and reads the result as parameters.
What Both Errors Have in Common
Neither is in the material we supplied. Both appear where the model needed a fact it did not have.
Both are also contradicted by its own reasoning. On the 397B's active parameters, the transcript reads "the active params would be much larger than 3B… more parameters work per token than the 35B", and the article says the opposite. On speed, the transcript reads "I should be careful not to invent speed numbers", and then it wrote 200 words of speed advice.
That is worth more than the errors themselves. The model knew both times. The failure is not in the thinking, it is in the gap between the thinking and the page, and nothing in the finished article marks where that gap is.
The Same Model on the Ultra 7, With Every Rule Removed
We ran it once more on the Core Ultra 7 155H laptop with the facts unchanged and the entire style section deleted. No word target, no banned phrases, no dash ban, no instruction about who the benchmark scores belong to.
It finished, in 984 words, after 6,178 words of thinking instead of 15,890. Same laptop, same weights, same 32,000 token context. It processed the prompt at 124.8 tokens per second and generated at 8.4, against 115.9 and 8.6 on its ruled run. That settles what stopped the second run: the rules were the load, not the hardware.
It is also the better looking article. Left alone, it built a fit matrix crossing all eleven cards against all three models, which is a more useful table than the one the constrained run produced. It was the only run that tried to work out what a 128,000 token context does to the 35B, and it is the only run that got a number wrong by doing arithmetic we had made impossible.
Then it made worse mistakes.
Everything between these two lines was written by Ornith 1.5 9B at Q4_K_M on the Intel Core Ultra 7 155H laptop, with our style rules removed. It is the complete output, unedited. Heading levels are demoted and our corrections are inserted as indented notes.
Ornith 1.5: A Longer-Context Model That Actually Fits on Your GPU
On 18 August 2026, the Ornith lab released Ornith 1.5 — a family of three open-source models, and with them a claim that the frontier of capability is no longer gated by how big your GPU is.
The family
Ornith 1.5 comes in three sizes:
- 9B — a dense model
- 35B — a mixture-of-experts (MoE) model
- 397B — a mixture-of-experts model
All three are released under the MIT license, share a 262,144-token context window, and include a vision encoder. The 397B is the flagship; its published numbers place it among the strongest models currently on the market.
The architecture: context that doesn't cost memory
Most models store every token of a conversation in attention, so a longer context means a larger memory footprint. Ornith takes a different approach. One layer in every four uses full attention; the other three layers hold a fixed-size state that does not grow with context.
The result is a long context for a tiny memory price. Because only a quarter of the layers store tokens, expanding the 35B from an 8,000-token prompt to a 128,000-token one adds only about 2.6 GB. That is what makes the 9B and 35B viable on everyday hardware — and it is the 397B's headline result that puts the whole family in the top tier.
The 35B routes 8 of its 256 experts per token — roughly 3 billion active parameters per token — giving it the breadth of a very large model while keeping compute focused.
What fits on what
With the weights plus a little room for the key-value cache, the fit is straightforward. Here is where each model lands across the cards in question:
| GPU | Memory | Ornith 9B | Ornith 35B | Ornith 397B |
|---|---|---|---|---|
| RX 580 8GB | 8 GB | ✓ | ✗ | ✗ |
| RTX 4060 | 8 GB | ✓ | ✗ | ✗ |
| RTX 3060 12GB | 12 GB | ✓ | ✗ | ✗ |
| Arc B580 | 12 GB | ✓ | ✗ | ✗ |
| RTX 4060 Ti 16GB | 16 GB | ✓ | ✗ | ✗ |
| RTX 3090 | 24 GB | ✓ | ✓ (tight) | ✗ |
| RTX 4090 | 24 GB | ✓ | ✓ (tight) | ✗ |
| RTX 5090 | 32 GB | ✓ | ✓ | ✗ |
| A100 80GB | 80 GB | ✓ | ✓ | ✗ |
| H200 | 141 GB | ✓ | ✓ | ✗ (needs two) |
| B300 | 288 GB | ✓ | ✓ | ✓ |
The 9B, at 7.2 GB in 4-bit quantization for a 32,000-token context, runs on any 8 GB card. The 35B, at 22.1 GB, needs a 24 GB card and fits with only about 2 GB to spare — so the 128,000-token context noted above would push it past that card's limit. The 397B is a different category.
FREE ERROR C, and this one is our fault. 22.1 GB is the figure at a 32,000 token context. The 2.6 GB is the cost of going from 8,000 tokens to 128,000. Adding them counts the 8,000 to 32,000 stretch twice.
The real ladder, from our own engine: 21.60 GiB at 8K, 22.07 at 32K, 23.99 at 128K. So the correct sum is 21.6 plus 2.4, and the 35B at a full 128,000 token context lands at 23.99 against a 24 GiB pool. It does miss a 24 GB card, which is where this paragraph ends up, but not for the reason given and not by the margin given.
We never supplied the 8,000 token baseline, so this sum could not be done correctly from the facts in front of it. The right move was to not attempt it. Both ruled runs left it alone, which spared them an error rather than earning them a point.
At 230 GB in 4-bit quantization, the 397B doesn't fit on any single card on the list. The largest, the RTX 5090 at 32 GB, holds only about one-seventh of it. It needs a B300 (288 GB), or a pair of H200s (141 GB each, 282 GB combined). A single H200 falls short.
Benchmarks
The lab published these figures for the 397B model. They are the maker's own numbers, not independent testing.
| Benchmark | Ornith 1.5 | Ornith 1.0 | DeepSeek-V4-Flash | GLM-5.2 | Claude Opus 4.8 |
|---|---|---|---|---|---|
| SWE-bench Verified | 86 | 82.4 | 81.6 | 83 | 85.8 |
| DeepSWE | 56 | 8 | 54.4 | 46.2 | 59 |
| Toolathlon-Verified | 71.2 | 43.2 | 70.3 | 48.2 | 76.2 |
| GPQA Diamond | 92.8 | 88.1 | 91.4 | 91.2 | 93.6 |
| Humanity's Last Exam (no tools) | 44.6 | 30.2 | 35 | 40.5 | 49.8 |
| BrowseComp | 86.6 | 79.7 | 84.8 | 85.6 | 84.3 |
| Frontier-Bench v0.1 | 13.5 | 2.7 | 6.1 | 5.1 | 21.1 |
The 397B is compared against DeepSeek-V4-Flash-0731 (284B parameters), GLM-5.2 (753B parameters), and Claude Opus 4.8 — a closed model whose parameter count has never been published. GLM-5.2 carries nearly twice as many parameters as Ornith's 397B.
The picture is clear. On most benchmarks Ornith 1.5 397B leads the two open models it competes against and nearly matches Claude Opus 4.8:
- SWE-bench Verified: 86 (top)
- BrowseComp: 86.6 (top)
- DeepSWE: 56 (top)
- GPQA Diamond: 92.8 (just below Claude's 93.6)
- Humanity's Last Exam: 44.6 (below Claude's 49.8)
The two places Claude stays ahead are the hardest suites — Frontier-Bench v0.1 (Claude 21.1, Ornith 13.5) and Toolathlon-Verified (Claude 76.2, Ornith 71.2) — where closed-model training apparently still matters.
FREE ERROR A. Claude Opus 4.8 scores 59 on DeepSWE, in the table immediately above this line. Ornith's 56 is not top. And Claude is ahead in five places, not two: DeepSWE, Toolathlon, GPQA Diamond, Humanity's Last Exam and Frontier-Bench.
It contradicts itself three lines earlier, where its own bullet list correctly puts Ornith below Claude on GPQA and on Humanity's Last Exam. Every number it needed was on the screen. This is not a gap in what it knew.
The jump from Ornith 1.0 is large on every benchmark. The most dramatic is DeepSWE: Ornith 1.0 scored 8, and Ornith 1.5 scores 56 — a roughly sevenfold leap, far steeper than the gains elsewhere.
What it means
Ornith 1.5 is a strong open-source result. It demonstrates that with the right architecture — hybrid attention, expert routing, and a long context that doesn't inflate memory — an MIT-licensed model can sit squarely between the smaller frontier models and the strongest closed ones. The 9B and 35B bring frontier-grade reasoning to the 8–24 GB class of cards; the 397B shows what a fully open model can do at the very top of the table.
FREE ERROR B. The 9B and the 35B have no published benchmark scores at all. Every figure in this article belongs to the 397B, which needs a 288 GB card. We stated that in the facts we handed it, and under our rules the same model closed its benchmark section by saying so. With the rules removed it wrote the opposite, twice, here and in "puts the whole family in the top tier".
This is the most consequential error in the experiment. It is the one that could put somebody on a download believing a 7.2 GB model carries a 92.8 on GPQA Diamond.
The one thing the comparison table can't tell you is whether those maker-quoted numbers hold up under independent testing — but on the benchmarks published, Ornith 1.5 is one of the best open models on the market.
What Freedom Cost
15 em dashes and one en dash, against zero in both ruled runs. That single tell is what most readers now use to decide a page was machine-written, and no amount of correct arithmetic buys it back.
The generalisation we were most afraid of, twice. Our facts said in plain words that every benchmark score belongs to the 397B and to nothing else. Under the rules, the RX 580 run closed its benchmark section by stating outright that the 9B and 35B have no published scores. With the rules removed, the same model wrote that the family sits "in the top tier" and that the 9B and 35B bring frontier-grade reasoning to ordinary cards. Nothing in the data supports that. It is the single sentence in this whole experiment that could mislead somebody into a download.
Two errors against its own table, in a paragraph that contradicts the bullet list three lines above it. That is the finding worth keeping, because it is not a knowledge failure. Every number it needed was on screen, and it still summarised them wrongly.
And one error we set it up for. It added a memory figure measured at 32,000 tokens to a growth figure measured from 8,000, which double counts the gap between them. The right baseline was never in the prompt. Neither ruled run attempted the sum, so neither was caught by it, and that is luck rather than judgement. The prompt now carries the whole ladder.
The ruled version is worse to look at and safer to publish. The free version would pass a casual read and fail a fact check.
What This Says, And What It Does Not
This is one 9 billion parameter model, at 4-bit, on an 8 GB card and a 9 GB shared pool, with no tools and no network. Nothing here transfers to a larger model, to a stronger card, or to the same weights running at 8-bit. It says nothing about what this model does inside an agent loop with a search tool, which is a different machine doing a different job.
Within that box, three things held across all three runs. It transcribed every supplied fact correctly. It followed the mechanical rules exactly when given them. And it invented, in the same confident tone, wherever we left a gap.
The practical version, for this size of model on this class of hardware: supply every fact you need, name explicitly every category you do not want invented, and check every sentence carrying a number you did not provide. If the runtime shows you the reasoning, read it, because in the ruled run it flagged both of its errors before making them.
FAQ
How fast is Ornith 1.5 9B on an RX 580? 16.3 tokens per second generating and 101.4 processing the prompt, at Q4_K_M with a 32,000 token context, using llama.cpp with the Vulkan backend.
How fast is Ornith 1.5 9B on an Intel Core Ultra 7 155H iGPU? 8.6 tokens per second generating and 115.9 processing the prompt, at Q4_K_M with a 32,000 token context, on llama.cpp with the Vulkan backend and a 9 GB shared memory pool. A second run on the same laptop gave 8.4 and 124.8. Generation is about half what the RX 580 manages, because an integrated GPU reads its weights over system memory it shares with the CPU. Prompt processing is the faster of the two, since that part of the work is bound by compute rather than by bandwidth.
Can an 8 GB graphics card run Ornith 1.5 9B? Yes. Our calculator puts it at 7.17 GiB at Q4_K_M with a 32,000 token context, which clears an 8 GB card with room for the display.
Is an old RX 580 faster than a new Intel laptop for local AI? For generating text, yes, by 1.9 times, because it has 256 GB/s of dedicated memory while the laptop shares system RAM with the CPU. For reading a prompt the laptop is 14% faster, since that part of the work is bound by compute rather than bandwidth.
Did the model write this whole article? No. It wrote the section between the two marked lines. Everything else is ours, including the corrections.
What changes if you remove the style rules? On our laptop run it finished faster and produced a better looking article, then made three factual errors instead of two, including attributing the 397B's benchmark scores to models that have none. It also used 16 em dashes against zero under the rules.