QLoRA VRAM Requirements: Why Every Number You Find Is Different
How Much VRAM Does QLoRA Need?
For a 7B model, published answers range from 5 GB to 20 GB, and all of them are defensible. Unsloth says 5 GB. Modal says 5 GB. Spheron says about 8 GB. RunPod says 8 to 10 GB. Introl says 12 to 20 GB. That is a 4x spread on the most searched number in fine-tuning, for the same model at the same precision.
None of these sources is wrong. They are answering different questions and mostly not saying which. The QLoRA VRAM requirements you actually face depend on four settings that most articles never mention, and once you know what they are, the disagreement stops being confusing and becomes a calculation.
This page reconciles the published figures, shows what each one assumes, and tells you which applies to the card you own. We have not trained anything ourselves. Every number below is attributed to whoever published it.
Five Sources, Five Answers, Same Job
All five checked on August 7, 2026, for a 7-billion-parameter model with 4-bit QLoRA.
| Source | 7B QLoRA | Assumptions published? |
|---|---|---|
| Unsloth, who make the framework | 5 GB | No. Only that it is a minimum |
| Modal | 5 GB | Partly. Gives a component breakdown |
| Spheron | ~8 GB | Yes, in full |
| RunPod | 8 to 10 GB | Mentions that usage varies, without values |
| Introl | 12 to 20 GB | No |
Notice the pattern in the right-hand column. The source that states its assumptions in full is not the one with the lowest number, and the source with the widest range states nothing at all. That is not a coincidence, and it is the whole explanation.
Spheron publishes the conditions behind every figure in its table: batch size 1, sequence length 512, the AdamW optimizer, and LoRA rank 64. Its 7B QLoRA row breaks down as 3.5 GB of quantized base weights, 0.2 GB of adapter parameters, 0.2 GB of gradients, 0.6 GB of optimizer state and 4 GB of activations, reaching roughly 8 GB.
Look at which component is largest. Activations are half the total, and activations are the one part that is not fixed by the model. They scale with how long your sequences are and how many you process at once.
Why The Published Numbers Disagree
Fine-tuning memory has four parts, and only the first is a property of the model.
- Base weights. Fixed. A 7B model at 4-bit is about 3.5 GB. This is the part every source agrees on.
- Adapter parameters and their gradients. Small, but they scale with LoRA rank. Rank 64 costs meaningfully more than rank 8, and most guides never say which rank they used.
- Optimizer state. Depends entirely on the optimizer. Modal notes that AdamW keeps roughly three copies at 4 bytes per parameter, which is why an 8-bit optimizer changes the arithmetic so much.
- Activations. The variable that decides everything. They scale with batch size and sequence length, and gradient checkpointing trades compute to shrink them.
So a 5 GB figure describes the floor: short sequences, batch size 1, a small rank, activation checkpointing on. An 8 GB figure describes rank 64 at 512 tokens. RunPod reaches 8 to 10 GB by taking the 3.5 GB of 4-bit weights and adding overhead. Introl’s 12 to 20 GB describes something closer to a real training run. Its width is the tell: Introl publishes no sequence length, batch size or optimizer, and a range that wide is a range across working conditions rather than a single configuration.
Every one of them is the same model. The difference is what the person was doing with it.
The same disagreement runs through full fine-tuning, where the published figures for a 7B are 67 GB from Modal, roughly 88 GB from Spheron, and 100 to 120 GB from Introl. Nobody is off by a factor of two on arithmetic this well understood. They are counting different things.
This is the same shape as a mistake we have covered before on the inference side, where a widely shared 59 tokens per second figure turned out to be aggregate throughput across 12 concurrent requests rather than what one person experiences. The number was real. The condition attached to it did not travel. The full breakdown of that case is here.
Unsloth’s Published Table, In Full
Unsloth make the framework, so their figures are the closest thing to a vendor floor. This is their complete table as published.
| Model size | QLoRA, 4-bit | LoRA, 16-bit |
|---|---|---|
| 3B | 3.5 GB | 8 GB |
| 7B | 5 GB | 19 GB |
| 8B | 6 GB | 22 GB |
| 9B | 6.5 GB | 24 GB |
| 11B | 7.5 GB | 29 GB |
| 14B | 8.5 GB | 33 GB |
| 27B | 22 GB | 64 GB |
| 32B | 26 GB | 76 GB |
| 40B | 30 GB | 96 GB |
| 70B | 41 GB | 164 GB |
| 81B | 48 GB | 192 GB |
| 90B | 53 GB | 212 GB |
| 405B | 237 GB | 950 GB |
Their caveat, quoted exactly: “Keep in mind that sometimes more VRAM is required depending on the model so these numbers are the absolute minimum.”
Treat that sentence as load-bearing. These are floors, not budgets.
The 27B To 40B Band Costs More Per Parameter
Divide that table’s QLoRA column by the parameter count and something appears that, as of August 2026, we have not seen discussed anywhere.
| Model size | QLoRA | GB per billion parameters |
|---|---|---|
| 3B | 3.5 GB | 1.17 |
| 7B | 5 GB | 0.71 |
| 8B | 6 GB | 0.75 |
| 9B | 6.5 GB | 0.72 |
| 11B | 7.5 GB | 0.68 |
| 14B | 8.5 GB | 0.61 |
| 27B | 22 GB | 0.81 |
| 32B | 26 GB | 0.81 |
| 40B | 30 GB | 0.75 |
| 70B | 41 GB | 0.59 |
| 81B | 48 GB | 0.59 |
| 90B | 53 GB | 0.59 |
| 405B | 237 GB | 0.59 |
Read down the column. Small models carry fixed overhead, so 3B sits at 1.17. The cost then drops into a 0.61 to 0.75 band through the teens, jumps to 0.81 at 27B and 32B, and from 70B upward settles at 0.59 and stays there to 405B. The jump is the part with no obvious explanation: 14B to 27B is the only place in the table where the cost per parameter goes sharply up.
A 27B model costs about 39% more VRAM per parameter to fine-tune than a 70B does. That is not a rounding artefact. It is a step, and it sits directly under the most popular consumer card tier.
We do not know the cause and we are not going to guess at one. Vocabulary size and embedding layout differ sharply between model families in that range, and the rows may not all be measured on comparable architectures. What we can say is that the discontinuity is in the vendor’s own published figures, it is reproducible by dividing two columns, and it changes which card you need. If you know the reason, we would genuinely like to hear it.
Which Number Applies To Your Card
Using Unsloth’s floors, and then the realistic range once sequence length and rank are accounted for.
| Your VRAM | Comfortable QLoRA | The catch |
|---|---|---|
| 8 GB | 3B, and 7B at the floor | 7B works at short sequences and low rank. Raise either and you will run out |
| 12 GB | 7B to 11B | Room for longer sequences at 7B, which matters more than model size |
| 16 GB | up to 14B | Free Colab T4 sits here, so 14B QLoRA costs nothing to try |
| 24 GB | 27B, tightly | 32B is over the line. 26 GB against 24 GB, and what you give up to close it is sequence length |
| 48 GB | up to 70B | 41 GB is the floor. RunPod reports about 46 GB in actual use, so a 48 GB card fits it only when optimized |
| 80 GB | 70B comfortably | The first tier where 70B stops being a squeeze |
| 237 GB+ | 405B | Multi-GPU. Not a single-card question any more |
The 24 GB row is the one worth sitting with. An RTX 4090 or 3090 can fine-tune a 27B model and cannot comfortably fine-tune a 32B one. The step in the section above is why 27B is already tight rather than roomy: at the pre-step rate a 27B would have cost around 17 GB, not 22. Five billion more parameters then costs 4 GB, and a 24 GB card has only 2 GB of headroom left once the 27B floor is paid.
If you are choosing hardware rather than working with what you own, our GPU comparison for local AI covers the cards in these tiers. And because fine-tuning is usually a project with an end rather than a permanent need, renting by the hour is often the better answer: an 80 GB A100 that costs used-car money to own rents for a couple of dollars an hour.
LoRA Versus QLoRA, In Memory Terms
The gap is larger than most people expect. From the same table, at 7B: 5 GB with QLoRA against 19 GB with LoRA. At 70B it is 41 GB against 164 GB.
QLoRA quantizes the frozen base model to 4-bit and trains the adapter on top. LoRA leaves the base at 16-bit. You are paying roughly 4x the memory for the base weights to stay at full precision, and for most tasks people report the quality difference is small relative to that cost.
Modal’s figures for a 7B put full fine-tuning at 67 GB, LoRA at 15 GB, 8-bit QLoRA at 9 GB and 4-bit QLoRA at 5 GB. Their worked example for full fine-tuning is 14 GB of parameters plus 42 GB of optimizer state plus 14 GB of gradients, which lands near 70 GB. Spheron puts full fine-tuning of a 7B at roughly 88 GB under its own stated conditions, and Introl at 100 to 120 GB.
Every publisher here puts full fine-tuning at between 5x and 13x what QLoRA costs for the same model. Divide across publishers rather than within one and the spread runs wider still. That ratio, not the absolute numbers, is the reason almost nobody outside a lab does it.
The LoRA column disagrees too. Unsloth publishes 19 GB for a 7B, Modal 15 GB, RunPod 16 to 20 GB, and Introl 24 to 32 GB. If you are sizing a card on the LoRA figure rather than the QLoRA one, that spread matters more, because it straddles the 24 GB tier where most consumer buying decisions happen.
What Fine-Tuning Will Not Fix
Worth knowing before you size any hardware, because it decides whether you need this at all.
LoRA teaches behaviour, not facts. Style, tone, output format and task behaviour transfer well. Knowledge does not, and a model that has been fine-tuned on documents it cannot recall will confidently invent neighbouring facts instead of admitting the gap.
The same effect shows up at a much larger scale in distillation. Inkling-Small was distilled from Inkling, and reasoning behaviour carried across while factual recall did not: SimpleQA fell to 20.6% against the parent’s 43.9%.
If your goal is for a model to know your documents, retrieval will get you there faster and cheaper than fine-tuning, and it will not need any of the hardware on this page. If your goal is for a model to respond in a particular way, fine-tuning is the right tool.
What We Do Not Know
- Why the 27B to 40B band costs more per parameter. Stated above as an observation from published figures. The cause is unestablished.
- How these floors behave at long context. Every figure here assumes short sequences. Nobody publishes a table of QLoRA memory against sequence length, which is the single most useful thing missing from this topic.
- Whether the rows are architecturally comparable. Unsloth’s table lists sizes, not model names, so two rows may be different families with different vocabulary sizes.
We compute memory. We do not own training hardware and have not run these jobs, so nothing here is presented as our own measurement.
FAQ
How much VRAM do I need for QLoRA on a 7B model?
Published answers span 5 GB to 20 GB. Unsloth and Modal both give 5 GB as the floor. Spheron gives about 8 GB at LoRA rank 64, batch size 1 and a 512-token sequence length, and RunPod 8 to 10 GB. Introl gives 12 to 20 GB. Longer sequences push it higher. Budget 12 GB if you want to work without thinking about it.
Why do QLoRA VRAM requirements vary so much between sources?
Because activations are roughly half the total and they are not a property of the model. They scale with sequence length and batch size, and they shrink with gradient checkpointing. A source quoting 5 GB and a source quoting 20 GB can both be right about the same model, and neither is obliged to tell you which conditions it used.
Can I fine-tune a 32B model on a 24 GB GPU?
Not in any way you would want to use. Unsloth’s floor for 32B QLoRA is 26 GB against 24 GB on a 3090 or 4090, and that figure is already a minimum. It can be forced: Axolotl publishes a 34B QLoRA example that ran on a single 24 GB RTX 4090 at 97% memory use, and paged optimizers will spill optimizer state to system RAM when the GPU runs out. What you pay is sequence length. Unsloth’s own issue tracker has a QwQ-32B run on a 3090 where the framework capped the sequence length at 256 tokens. A 27B model at 22 GB fits properly, and that is the honest answer for a 24 GB card.
Can I fine-tune a 70B model at home?
Not on one consumer card. QLoRA on a 70B needs 41 GB at the floor, so the entry point is a 48 GB professional card or rented time on an 80 GB A100 or H100.
Is QLoRA worse than LoRA?
It uses roughly a quarter of the memory because the frozen base model is held at 4-bit rather than 16-bit. Whether the quality difference matters depends on the task, and it is a smaller difference than the memory saving would suggest.
Do I need to fine-tune to make a model know my documents?
Usually not. LoRA transfers behaviour rather than knowledge, and a fine-tuned model will often invent plausible neighbouring facts rather than recall the source. Retrieval is the better tool for knowledge and needs none of this hardware.
What can I fine-tune on a free Google Colab T4?
A T4 provides 16 GB, which clears Unsloth’s published floor for QLoRA up to 14B at 8.5 GB, so the entry cost for trying this is zero. One caveat: Google does not guarantee which accelerator a free session gets, stating only that the types available vary over time. Check what you were allocated before sizing anything around it.
What Python version does Unsloth need?
Unsloth’s documentation states Python 3.11 to 3.13 for Unsloth Studio training, and notes that Python 3.13 is supported for Unsloth Core. Check the current requirements before installing, since this has already changed once.
How much VRAM does full fine-tuning need?
For a 7B, Modal publishes 67 GB, Spheron roughly 88 GB and Introl 100 to 120 GB. Each publisher’s own ratio against its own QLoRA figure runs from about 5x to 13x, and Spheron states its own as 11x. Compare across publishers and the spread widens further. Either way it is the ratio, not the absolute number, that explains why full fine-tuning is rare outside labs.
Does this change how much VRAM I need to run the model afterwards?
No. Training memory and inference memory are separate questions with different arithmetic. Once the adapter is merged, the model costs what that model costs to run. Our calculator sizes the inference side, and the model checker tells you what your current card handles.