Why it matters

Fine-tuning economics affect what's feasible. Understanding shapes strategic decisions.

Advertisement

The architecture

Full fine-tune: expensive; scales with model size.

LoRA: much cheaper; small trainable params.

QLoRA: cheapest; 4-bit base + LoRA.

Fine-tune cost tiersFull fine-tune$1000s for 7BLoRA10x cheaperQLoRA50x+ cheaperQLoRA fine-tune of 70B model possible on single 24GB GPU for hours + <$50
FT cost comparison.
Advertisement

How it works end to end

Rules of thumb: 7B full fine-tune ~$1000+ on A100s. LoRA ~1/10th. QLoRA ~1/50th.

Data: more data = more time linearly.

Epochs: usually 1-3 for LLM fine-tune.

Cost sources: compute + storage + data prep.