It answers from the passage — or admits it can't.
Google's Gemma 2 2B, fine-tuned with QLoRA on the same 10,000 synthetic legal and financial Q&A pairs used to train the 125M model built from scratch on this site. A quarter of those examples pair a question with passages that do not answer it, so the model learns to say so rather than invent something.
Same data, same eval, 20× the parameters
Both models were trained on the identical 10,000-pair dataset and scored on the identical 506 held-out rows. The only variable is the pretrained backbone.
How it was built
Four steps, about five dollars.
1 · Re-tokenize
The dataset was reused unchanged, but re-tokenized: Gemma's vocabulary is 256,000 tokens against the custom 16,384 used by the from-scratch model. Gemma 2 has no system role, so the system prompt is folded into the user turn.
2 · QLoRA
The base model is frozen and quantized to 4-bit NF4; only rank-16 LoRA adapters train — 1.28% of parameters. Loss is computed on answer tokens only, so the model is never scored on reproducing the question or the passage.
3 · Eight GPUs
A calibration run measured 10.5K tokens/sec on one H100. Eight of them reached 83.6K — 99.5% scaling efficiency — putting a 2-epoch run at 4.6 minutes.
4 · Serve
The adapter is merged back into bf16 weights and served from a single L4 that
sleeps when idle. Generation stops on <end_of_turn>, Gemma's
turn terminator — not on <eos>.
Playground
Running live on the fine-tuned weights — no cached responses.