Preview — Pro guide
You are seeing a portion of this guide. Sign in and upgrade to unlock the full article, quizzes, and interview answers.
RAFT — Retrieval-Augmented Fine-Tuning: When RAFT Beats RAG and When It Does Not
RAFT (Stanford / industry follow-ons) trains models to ignore distractor documents and cite the right passages — closing the gap where vanilla RAG retrieves noise and the model hedges. This guide covers the distractor-augmented training recipe, comparison to supervised fine-tune without retrieval, evaluation on open-book QA, and failure modes when your doc corpus drifts faster than retrain cadence.
The Problem RAFT Targets
**RAG** retrieves documents and **conditions** generation on them — but the model may **attend to distractors** (irrelevant retrieved chunks) or **refuse** to commit when context conflicts. **Vanilla fine-tuning** without retrieval **bakes stale facts** into weights. **RAFT-style training** (Hsieh et al., 2024 lineage — *Retrieval-Augmented Fine Tuning*) **supervises** the model on **prompts that include both oracle documents and distractors**, teaching **discrimination** plus **answer quality** in one objective — closer to **exam with open notes** training than either pure RAG or pure SFT alone.