Preview — Pro guide
You are seeing a portion of this guide. Sign in and upgrade to unlock the full article, quizzes, and interview answers.
Sections
Related Guides
Embeddings & Vector Databases: ANN Search at Scale
ML System Design
Feature Stores: Online/Offline Architecture & Training-Serving Consistency
ML System Design
Offline vs Online Evaluation: Why Metrics Disagree and What to Do About It
ML System Design
ML Monitoring & Drift Detection: Keeping Models Healthy in Production
ML System Design
Two-Stage Retrieval & Ranking: The Architecture Behind Every Large-Scale Recommender
The dominant architecture powering Google, YouTube, TikTok, Pinterest, and Spotify — two-tower retrieval followed by multi-stage ranking. Covers the fundamental constraint that makes this necessary, in-batch negatives, hard negative mining, and the full 4-stage production pipeline.
Why You Can't Score Everything — The Scale Impossibility
The fundamental tension in large-scale recommendation: you want to find the most relevant items from a corpus of 100M–1B items for each user, but you cannot afford to run an expensive model on all of them per request.
At YouTube scale: 2B users, 800M videos. If scoring one video for one user takes 0.1ms (fast!), scoring all 800M videos takes 22 hours per request. That's the brute-force impossibility.
The only solution: a cascade of stages where each stage dramatically narrows the candidate set before the next, more expensive stage. This is the two-stage (or multi-stage) retrieval-ranking paradigm.
The trade-off at each stage: recall vs precision. The retrieval stage optimizes for recall — it must not miss any item the user would love. The ranking stage optimizes for precision — it must put the best items at the top. You can afford to be imprecise at retrieval (it's fast and cheap) but you cannot miss good candidates, because items not retrieved cannot be ranked.
The key metric distinction: retrieval is evaluated on Recall@K (did the ground truth item appear in the top K retrieved?). Ranking is evaluated on NDCG@10 or Precision@1 (is the highest-quality item at the top?).