Preview — Pro guide
You are seeing a portion of this guide. Sign in and upgrade to unlock the full article, quizzes, and interview answers.
Sections
Related Guides
MLSD Case Study: Graph-Aware Spam Detection
ML System Design
ML System Design: Social Feed Ranking System
ML System Design
Imbalanced Classification: Metrics, Class Weights, SMOTE, and Threshold Tuning
Machine Learning
Probability Calibration: When Your Model's Probabilities Actually Mean Something
Machine Learning
ML System Design: Notification Ranking System
ML System Design
MLSD Case Study: Multimodal Content Moderation Systems
Design a TikTok/YouTube/Meta-style content moderation stack with multimodal models, policy-aware inference, human-in-the-loop review, and continuous policy evolution. Covers latency tiers, precision/recall tradeoffs by harm class, and model-policy coupling.
Problem Framing: Moderation Is Risk Management, Not Binary Classification
Content moderation systems optimize for asymmetric risk under resource constraints: missing severe harm (CSAM, credible threats, self-harm promotion) has catastrophic legal, ethical, and reputational costs that dwarf the cost of removing borderline content. But over-removal damages creator trust, suppresses legitimate speech, and creates a chilling effect that degrades platform health over time. Neither extreme is acceptable at scale.
The fundamental design challenge is that "content moderation" is not a single problem — it is a portfolio of distinct problems with different risk tolerances, label availability, and serving requirements:
- CSAM / extreme harm: maximum recall, zero tolerance for FN, automated blocking plus mandatory legal reporting.
- Coordinated inauthentic behavior: precision-recall balance, network signals matter more than content.
- Borderline speech / satire: high precision required before hard action; context and regional policy variation matter enormously.
- Spam / low-quality content: balanced FP/FN, bulk handling, cost-sensitive.
Strong interview answers separate three layers that must be designed and maintained independently:
- Policy layer: defines the rulebook, severity tiers, and regional/legal variants. Changes here invalidate model training data retroactively.
- Model layer: estimates violation likelihood across modalities (text, image, video, audio) for each policy class using separate or multi-task models.
- Operations layer: decides auto-action vs. human review escalation based on confidence, severity, SLA, and reviewer bandwidth as a resource constraint.
A design that conflates these three layers will fail silently when policy changes — the model keeps flagging by old rules while the policy has moved on.