Preview — Pro guide
You are seeing a portion of this guide. Sign in and upgrade to unlock the full article, quizzes, and interview answers.
Thread Pool Pattern: Fixed, Cached, Scheduled, and Work-Stealing Executors
Thread pools are the primary concurrency primitive in production Java and Python systems — they prevent thread explosion, manage CPU resources, and control backpressure. This guide covers thread pool mechanics, the four Java ExecutorService variants, rejection policies, work-stealing queues (ForkJoinPool), and the deadlock patterns that trap candidates at Meta, Google, and Amazon LLD rounds.
Why Thread Pools Are the Foundation of Concurrent Systems
Creating and destroying threads is expensive: each thread allocates ~1MB of stack memory and requires a context switch on creation. An application that creates a new thread per request will exhaust memory at moderate load (1000 concurrent requests = 1GB of stack memory) and spend more time on thread lifecycle than on actual work.
Thread pools solve this by pre-creating a fixed set of threads that pick up tasks from a shared queue. The key properties:
- Thread reuse: threads live across many tasks, eliminating creation/destruction overhead
- Work queue: tasks wait in a bounded queue when all threads are busy (backpressure)
- Lifecycle management: a single shutdown() call drains the queue and terminates threads
Every production server — HTTP servers, database connection pools, background job workers, batch processors — uses a thread pool internally. Understanding thread pool design is understanding how concurrent systems work at their core.
Thread pools appear in LLD interviews in two forms:
- Direct: "Design a thread pool implementation from scratch"
- Applied: "Design a web crawler / background job system / rate limiter" — the correct answer includes a thread pool for concurrency management