Chain-of-Thought, Test-Time Compute & Multi-Step Reasoning

Chain-of-thought (CoT) prompting (Wei et al., 2022) elicits intermediate natural-language steps before the final answer. It improves many multi-step tasks (math word problems, structured logic, some coding) because the autoregressive model spends more serial decode steps on the generation plane before committing to a label — a form of implicit test-time work, not a proof.

Senior interviews expect you to stop at "add think step by step." You must connect CoT to cost and P95 latency (operations), sampling variance and judge design (evaluation), overthinking and systematic algebraic mistakes (reliability), and user-visible vs. internal scratchpads (product, legal). Reasoning-tuned models (industry: RL or process-supervised post-training on math/code) push the same trade as test-time compute — spend more wall-clock at inference to raise quality on the hard tail, without always increasing parameter count (typical framing since ~2024).

Self-consistency (Wang et al., 2022) samples k independent CoT completions (non-zero temperature or nucleus sampling), parses a final answer from each chain, and returns the majority answer. It cuts uncorrelated slip errors on tasks with a discrete, canonical final label (e.g. normalized numerics, multiple-choice letter).

Voting fails when: (1) the output is free-form and there is no stable equality on answers; (2) all chains make the same systematic error (violated independence); (3) the SLA is too tight for k full decodes; (4) you need calibrated probabilities, not a mode. LLM-as-judge to pick the best chain is better than nothing but reintroduces position bias and model blind spots — prefer verifiers (Python, SQL, typecheck) wherever the environment allows.

Test-time compute is the 2024+ industry term for: spend more inference work (longer internal chains, more samples, or search) to improve the per-query outcome. Reasoning-tuned models (post-trained with RL on outcome labels, process supervision, or long CoT distillation on math/code) often raise ceilings on contest math and code without matching parameter scaling alone.

In PEARL terms: the L (latency / cost) interview must include routing — only route to an expensive reasoning tier when a cheap classifier, heuristic, or a first-pass cheap model flags hard. Track $ per resolved task, not only $/1M tokens.