Preview — Pro guide
You are seeing a portion of this guide. Sign in and upgrade to unlock the full article, quizzes, and interview answers.
Non-parametric Tests: Mann–Whitney U, Kruskal–Wallis, Permutation Tests, and When Normality Fails
Revenue, dwell time, and latency are skewed — t-tests on raw values assume fragile things. This guide covers rank-based tests (Mann–Whitney, Kruskal–Wallis), exact permutation logic, median vs mean hypotheses, ties, and when robust alternatives (bootstrap, trimmed means) beat ranks for product A/B analysis.
When Classical t-Test Assumptions Lie
The two-sample t-test assumes **approximately normal** sampling distribution of the mean (often satisfied by CLT for large ) and, for the equal-variance version, **homoskedasticity**. Skewed metrics (session duration, purchase amount) with **moderate n per arm** can produce **anti-conservative or misleading** p-values when outliers dominate the mean difference. **Non-parametric** rank tests replace values with **ranks**, down-weighting extreme tails automatically. They test **stochastic ordering** (Mann–Whitney: does treatment tend to produce larger values?) rather than equality of means — that distinction trips candidates.