Time Series Forecasting: ARIMA, Prophet, LightGBM, and Deep Learning

Time series forecasting is the one ML problem where your instinct to "just throw a neural net at it" is almost always wrong. The M4 competition (Makridakis 2018, 100K series) and M5 competition (2020, Walmart sales) both demonstrated that the winning approaches were either statistical models (ETS, ARIMA) ensembled with ML, or gradient-boosted trees on carefully engineered lag features — not deep learning. The 2020 M5 winner used LightGBM with lag, rolling, and calendar features, beating every neural architecture submitted.

Three properties of time series break the IID assumption that powers most ML:

1. Temporal dependence. Observations are correlated with their own past. You cannot shuffle-and-split a time series. A random 80/20 split leaks future information into training — the model sees tomorrow's seasonality pattern while predicting yesterday. The only valid evaluation is expanding or rolling-window backtest with a strict temporal cutoff.

2. Non-stationarity. The statistical properties (mean, variance, autocovariance) drift over time. Classical models like ARIMA require stationarity and force you to explicitly difference the series. Modern ML models tolerate non-stationarity but still degrade silently when the generating process changes — COVID-19 broke every pre-2020 retail forecast.

3. Autocorrelation in residuals. OLS assumes independent errors. Time series residuals are auto-correlated, which invalidates standard confidence intervals and p-values. This is why naive sklearn regression on t → y reports overconfident, wrong metrics.

The interview trap: a junior candidate says "I'd use an LSTM." A senior says "I'd start with ETS or Prophet as a baseline, engineer lag and calendar features, try LightGBM, and only reach for DeepAR or N-BEATS if I have many series and exogenous regressors."