Walk-forward testing: why your backtest is lying to you

The objective was a trading system with a 70% win rate. I built it; I did not hit 70%. The mechanism that stopped me — out-of-sample validation — is the most useful thing I've applied all year.

In-sample vs out-of-sample

An in-sample backtest optimises a strategy against the same data it's evaluated on. With enough free parameters, any strategy can be fit to history; the result is a curve that captures noise, not edge. That's overfitting, and it's the default failure mode of strategy development.

Walk-forward analysis (WFA)

WFA slides a train/test window across the series: optimise on a training segment, evaluate on the immediately following out-of-sample segment, advance, repeat. Every trade is judged on data the optimiser never saw.

I also purge and embargo the split — drop overlapping labels and leave a gap between train and test — so no information leaks across the boundary. This ran over 4.6 years of forex and gold data. That's the gate; if a strategy can't pass it, it doesn't get capital.

What the gate killed

Three strategies, plus a daily swing variant, all dead:

Breakout — Donchian-channel momentum. Strong in-sample, degraded to noise out-of-sample in ranging regimes.
Mean-reversion — fade-to-mean. Printed profit in-sample; the "dip" kept extending out-of-sample.
Pullback — trend-continuation on retrace. ~68% in-sample, roughly break-even after costs out-of-sample.

All three produced compelling in-sample metrics and failed the out-of-sample test. That's the pattern: the backtest wasn't lying about the past; it was lying about the future.

Why strategies fail out-of-sample

Non-stationarity. A parameter set fit to a trending regime fails when the regime changes. History isn't one distribution; it's several, concatenated.
Parameter fragility. A sharp optimum rather than a plateau indicates a curve fit. I require the edge to survive a neighborhood of parameter values, not a single point.
Execution realism. Bar-close fills are optimistic. I model spread, commission, and slippage, and price event-driven entries in event time, not close time.

The validation methodology

I stopped trusting single numbers and started beating every idea to death:

Parameter sweeps over a grid, not single optima.
Pessimistic cost model — spread, commission, slippage baked in.
Bias audit — look-ahead, survivorship, and selection/multiple-testing corrections.
Deflated metrics that account for the number of trials.

An idea is killed in simulation before it can be funded live.

The current state

The surviving strategy — a single breakout-only setup — is in paper trading and currently net negative: two closed positions, both stop-outs, −1,932 GBP realised.

I report that without spin, because the alternative is the exact self-deception this article is about. A walk-forward pass certifies honesty, not profitability. It tells you what would actually have happened.

Conclusions

Win rate is a weak objective. Expectancy (mean R per trade) and robustness matter more; a lower win rate with an asymmetric payoff can dominate.
Out-of-sample is the only estimate that matters. Everything else is in-sample storytelling.
Most candidate strategies fail validation. That's the filter working, not the process failing.

The 70% target is still open. I'll only believe it when the walk-forward gate and the live paper trades both agree.