Research & signal quality
Backtesting without look-ahead bias or overfitting
Build a reproducible trading strategy test with point-in-time data, realistic fills, held-out periods, and explicit failure criteria.
By atradeaday · Published
A backtest answers what a specified rule would have done under specified data and execution assumptions. It does not prove that the rule will work in the future. The most useful test is one whose assumptions and failure cases can be inspected.
Freeze the rule and the data contract
Record the instrument universe, source, timezone, session boundaries, indicator warm-up, and missing-data policy. Specify the entry, exit, cancellation, and sizing rules. If discretionary choices remain, describe them rather than silently choosing the most favorable interpretation on each chart.
Keep the raw input and strategy version so another run can reproduce the result. Survivorship matters: a universe made only of assets still trading today can omit historical failures and delistings.
Respect information availability
A strategy acting on a completed candle cannot execute at an earlier intrabar price. A swing confirmed by later candles becomes available only after those candles arrive. A daily value joined onto an hourly series must not use the day's final result before the daily close.
A useful debugging exercise is to truncate the input at each decision time and recalculate the signal. If the historical signal changes when later observations are removed, investigate whether the rule uses future information or a repainting feature.
Model execution conservatively
Include spread, commissions, slippage, funding where applicable, and non-fills. When both stop and target occur within one historical candle, the candle alone does not reveal which occurred first. Use more detailed data or an explicitly conservative ordering assumption.
Report sensitivity: if a small cost increase removes the apparent edge, the strategy may rely on unrealistic execution. The SEC's fee bulletin explains why costs matter to returns; the actual cost model must match the instrument and venue.
Separate selection from evaluation
Use one period to develop rules and a later untouched period to evaluate them. Walk-forward evaluation repeats this process through time. Once a held-out period is used to pick parameters, it is no longer untouched evidence.
Keep a record of all tested variants. Trying hundreds of combinations and publishing only the winner hides selection risk. Prefer settings that behave reasonably across nearby values over an isolated historical optimum.
Report the whole distribution
Include sample size, date range, exposure, turnover, average win and loss, net expectancy, drawdown, and worst periods. Split results by instrument and market condition. Paper trading and subsequent live observation can reveal operational failures that a historical simulation cannot model.
Continue with evaluating trading signals for a compact scorecard that goes beyond win rate.