Home / Biases Checklist

Backtesting Biases Checklist

A reference table of the biases that inflate backtest performance, showing how each one quietly enters a study and the specific practice that removes it.

Biases Checklist: Backtesting biases are systematic errors that make a strategy look better on history than it can be live. The main ones are look-ahead bias (using information not yet available), survivorship bias (testing only instruments that still exist), selection and sample bias (an unrepresentative universe or period), data-snooping and its statistical face p-hacking (trying many variants until one looks significant), overfitting and curve-fitting (fitting noise with too many parameters), and confirmation bias (accepting good results uncritically). Each is fixed by a specific discipline, not by good intentions: point-in-time data, strict as-of timestamps, reserved out-of-sample data, limits on the number of trials and few parameters.

Treat each bias below as a defect to be actively disproven, not a risk to be vaguely acknowledged. They share a common signature: every one makes the backtest look better than the future can be, and every one is invisible unless you specifically test for it. For depth on any row, follow the links into the biases pillar.

The biases and their fixes

BiasHow it sneaks inThe fix
Look-ahead biasActing on data not knowable at the decision moment: filling at the signal bar's close, using restated fundamentals, or repainting indicators.Enforce strict as-of timestamps; fill on the next bar; only use data stamped before the bar you trade. See look-ahead bias.
Survivorship biasTesting only on instruments that still exist today, dropping the delisted, merged and bankrupt names.Use a point-in-time, survivorship-free universe that includes dead symbols. See survivorship bias.
Selection biasCherry-picking a favourable index constituent, a calm period, or the instruments that happened to work.Fix the universe and period by rule before testing; test across many instruments and regimes. See selection bias.
Sample biasDrawing conclusions from a period that is not representative, such as only a bull market.Test across multiple market regimes and long enough history to include stress. See sample bias.
Data-snoopingReusing the same data to try many hypotheses until one appears significant by chance.Limit the number of trials, adjust for multiple testing, and keep a genuinely untouched hold-out. See data-snooping.
p-hackingTweaking filters or parameters until a result crosses a significance threshold, hiding the failed attempts.Fix the analysis before looking; report the number of variations tried; prefer a deflated Sharpe. See data-snooping.
OverfittingTuning so closely to history that the model captures noise, producing a great backtest that fails live.Use few parameters, seek broad plateaus, and confirm out-of-sample and walk-forward. See overfitting.
Curve-fittingAdding rules or adjusting thresholds until the equity curve looks good on the test data.Prefer simple logic with an economic rationale; test parameter sensitivity for a plateau, not a spike. See curve-fitting.
UnderfittingA model too simple or constrained to capture the real structure, performing poorly in and out of sample.Add justified structure only where it improves out-of-sample results, not in-sample fit. See underfitting.
Confirmation biasAccepting a good backtest uncritically while re-examining a bad one until it improves.Pre-register the test and its pass criteria; apply the same scrutiny to good and bad results. See confirmation bias.

The unrealistic-cost trap

Separate from the statistical biases, the most common way a backtest flatters itself is by assuming perfect fills at the signal price with zero cost. On Indian equities and F&O, STT, exchange transaction charges, GST on brokerage, SEBI turnover fees, stamp duty, the bid-ask spread and slippage all apply. An intraday options strategy that ignores STT on the sell side and realistic spreads on illiquid strikes can look profitable and lose money live. Model costs per leg, per trade, and re-run with costs doubled to see how fragile the edge is.

Why biases compound

These errors are not independent. A researcher who selects a favourable universe (selection bias), searches many variants on it (data-snooping), keeps only the good result (confirmation bias) and reports it with optimistic fills (no cost model) can turn pure noise into a compelling equity curve. Because each bias individually is easy to rationalise, the defence is procedural: reserve out-of-sample data first, fix the universe and analysis before looking, count and disclose every trial, and model costs honestly.

Frequently asked questions

What is the difference between overfitting and curve-fitting?
They describe the same failure from two angles. Curve-fitting is the act of adjusting parameters or adding rules until the historical equity curve looks good; overfitting is the resulting state, where the model has captured noise specific to that data rather than a repeatable pattern. Curve-fitting is the mechanism, overfitting is the outcome.
How does look-ahead bias typically enter a backtest?
Usually through timing. Common routes are filling an order at the close of the same bar the signal was computed from, using fundamentals as later restated rather than as first reported, applying corporate-action adjustments across the whole series, and normalising features with full-sample statistics. Each lets the strategy use information it could not have had live.
Why is survivorship bias so easy to miss?
Because the missing data is invisible by construction. If your dataset only contains instruments that still trade today, there is nothing on screen to remind you that the delisted and bankrupt names were dropped. The backtest runs cleanly and looks reasonable; it is simply built on the winners only.
What is data-snooping and how is it controlled?
Data-snooping is running so many strategy variations on the same data that one succeeds by chance. Control it by limiting the number of trials, keeping a genuinely untouched hold-out, adjusting significance for the number of tests, and preferring an idea with an economic rationale over one found purely by search.
Can I remove all bias from a backtest?
No. You can eliminate the gross ones such as look-ahead and survivorship with the right data and discipline, and you can limit overfitting and snooping with restraint and out-of-sample testing. But some residual bias always remains, which is why forward testing on live data and honest costs are the final checks before risking capital.
Is confirmation bias really a backtesting problem?
Yes, and it is underrated. It shapes which results you trust: a profitable backtest gets accepted quickly while a losing one is re-run and tweaked until it improves. That asymmetric scrutiny quietly selects for luck. The fix is to decide the test and its pass criteria in advance and apply them equally to good and bad outcomes.

Last reviewed 11 July 2026. Educational content only — not investment advice.

Educational content only — not investment advice. See our Risk Disclosure and Methodology.