Home / Backtesting Biases / Confirmation Bias
BiasBeginner

Confirmation Bias

Confirmation bias is the human tendency to seek, favour and remember evidence that supports a belief you already hold, which in backtesting leads researchers to design, interpret and report tests in ways that confirm a strategy they are already attached to.

Quick answer: Confirmation bias is the human tendency to seek, favour and remember evidence that supports a belief you already hold, which in backtesting leads researchers to design, interpret and report tests in ways that confirm a strategy they are already attached to.

In simple words

Confirmation bias is looking for reasons you are right and skimming past reasons you are wrong. In backtesting it makes you run the test that flatters your idea, notice the good results, and explain away the bad ones. It is not a data problem but a human one, and it quietly steers every other bias in a favourable direction.

Purpose

This concept exists because backtesting is done by humans who become attached to their ideas, and that attachment shapes which tests are run and how results are read, making confirmation bias the psychological engine behind many of the technical biases.

Professional explanation

What confirmation bias is

Confirmation bias is a well-documented cognitive tendency to favour information that supports what we already believe and to discount information that contradicts it. In research it manifests as asking questions likely to yield a supportive answer, weighting confirming evidence more heavily than disconfirming evidence, and remembering hits while forgetting misses. It is not deliberate dishonesty; it operates below awareness, which is exactly what makes it dangerous in a discipline that depends on ruthless self-scrutiny.

How it corrupts a backtest

A researcher attached to an idea will, often unconsciously, choose the instrument, period and parameters that make it look good, stop searching once a favourable result appears, and interpret ambiguous outcomes charitably. Losing trades get explained away as anomalies while winning trades are taken as proof. The test is nominally objective, but the choices surrounding it are steered toward the desired conclusion, so the backtest measures the researcher's hopes as much as the strategy's merit.

Why it is the engine behind other biases

Confirmation bias supplies the motive that turns technical pitfalls into actual errors. It is why you keep tweaking until the curve looks good, which is curve fitting; why you test only the market where the idea worked, which is selection bias; why you stop the parameter search at the flattering peak, which is data snooping. Each of those biases requires a human decision to stop looking or to look in a particular place, and confirmation bias is the tendency that makes those decisions favour the prior belief.

The asymmetry of scrutiny

A tell-tale sign is asymmetric scrutiny: results that support the idea are accepted quickly, while results that contradict it are examined for bugs until they go away. This ratchet systematically pushes conclusions in the favourable direction, because errors that help are kept and errors that hurt are hunted down. Genuine science demands the opposite discipline, scrutinising favourable results at least as hard as unfavourable ones, which is psychologically uncomfortable and therefore rare without deliberate structure.

Guarding against it with process

Because willpower alone does not defeat a subconscious bias, the defences are structural. State the hypothesis and the success criteria in advance, so you cannot move the goalposts after seeing results. Pre-register the test design and the parameter ranges. Reserve out-of-sample data and commit to using it once, whatever it shows. Actively try to falsify the idea rather than confirm it, seeking the conditions under which it should fail. Where possible, have a colleague review the design blind to your hopes, and keep an honest log of every variant so you cannot quietly forget the failures.

The role of adversarial thinking

The most effective antidote is to adopt the stance of a sceptic trying to disprove your own strategy. Ask what would have to be true for the edge to be an illusion, then test for exactly that: check the look-ahead timing, widen the sample, degrade the parameters, add realistic costs. A strategy that survives a sincere attempt to kill it is far more trustworthy than one that merely survived a search for reasons to believe it. Confirmation bias is beaten not by trusting yourself less in the abstract but by building tests designed to prove yourself wrong.

Practical example

Illustrative example (Indian market)

You are convinced a particular Nifty breakout pattern works, so you test it first on the two years you remember it working well, on capital of Rs 5,00,000, and it looks strong. A weaker result on other years you attribute to unusual conditions, and a losing stretch you dismiss as a data glitch you never quite investigate. Because you stopped looking once the idea was confirmed and scrutinised only the disappointing results, your conclusion reflects your prior belief. A confirmation-resistant process would have fixed the test period and success criteria in advance, examined the good years as sceptically as the bad, and committed to a single out-of-sample verdict, which typically deflates the original enthusiasm.

Retail trading communities often amplify confirmation bias: a popular Bank Nifty setup is shared with screenshots of its winning trades, and members test it expecting success, notice the wins and overlook the losses. The collective belief that it works makes each individual more likely to design a backtest that confirms it and to quietly drop the runs that did not.

Limitations

  • It is subconscious, so awareness alone does not remove it and self-report cannot detect it
  • Structural defences like pre-registration are harder to enforce in informal retail research
  • An honest process can still be undermined if success criteria were vague to begin with
  • It interacts with every technical bias, so it cannot be isolated and fixed on its own
  • Even adversarial testing can be gamed if the researcher secretly wants the idea to survive

Why it matters in practice

  • It is the psychological driver that turns technical pitfalls into real backtest errors
  • It biases which tests are run and how results are read, upstream of any calculation

Common mistakes

  • Running first the test most likely to confirm the idea, then stopping once it does
  • Scrutinising losing results for bugs while accepting winning results uncritically
  • Explaining away contradictory periods as anomalies without investigating them
  • Moving the success criteria after seeing the results to keep the conclusion
  • Remembering the winning trades and forgetting the losing ones when judging a setup
  • Adopting a community-favoured setup and testing it expecting, and finding, success

Professional usage

Rigorous researchers assume they are biased toward their own ideas and build process to counteract it. They pre-specify the hypothesis, success criteria and parameter ranges, reserve out-of-sample data for a single honest verdict, and deliberately try to falsify the strategy rather than confirm it. Many invite blind review from a colleague and log every variant so failures cannot be forgotten. The governing principle is that a strategy is trustworthy only after a sincere attempt to disprove it has failed, not after a search for supporting evidence has succeeded.

Key takeaways

  • Confirmation bias is favouring evidence that supports a strategy you already believe in
  • In backtesting it steers which tests you run and how you read the results
  • It is the psychological engine behind curve fitting, selection bias and data snooping
  • Its signature is asymmetric scrutiny of favourable versus unfavourable results
  • Beat it with pre-specified criteria, out-of-sample discipline and adversarial testing

Frequently asked questions

What is confirmation bias in backtesting?
Confirmation bias is the tendency to seek and favour evidence supporting a strategy you already believe in, and to discount evidence against it. In backtesting it leads a researcher to design tests that flatter the idea, notice the good results, and explain away the bad ones, so the backtest reflects prior belief as much as the strategy's merit.
How is confirmation bias different from the technical biases?
The technical biases, like look-ahead or survivorship, are properties of data and method. Confirmation bias is psychological: it is the human tendency that motivates the choices producing those technical errors. It sits upstream, steering which tests are run and how results are interpreted.
Why is confirmation bias so dangerous in research?
Because it operates below awareness, so it feels like ordinary judgement rather than error, and because backtesting depends on ruthless self-scrutiny that the bias quietly undermines. It systematically pushes conclusions toward what the researcher already hoped, without any deliberate dishonesty.
How does confirmation bias drive other biases?
It supplies the motive: it is why you keep tweaking until the curve looks good, which is curve fitting; why you test only the market where the idea worked, which is selection bias; and why you stop the parameter search at the flattering peak, which is data snooping. Each needs a human choice that the bias tilts favourably.
What is asymmetric scrutiny?
Asymmetric scrutiny is examining unfavourable results for errors until they disappear while accepting favourable results without question. This ratchet keeps helpful errors and removes harmful ones, so it pushes conclusions in the favourable direction. Countering it means scrutinising good results at least as hard as bad ones.
How do I guard against confirmation bias?
Use structure, not willpower: state the hypothesis and success criteria in advance, pre-register the test design and parameter ranges, reserve out-of-sample data for a single honest verdict, and actively try to falsify the idea. Blind review by a colleague and an honest log of every variant also help.
What is adversarial testing?
Adversarial testing means adopting the stance of a sceptic trying to disprove your own strategy: ask what would make the edge an illusion, then test for exactly that, checking timing, widening the sample, degrading parameters and adding costs. A strategy that survives a sincere attempt to kill it is far more trustworthy.
Can I remove confirmation bias by being aware of it?
Awareness helps but is not enough, because the bias is subconscious and self-report cannot reliably detect it. The reliable defences are structural, such as pre-specified criteria and committed out-of-sample use, which constrain your choices so the bias has fewer opportunities to act.
How does confirmation bias affect how I read results?
It makes you interpret ambiguous outcomes charitably, accept confirming evidence quickly, and remember winning trades while forgetting losing ones. The same backtest can look convincing or unconvincing depending on the prior belief you bring to it, which is why the interpretation must be disciplined.
Does trading in a community increase confirmation bias?
It can. When a setup is popular and shared with winning examples, members test it expecting success, notice the wins and overlook the losses, and the collective belief reinforces each individual's tendency to confirm. Independent, pre-specified testing is the antidote to this social amplification.
How does pre-registration help?
Pre-registration fixes the hypothesis, success criteria and parameter ranges before you see results, so you cannot move the goalposts afterward to keep a favourable conclusion. It converts a flexible search that the bias can steer into a committed test whose verdict you must accept.
Is confirmation bias the same as overfitting?
No. Overfitting is a technical property of a model that has learned noise. Confirmation bias is a psychological tendency. They are linked because confirmation bias motivates the over-tuning that produces overfitting, but one is about the model and the other about the researcher.
Why should I try to disprove my own strategy?
Because a strategy that survives a genuine attempt to falsify it is far more trustworthy than one that merely survived a search for supporting evidence. Trying to disprove it forces you to test the conditions under which it should fail, which is exactly where confirmation bias would otherwise stop you looking.
How does confirmation bias relate to why backtests fail?
It is the human root cause: it biases which tests are run and how they are read, so it turns avoidable technical pitfalls into real errors and helps explain why strategies that looked convincing in research fail live. Controlling it is as important as any statistical correction.

Voice search & related questions

Natural-language questions people ask about Confirmation Bias.

What is confirmation bias in simple terms?
It is looking for reasons you are right and glossing over reasons you are wrong. In backtesting it makes you run the test that flatters your idea and explain away the results that do not.
Why does confirmation bias hurt my backtesting?
Because it quietly steers your choices toward the answer you already want. You stop searching once the idea looks good and scrutinise only the disappointing results, so your conclusion reflects your hopes.
How is confirmation bias linked to overfitting?
Confirmation bias is why you keep tweaking until the backtest looks perfect. That over-tweaking is curve fitting or overfitting, so the human bias drives the technical one.
How do I fight confirmation bias?
Decide your rules and success criteria before you look, keep some data untouched for one honest test, and actively try to prove your own idea wrong instead of right.
Can I just be aware of confirmation bias to avoid it?
Awareness alone is not enough because it is subconscious. You need structure, like pre-set criteria and a commitment to accept the out-of-sample result whatever it shows.
Does following trading groups make confirmation bias worse?
Often yes. When a setup is popular and shared with winning screenshots, you test it expecting success and overlook the losses, so the crowd reinforces your bias toward believing it.

Sources & references

    Last reviewed 11 July 2026. Educational content only — not investment advice. Markets and rules change; verify current conventions with SEBI, NSE/BSE and your broker.

    ← Previous
    Sample Bias
    Educational content only — not investment advice. Examples use illustrative numbers and simplified models. Backtested results are hypothetical and trading derivatives involves substantial risk. See our Risk Disclosure and SEBI Disclaimer.