Home / Backtesting Fundamentals / Historical Data
DataBeginner

Historical Data

Historical data is the recorded market information (prices, volumes, corporate actions and reference data) that a backtest replays, and its accuracy, resolution and point-in-time correctness set the ceiling on how trustworthy any backtest can be.

Quick answer: Historical data is the recorded market information (prices, volumes, corporate actions and reference data) that a backtest replays, and its accuracy, resolution and point-in-time correctness set the ceiling on how trustworthy any backtest can be.

In simple words

Historical data is the past that your backtest replays. It comes in different grains, from daily closes to every tick, and it must reflect what was actually knowable at each moment, including old splits and dividends handled correctly. If the data is wrong or secretly uses information from the future, every result built on it is wrong too, no matter how clever the strategy.

Purpose

This page explains what market data a backtest consumes, how it must be adjusted and structured, and why data integrity is the foundation on which every downstream metric depends.

Visual explanation

Historical Data

Raw feeds flow through cleaning, corporate-action adjustment and point-in-time alignment before becoming backtest-ready data.

Data PipelineSource / FeedIngestNormaliseCorp-actionAdjustValidateStoreServe

Professional explanation

Types and resolution of market data

Historical data ranges from low to high resolution: end-of-day bars, intraday bars at one minute or five minutes, and tick or order-book data recording every quote and trade. Each carries a distinct trade-off. Daily data is compact and adequate for positional and swing strategies; intraday data is heavier and needed for shorter horizons; tick data is enormous and only justified for microstructure or execution research. Choosing a resolution finer than the strategy needs multiplies storage and processing cost without improving the conclusion.

OHLCV and its hidden assumptions

The common bar format records open, high, low, close and volume for each period. It is compact but discards the path within the bar: you know the high and low occurred, but not in what order or at what time. This matters because a backtest that assumes it could transact at the bar's low or high is often assuming a fill it could never have achieved. Intrabar path ambiguity is a frequent, quiet source of look-ahead bias, and conservative logic should assume the worst plausible sequence within a bar.

Corporate-action adjustment

Raw prices are distorted by splits, bonuses, dividends and rights issues. A 5-for-1 split shows a raw price collapse of 80 percent that is not a loss at all, and an unadjusted series would trigger spurious signals. Back-adjusted series correct historical prices so returns are continuous across these events, but adjustment introduces its own subtleties: dividend-adjusted total-return series differ from price-return series, and rolling futures contracts requires a stitching convention. You must know exactly which adjustment your data uses, because it changes every computed return.

Point-in-time correctness

The most important and most neglected property is point-in-time correctness: each historical record must reflect only information available at that moment. Index constituents change, so a backtest must use the members as they stood on each date, not today's list. Fundamentals are restated, so as-reported figures, not later revisions, are what a real trader would have seen. Reference data such as sector classification or free-float also evolves. A dataset that silently applies today's knowledge to the past is a structural source of look-ahead and survivorship bias.

Gaps, errors and outliers

Real data is dirty. Feeds contain missing bars, zero or negative prices, bad ticks, duplicated rows, timezone misalignments and mislabelled holidays. A single erroneous print, such as a fat-finger tick at half the true price, can fabricate a trade that never could have happened and distort the whole equity curve. Cleaning is not optional housekeeping; it is a substantive modelling decision, because how you treat a gap, a suspension or an outlier directly changes which trades the backtest generates.

Sources and their trade-offs in India

Indian historical data is available from exchange archives, licensed vendors and broker APIs, each with different coverage, adjustment conventions and reliability. Free sources are convenient but often lack point-in-time constituent history, carry adjustment errors, or silently exclude delisted names. Reliable F&O backtesting also requires a coherent contract history with correct expiry and roll handling. The cost of good data is real, but it is small compared to the cost of trading a strategy validated on flawed data.

Practical example

Illustrative example (Indian market)

You backtest a Nifty 50 mean-reversion strategy on Rs 5,00,000 using daily data. Two data issues silently inflate the result. First, the universe is today's Nifty 50, so a name added recently after a strong run is credited with gains it earned before joining, a point-in-time error. Second, one stock's 1:1 bonus in the sample is unadjusted, showing a 50 percent overnight fall that your mean-reversion rule reads as an extreme buy signal, generating a large fictitious winning trade. After rebuilding with point-in-time constituents and properly bonus-adjusted prices, several of the best trades disappear and the edge shrinks to something far more modest.

On NSE, F&O contracts expire monthly, so a continuous backtest must roll from the expiring contract to the next and decide how to handle the price gap at roll. A naive concatenation that ignores the roll spread injects artificial jumps that a trend strategy can mistake for signals.

Advantages

  • Correct data is the foundation every reliable metric depends on
  • Point-in-time datasets structurally remove survivorship and look-ahead at the source
  • Choosing the right resolution keeps research fast and affordable

Limitations

  • High-quality point-in-time data is expensive and often licensed
  • OHLCV bars hide the intrabar path, creating fill ambiguity
  • Adjustment conventions differ between vendors, so returns are not directly comparable
  • Deep history may reflect a market structure that no longer exists

Why it matters in practice

  • Data quality sets the hard ceiling on backtest trustworthiness
  • A brilliant strategy on flawed data is a flawed result, not a brilliant one

Common mistakes

  • Using today's index constituents to backtest the past, importing survivorship bias
  • Backtesting on unadjusted prices so splits and bonuses fire false signals
  • Assuming fills at the bar's high or low, which the intrabar path may never have allowed
  • Mixing price-return and total-return series without realising dividends change the result
  • Concatenating futures contracts without handling the expiry roll
  • Trusting a free feed without checking for missing bars, bad ticks and delisted names

Professional usage

Serious quant teams treat data as a first-class asset. They ingest from reputable sources, reconcile across vendors, store point-in-time snapshots so any historical date can be reconstructed as it truly was, and version the dataset alongside the strategy code. They document the adjustment convention explicitly and audit for gaps and outliers before a single backtest is run, on the principle that data errors are indistinguishable from alpha until you look.

Key takeaways

  • Historical data is the raw material of every backtest and sets its ceiling
  • Data must be corporate-action adjusted and point-in-time correct
  • OHLCV bars hide the intrabar path, a quiet source of fill ambiguity
  • Bad data is indistinguishable from a real edge until you audit for it

Frequently asked questions

What historical data do I need to backtest?
You need price and volume data at a resolution matching your strategy's horizon, plus corporate-action records for adjustment and, ideally, point-in-time reference data such as historical index constituents. For F&O you also need a coherent contract history with correct expiries and roll handling.
What is point-in-time data?
Point-in-time data reflects only what was actually known on each historical date: the index members as they stood then, fundamentals as first reported rather than later restated, and reference data as it existed at the time. It is essential for preventing look-ahead and survivorship bias at the source.
Why must prices be adjusted for corporate actions?
Because splits, bonuses, dividends and rights issues cause raw price jumps that are not real gains or losses. An unadjusted series would trigger spurious signals, so back-adjustment corrects historical prices to make returns continuous across these events, though the exact convention used changes every computed return.
What resolution of data should I use?
Match the resolution to the strategy. Daily bars suit positional and swing systems, intraday bars suit shorter horizons, and tick or order-book data is only justified for microstructure or execution research. Using data finer than needed multiplies cost without improving the conclusion.
What is the problem with OHLCV bars?
An open-high-low-close-volume bar records that a high and low occurred but not the order or timing within the bar. A backtest that assumes it transacted at the bar's low or high may be assuming a fill that the intrabar path never allowed, which is a common quiet source of look-ahead bias.
Can I use free historical data?
You can for learning, but free feeds often lack point-in-time constituent history, contain adjustment errors, and silently exclude delisted names, all of which bias results. For decisions involving real capital, reconciled or licensed data with documented adjustment conventions is far safer.
How does bad data create fake profits?
A single erroneous print, such as a fat-finger tick at half the true price, can fabricate an entry and exit that could never have happened, injecting a fictitious winning trade. Because such errors are indistinguishable from real signals, uncleaned data can manufacture an edge that does not exist.
What is total-return versus price-return data?
Price-return data tracks only price changes, whereas total-return data also reinvests dividends, so the two diverge over time. Using one when you meant the other misstates returns, especially for high-dividend stocks or long horizons, so you must know which series your data represents.
How is futures data handled across expiries?
Because each futures contract expires, a continuous series must roll from the expiring contract to the next, and the roll introduces a price gap from the spread between contracts. The stitching convention, such as back-adjusting for the roll, must be chosen deliberately, since a naive concatenation injects artificial jumps.
How much history is enough?
Enough to include several market regimes and at least one stress event, and enough trades that the result is not driven by a few lucky ones. Very old data can help by adding regimes but may reflect obsolete market structure, so relevance matters alongside length.
Does data quality really outrank strategy design?
In the sense that it caps everything, yes. No modelling can recover from corrupted or biased data, so a mediocre strategy on clean point-in-time data is more trustworthy than a brilliant one on flawed data. Data integrity is the foundation the rest is built on.
What corporate actions matter most on NSE?
Splits, bonuses, dividends, rights issues and mergers all distort raw prices. Bonuses and splits cause large mechanical price drops that unadjusted backtests misread as crashes, while dividends matter for total-return accuracy, so each must be handled by the adjustment convention.
How do I detect data errors before backtesting?
Scan for missing bars, zero or negative prices, extreme single-bar returns, duplicated timestamps and volume anomalies, and reconcile against a second source where possible. Flagging outliers and confirming holidays and timezones catches most errors that would otherwise fabricate trades.

Voice search & related questions

Natural-language questions people ask about Historical Data.

What is historical data in backtesting?
It is the recorded past of the market, prices and volumes and corporate actions, that your backtest replays so your rules can trade through it.
Why do I need to adjust old prices?
Because splits, bonuses and dividends make raw prices jump for reasons that are not real gains or losses. Adjusting them stops your strategy reacting to fake moves.
What does point-in-time data mean?
It means the data shows only what was actually known on each date, like the index members back then, not today's list, so you do not cheat with hindsight.
Can I just use free market data?
For learning, yes, but free data often misses delisted stocks and has adjustment errors, which quietly biases results. For real money, use reconciled or licensed data.
Why does data quality matter so much?
Because everything is built on it. If the data is wrong or biased, every metric you compute is wrong too, no matter how good the strategy looks.
What is wrong with daily high-low bars?
They tell you the high and low happened but not when, so assuming you bought at the low can be a fill you could never actually have got.

Sources & references

    Last reviewed 11 July 2026. Educational content only — not investment advice. Markets and rules change; verify current conventions with SEBI, NSE/BSE and your broker.

    ← Previous
    Why Backtesting Fails
    Educational content only — not investment advice. Examples use illustrative numbers and simplified models. Backtested results are hypothetical and trading derivatives involves substantial risk. See our Risk Disclosure and SEBI Disclaimer.