Home / Data Quality / Missing Data
DataBeginner

Missing Data

Missing data is any absent value in a series — from trading halts, holidays, illiquidity, feed outages or corrupt records — and the danger is not the gap itself but the fill: replacing a missing value carelessly, especially by carrying a future value backwards, injects information a real trader never had and biases the backtest.

Quick answer: Missing data is any absent value in a series — from trading halts, holidays, illiquidity, feed outages or corrupt records — and the danger is not the gap itself but the fill: replacing a missing value carelessly, especially by carrying a future value backwards, injects information a real trader never had and biases the backtest.

In simple words

Missing data is the holes in a price history: days the market was shut, a stock that did not trade, a feed that dropped out, or a cell that reads NaN. Every backtest has to decide what to do with these holes, and that decision matters more than it looks. Fill them the wrong way and you either invent prices that never existed or, worse, quietly leak future information into the past.

Purpose

Handling missing data exists because no real dataset is complete, and the method you use to patch the holes is itself a modelling assumption that can add or remove bias without you noticing.

Professional explanation

Why data goes missing

Gaps arise for very different reasons, and the reason dictates the correct treatment. Exchange holidays and weekends are expected absences that a calendar should handle, not a fill. A trading halt or a lower-circuit day means the instrument genuinely could not be traded, so a filled price would misrepresent a period when no exit was possible. Illiquid instruments simply do not print on some days. And feed outages or corrupt records are true data faults where the value existed but was lost. Conflating these — treating a halt like a holiday, or an outage like an illiquid day — produces subtly wrong backtests.

The forward-fill versus back-fill distinction

The single most important rule is directional. Forward-filling — carrying the last known value forward until a new one arrives — is generally safe because it only ever uses information already available at that time. Back-filling — carrying a future value backwards to patch an earlier gap — is dangerous because it inserts a price that was not yet known, a direct form of look-ahead. Interpolating between a past and a future point is a subtler version of the same sin, because the interpolated value depends on data from after the gap. As a default, a backtest should forward-fill or leave gaps explicit, never back-fill prices used for decisions.

Halts and untradeable periods

When an instrument is halted or locked at a circuit limit, the honest model is not to fill a plausible price but to record that no trade was possible. A backtest that fills a halted day with an interpolated price may show an exit or entry that could never have executed, understating the risk of being trapped in a position. This is especially relevant for Indian small and mid caps that hit upper or lower circuits, where a strategy might in reality be unable to sell for several days while the backtest happily books an exit at a made-up price.

NaNs and how they propagate

Explicit missing markers such as NaN are actually safer than silent gaps because they are visible, but they must be handled deliberately before indicators are computed. A single NaN feeding into a moving average or a standard-deviation window can poison the whole window, producing a NaN signal or, if the library silently drops it, a statistic computed over the wrong sample. Worse is a NaN that gets coerced to zero, which a price series reads as a 100 percent crash and a return series reads as a total loss. Every pipeline needs an explicit, logged policy for NaNs rather than relying on a library default.

How the fill choice biases results

Each fill method leaves a fingerprint. Forward-filling a stale price flattens volatility and can suppress or delay signals, making a mean-reversion system look calmer than reality. Interpolation smooths the series and understates true gaps and drawdowns. Filling untradeable halts creates fills that inflate performance by pretending liquidity existed. Even dropping missing rows biases results if the missingness is not random — illiquid or stressed names go missing precisely when they matter most, so deleting them is a quiet cousin of survivorship bias. The correct method is the one whose assumption matches why the data is missing, and it should be stated, not defaulted.

Forward-fill vs Back-fill

AspectForward-fillBack-fill
DirectionCarries last known value forwardCarries a future value backward
Look-ahead riskNone — uses only past dataHigh — inserts future information
Effect on seriesMay flatten volatility if staleCan invent knowledge of the future
Safe for decisions?Generally yesNo
Typical useDefault gap handlingAvoid for anything traded on

Practical example

Illustrative example (Indian market)

You backtest a portfolio that includes an NSE mid-cap stock which hits its lower circuit for three straight days, printing no tradeable volume, on capital of Rs 5,00,000. Your data has gaps on those three days. If you interpolate a smooth path between the pre-halt price of Rs 480 and the post-halt price of Rs 410, your backtest shows an orderly decline and books an exit at, say, Rs 445 on day two. In reality the stock was locked and you could not have sold at any price until it reopened at Rs 410. The interpolated fill understated your loss by Rs 35 a share and invented liquidity that did not exist. Recording the halt as untradeable, and marking the exit at the real reopening price of Rs 410, produces the honest and far worse outcome.

Indian small and mid caps regularly hit upper or lower circuits, and index constituents observe NSE trading holidays that differ from global calendars. A backtest that treats a circuit-locked day as a normal filled bar, or that misaligns Indian holidays with foreign data in a multi-market test, will book fills that were impossible and generate signals on days the market was shut.

Limitations

  • The correct fill depends on why data is missing, and that reason is often not recorded in the feed
  • Forward-filling stale prices flattens volatility and can hide real risk
  • Any back-fill or interpolation across a gap introduces look-ahead by using post-gap information
  • Dropping missing rows biases results when missingness is non-random, as with stressed or illiquid names
  • Circuit halts mean no trade was possible, which most fill methods silently ignore

Common mistakes

  • Back-filling or interpolating prices that are then used to make trading decisions
  • Filling a halted or circuit-locked day with a plausible price and booking a fill that could not have happened
  • Letting a NaN be coerced to zero, which reads as a total crash in a price or return series
  • Deleting rows with missing values when the missingness is correlated with stress or illiquidity
  • Using a library default fill without knowing or logging which method it applied
  • Aligning multiple markets without reconciling different holiday calendars, creating false gaps or signals

Professional usage

Serious researchers first classify why a value is missing — holiday, halt, illiquidity or fault — and treat each differently, using a trading calendar for expected absences and marking halts as untradeable rather than filling them. For decision data they forward-fill or leave gaps explicit and never back-fill, and they log every fill so the pipeline is reproducible. They also test sensitivity to the fill choice, because a strategy whose result changes materially with the missing-data policy is resting on an artefact rather than an edge.

Key takeaways

  • Missing data comes from holidays, halts, illiquidity and faults, and each needs different handling
  • Forward-filling is generally safe; back-filling and interpolation leak future information
  • Halted or circuit-locked days should be marked untradeable, not filled with invented prices
  • A NaN coerced to zero is read as a total crash, so NaN policy must be explicit
  • Dropping non-random missing rows is a quiet form of survivorship bias

Frequently asked questions

What is missing data in a price series?
Missing data is any absent value — from exchange holidays, trading halts, illiquidity, feed outages or corrupt records. It shows up as gaps in the series or as explicit markers like NaN, and every backtest must decide how to handle it.
Why is missing data a problem for backtesting?
Because the way you patch the gaps is itself an assumption. Fill them carelessly and you either invent prices that never traded or leak future information into the past, both of which bias the backtest without any obvious warning.
What is the difference between forward-fill and back-fill?
Forward-fill carries the last known value forward and uses only information available at the time, so it is generally safe. Back-fill carries a future value backwards to patch an earlier gap, which inserts information the trader did not yet have and causes look-ahead.
Why is back-filling dangerous?
Because it fills an earlier gap with a value from later, so a decision made on that bar is using data from the future. This is a direct form of look-ahead bias and it inflates backtest performance.
How should I handle a trading halt in a backtest?
Mark the halted period as untradeable rather than filling it with a plausible price. During a halt or a circuit lock no trade was possible, so any filled exit or entry is fictional and understates the risk of being trapped in the position.
What happens if a NaN becomes zero in price data?
A price of zero reads as a total collapse, and a return computed from it reads as a 100 percent loss, so a single coerced NaN can create a catastrophic phantom move. NaN handling must be explicit and logged, never left to a silent default.
Is it okay to delete rows with missing data?
Only if the missingness is random. Illiquid or stressed instruments tend to go missing exactly when they matter most, so deleting those rows quietly removes the worst outcomes, which is a cousin of survivorship bias.
How does forward-filling affect volatility?
Carrying a stale price forward flattens the series, understating volatility and true gaps. This can make a mean-reversion strategy look calmer and less risky than reality, so forward-fill is safe for look-ahead but not free of distortion.
How should exchange holidays be handled?
With a trading calendar, not a fill. Holidays and weekends are expected absences, so the backtest should simply not expect data on those days rather than interpolating prices for a market that was closed.
Can interpolation cause look-ahead bias?
Yes. Interpolating between a point before the gap and a point after it makes the filled value depend on data from after the gap, so any decision using it leaks future information, just like back-filling.
Why do different markets create false gaps?
Because their holiday calendars differ. Aligning an Indian series with a foreign one without reconciling holidays produces days where one market has data and the other does not, creating false gaps or signals on days a market was shut.
How do I know if my fill method is biasing results?
Test sensitivity by re-running the backtest under different fill policies. If the result changes materially between forward-fill, interpolation and dropping rows, the strategy is resting on the fill assumption rather than a genuine edge.
What is the safest default for missing decision data?
Forward-fill or leave the gap explicit and skip the decision. Both use only information available at the time, whereas any method that reaches across the gap to a future value risks look-ahead.
Do NaNs help or hurt data quality?
Explicit NaNs are actually helpful because they make missingness visible and force a deliberate policy. The danger is a silent gap or a NaN that a library quietly drops or coerces, because those hide the problem instead of surfacing it.

Voice search & related questions

Natural-language questions people ask about Missing Data.

What is missing data in a backtest?
It is the holes in your price history — days the market was shut, a stock that did not trade, or a value that got lost. How you fill those holes can quietly bias your results.
What is the safest way to fill a gap in prices?
Carry the last known price forward. That only uses what you already knew at the time. Never fill a gap using a price from later, because that leaks the future into the past.
Why is filling a halted day a problem?
Because during a halt you could not trade at all. If your backtest fills a nice price and books an exit, it is pretending you could sell when you actually could not.
Can I just delete the missing rows?
Only if they are missing at random. Weak or stressed stocks tend to go missing exactly when they crash, so deleting them hides your worst trades and flatters the result.
What happens if a missing price becomes zero?
Your backtest reads it as the stock going to nothing, a total crash. One stray zero from a missing value can create a huge fake loss, so you have to handle those cases on purpose.
How should I handle stock market holidays in my data?
Use a trading calendar instead of filling them. Holidays and weekends are days the market was simply shut, so you should not expect prices then rather than inventing them.

Sources & references

    Last reviewed 12 July 2026. Educational content only — not investment advice. Markets and rules change; verify current conventions with SEBI, NSE/BSE and your broker.

    ← Previous
    Corporate Actions
    Educational content only — not investment advice. Examples use illustrative numbers and simplified models. Backtested results are hypothetical and trading derivatives involves substantial risk. See our Risk Disclosure and SEBI Disclaimer.