Backtesting Glossary — Every Term Defined (India)

Absolute return Metric

The total percentage gain or loss over the whole test period, ignoring how long it took. It is simple but not comparable across horizons, which is why it is usually converted to an annualised figure. also: total return

Adjusted prices Data

A historical price series modified to remove the mechanical jumps caused by splits, bonuses and dividends, so returns are continuous. Using unadjusted prices creates false gaps that distort signals and performance around corporate actions. also: adjusted close, back-adjusted prices

Alpha Metric

The component of return attributable to skill or a genuine predictive edge, above what market exposure (beta) alone would deliver. In practice it is the risk-adjusted excess return a strategy adds over its benchmark. also: excess return

Anchored walk-forward Robustness

A walk-forward variant where the training window always starts at the same fixed point and grows over time, rather than rolling with a constant length. It uses all available history for fitting but adapts more slowly to recent regime shifts.

Annualized return Metric

A return rescaled to a one-year basis so results over different periods can be compared. For compounding series this is the CAGR; care is needed not to annualise a short sample, which overstates the reliability of the figure. also: annualised return

Arithmetic mean Metric

The simple average of period returns. It overstates realised compounded growth whenever returns vary, because losses require larger subsequent gains to recover, which is why CAGR uses the geometric mean instead. also: arithmetic average

Autocorrelation Statistics

The correlation of a series with its own past values. Positive autocorrelation in returns inflates naively annualised Sharpe ratios and invalidates the independence assumed by many statistical tests and Monte Carlo trade shuffles.

Average drawdown Metric

The mean depth of all drawdown episodes across a backtest, capturing the typical decline rather than the single worst one. It gives a fuller picture of the everyday pain of holding a strategy than the maximum drawdown alone.

Backtesting Core

Simulating a set of trading rules on historical data to estimate how the strategy would have behaved, including returns, drawdowns and turnover. A backtest is a hypothesis test about a rule set that reduces uncertainty, never a prediction or a promise of future results. also: historical simulation, strategy backtest

Benchmark Metric

A reference return stream, such as buy-and-hold Nifty 50, against which a strategy's performance is judged. A strategy that underperforms a cheap passive benchmark on a risk-adjusted basis has not demonstrated a useful edge.

Benchmark comparison Metric

Evaluating a strategy relative to a passive alternative rather than in isolation, on both return and risk-adjusted terms. Beating cash is trivial; the honest test is whether the strategy beats a simple index after costs and for the risk taken.

Beta Metric

A measure of how much a strategy or instrument moves with its benchmark. Return explained by beta is ordinary market exposure, distinct from alpha, which is the skill-based excess; a strategy can post high returns that are pure beta.

Bid-ask spread Execution

The gap between the highest price buyers will pay and the lowest sellers will accept. Crossing it is an immediate cost of trading and a basic measure of liquidity; on illiquid Nifty option strikes it can dwarf brokerage. also: spread, bid-offer spread

Bootstrap Statistics

A resampling method that repeatedly draws from observed returns or trades, usually with replacement, to build an empirical distribution of a statistic. It lets you estimate confidence intervals for Sharpe, drawdown or expectancy without assuming a specific return distribution.

Brokerage Execution

The fee a broker charges to execute an order, often a flat amount per order or a percentage of turnover. Together with STT, GST, exchange and SEBI charges and stamp duty it forms the friction a realistic Indian backtest must subtract from every trade.

CAGR Metric

The compound annual growth rate, the constant yearly rate that would take starting equity to ending equity over the period. It smooths a lumpy equity curve into one annualised figure and, being geometric, already accounts for compounding. also: compound annual growth rate

Calmar ratio Metric

A risk-adjusted measure equal to annualised return (CAGR) divided by the absolute maximum drawdown over the same period, usually three years. It captures return relative to the worst peak-to-trough loss endured but is dominated by that single deepest drawdown.

Confirmation bias Bias

The tendency to notice and keep evidence that supports a favoured strategy while dismissing evidence against it. In research it shows up as accepting a good backtest uncritically but re-examining a bad one until it improves.

Corporate actions Data

Events such as dividends, splits, bonus issues and mergers that change an instrument's shares or price mechanically. Backtests must adjust historical prices for them, but the adjustment must not leak knowledge of the future action into earlier bars.

Cross-validation Robustness

A resampling scheme that splits data into multiple train-and-test folds so every observation serves in a test set once. Standard cross-validation must be adapted for time series because naive shuffling leaks future information into the training folds.

Curve fitting Bias

Adjusting parameters or adding rules until the equity curve looks good on past data. It is the mechanism by which overfitting happens and is the single most common way backtests come to mislead. also: curve-fitting

Data dredging Bias

Searching exhaustively through data for any pattern that looks profitable, without a prior hypothesis. Because enough random search always finds something, dredged patterns are usually noise that vanishes out of sample. also: data fishing

Data quality Data

The completeness, accuracy, adjustment and point-in-time correctness of the data feeding a backtest. Silent errors, such as bad ticks or wrongly adjusted prices, are a common source of illusory edges that look authoritative.

Data snooping Bias

Reusing the same dataset to test many hypotheses until one appears significant by chance. Without correcting for the number of trials, the winner is often a false positive rather than a real edge. also: data mining bias

Deflated Sharpe ratio Bias

A corrected Sharpe ratio that adjusts for the number of strategy variations tried, the length of the sample and the non-normality of returns. It counters the way testing many configurations inflates the best observed Sharpe purely by chance. also: DSR

Degrees of freedom Bias

The number of free parameters and choices a strategy can flex to fit the data, including rules, thresholds and filters. The more degrees of freedom relative to the amount of data, the easier it is to fit noise and the higher the risk of overfitting.

Downside deviation Statistics

The standard deviation computed only from returns below a target (often zero), ignoring upside variability. It is the denominator of the Sortino ratio and reflects the intuition that investors fear losses, not gains. also: downside risk

Drawdown Metric

The decline in equity from a prior high-water mark at any point in time, expressed as a percentage or amount. A strategy spends much of its life in some level of drawdown between new equity highs.

Drawdown duration Metric

The length of time between an equity peak and the recovery back to that peak, distinct from the depth of the drawdown. Long recovery times test a trader's patience even when the eventual drawdown depth is modest. also: time under water, recovery time

Embargo Robustness

A buffer period immediately after each test fold during which training data is discarded, preventing serially correlated information from bleeding across the boundary. Combined with purging, it makes cross-validation defensible for financial time series.

Equity curve Metric

The running record of a strategy's account value over time. Its slope shows return, its dips show drawdowns, and its smoothness reflects risk-adjusted quality, making it the single most informative backtest chart.

Event-driven backtest Core

A backtest that steps through data one event at a time, passing each bar or tick to the strategy as it would arrive live. It is slower but mirrors production logic and naturally prevents look-ahead bias.

Execution assumptions Execution

The rules a backtest uses to decide how and at what price simulated orders fill, including timing, order type and cost treatment. Optimistic assumptions, such as filling at the signal price with no cost, are a chief reason live results trail backtests.

Expectancy Metric

The average profit or loss expected per trade, computed as win rate times average win minus loss rate times average loss. Positive expectancy after costs is the mathematical definition of an edge.

Fill price Execution

The price at which a simulated or real order is assumed to execute. Choosing the next bar's open rather than the signal bar's close, and adding slippage, keeps the fill honest; filling at the triggering price is a form of look-ahead bias. also: execution price

Forward testing Robustness

Evaluating a fixed strategy on new data as it arrives, whether on paper or with small live capital. Unlike a backtest it cannot be re-run or tuned, so it is a cleaner, if slower, test of genuine edge. also: forward test, out-of-time testing

Geometric mean Metric

The compounded average return per period, found by multiplying the growth factors and taking the root. It is always at or below the arithmetic mean, and the gap, driven by volatility, is why a bumpy equity curve compounds more slowly than its average suggests. also: geometric average

High-water mark Metric

The highest equity value a strategy has reached to date. Drawdown is measured from this mark, and in fund contexts performance fees are often charged only on gains above the previous high-water mark. also: high water mark, HWM

Historical data Data

Past market data used to research and backtest strategies. Its quality, adjustment and point-in-time accuracy determine how trustworthy any backtest built on it can be.

Hold-out set Robustness

A contiguous block of data set aside untouched until a single, final test of the developed strategy. Its value is destroyed the moment you peek, tweak and re-test against it, at which point it silently becomes in-sample. also: holdout set

Hypothesis Core

The economic or behavioural reason a strategy is expected to work, stated before testing. A backtest exists to challenge a hypothesis; an idea with a plausible rationale is far less likely to be a data-mined artefact than one discovered purely by search.

In-sample Core

The portion of historical data used to develop and fit a strategy's parameters. Performance measured here is optimistic by construction, because the rules were chosen to fit exactly this data. also: training data, in-sample data

Information ratio Metric

The active return of a strategy over its benchmark divided by the tracking error, the standard deviation of that active return. It measures how consistently a strategy adds value relative to the benchmark rather than in absolute terms. also: IR

K-fold Robustness

A cross-validation scheme that divides the data into k equal folds, training on k−1 and testing on the remaining one, then rotating. For financial time series it requires purging and embargoing to prevent leakage across the fold boundaries. also: k-fold cross-validation

Kelly criterion Risk

A formula that gives the bet fraction maximising the long-run growth rate of capital for a known edge. Full Kelly is highly volatile and assumes the edge is estimated exactly, so practitioners use a fraction of it to reduce drawdowns.

Liquidity Execution

The ease of trading an instrument in size without moving its price much, reflected in volume, depth and spread. Illiquid instruments produce wider spreads, larger slippage and backtest fills that cannot be achieved live.

Look-ahead bias Bias

Using information in a backtest that would not have been known at the moment of the simulated decision, such as filling on a close the signal was derived from. It manufactures returns that are impossible to capture live. also: lookahead bias, forward-looking bias

Lookback window Core

The number of past bars or the time span a rule or indicator uses to compute its current value. Too short makes signals noisy, too long makes them slow, and the choice is a frequent target of overfitting. also: lookback period

Market impact Execution

The adverse price movement caused by one's own order consuming available liquidity. It matters most for large orders in thin instruments and is routinely underestimated by backtests that assume unlimited liquidity at the quoted price.

Maximum drawdown Metric

The largest peak-to-trough decline in equity before a new peak is reached, usually stated as a percentage. It is the core measure of pain and of how much capital a strategy can put at risk, and it defines survivability more than return does. also: max drawdown, MDD

Money management Risk

The set of rules governing how much capital is risked per trade and across the portfolio, independent of the entry signal. Sound money management can keep a mediocre edge survivable, while poor sizing can ruin a genuinely profitable one.

Monte Carlo simulation Robustness

A technique that generates many randomised scenarios, for example by resampling the trade sequence, to study the range of possible outcomes rather than the single realised path. It helps estimate the distribution of drawdowns and how much of a result was luck. also: Monte Carlo analysis

Multiple testing Statistics

Running many statistical tests or strategy variants and judging each by the usual significance threshold, which guarantees false positives as the count grows. Corrections such as Bonferroni or a deflated Sharpe adjust the bar for the number of trials. also: multiple comparisons

OHLC Data

The open, high, low and close prices summarising trading over a fixed interval such as a day or a minute. It is the most common compact format for charting and bar-based backtests, though it hides the path within each bar. also: OHLCV, candle, bar

Optimisation Robustness

Searching over parameter values to find the settings that performed best on historical data. It is useful but dangerous: the more combinations tried, the more the best result reflects luck, which is why walk-forward and out-of-sample checks are essential. also: parameter optimisation, optimization

Out-of-sample Robustness

Data deliberately withheld during development and used only afterwards to test a fixed strategy. It gives a more honest estimate of live behaviour precisely because the rules never saw it during tuning. also: OOS, hold-out

Out-of-sample testing Robustness

The practice of evaluating a finished, parameter-frozen strategy on data reserved from development. It is the most basic guard against overfitting, though a single one-shot test on one period is weaker than rolling walk-forward evaluation.

Overfitting Bias

Tuning a strategy so closely to historical data that it captures noise rather than a repeatable pattern, producing an excellent backtest that fails live. More parameters and more tuning iterations both increase the risk.

p-hacking Bias

Tweaking data choices, filters or parameters until a result crosses a significance threshold. In backtesting it is the statistical face of curve-fitting: the reported result looks significant only because the failed attempts are hidden.

p-value Statistics

The probability of observing a result at least as extreme as the one measured if the strategy had no real edge. A small p-value is weak evidence against no-edge, but it is easily gamed by testing many variants, so it must be adjusted for multiple testing.

Paper trading Robustness

Running a strategy against live market data with simulated orders and no real money. It surfaces data-feed, timing and operational problems that a backtest cannot, but fills are still assumed rather than truly executed. also: simulated trading

Parameter plateau Robustness

A wide region of parameter values that all deliver similar performance, indicating the edge does not depend on one lucky setting. A plateau is reassuring; an isolated spike surrounded by poor results usually will not survive live.

Parameter sensitivity Robustness

How much a strategy's performance changes when its input parameters are varied slightly. Robust strategies sit on broad plateaus where nearby settings all perform similarly; a sharp, isolated peak is a classic sign of curve-fitting. also: sensitivity analysis

Payoff ratio Metric

The average winning trade divided by the average losing trade, measuring how large wins are relative to losses. Combined with the win rate it determines expectancy; a low win rate can still be profitable with a high payoff ratio. also: win-loss ratio, reward-risk ratio

Point-in-time data Data

Historical data reconstructed to show exactly what was known at each past moment, including figures as originally reported before later revision. Using it prevents look-ahead bias from restated fundamentals or changed index membership. also: PIT data

Position sizing Risk

Deciding how many units, lots or contracts to trade on a given signal, based on capital, risk per trade and stop distance. It usually matters more to long-run results and to the shape of the drawdown than the entry rule itself.

Probability of backtest overfitting Bias

A framework that estimates how likely the best in-sample configuration is to underperform out of sample, given how many variations were tried. A high value warns that the selected strategy is probably an artefact of the search. also: PBO

Profit factor Metric

Gross profit divided by gross loss across all trades. A value above 1 means the strategy made money in the sample, but a single large winner can inflate it, so it needs a decent sample size and realistic costs to be meaningful.

Purging Robustness

Removing training observations whose labels overlap in time with the test set, so information from the test period cannot leak into training. It is essential when a label spans multiple bars, as in triple-barrier or holding-period labelling.

R-multiple Metric

A trade's profit or loss expressed in units of the initial risk (1R) taken on it, so a trade risking 1R that gains twice its risk is +2R. It normalises outcomes across trades of different sizes and turns expectancy into a size-independent figure. also: R multiple

Recovery factor Metric

Net profit divided by the absolute maximum drawdown over the test. It shows how many times the strategy earned back its worst decline, but like the Calmar ratio it is dominated by the single deepest drawdown observed.

Regime change Robustness

A shift in the prevailing market environment, such as trending to range-bound or low to high volatility, in which a strategy's behaviour changes. Many strategies work in one regime and lose in another, so regime shifts are a leading cause of live underperformance. also: regime shift

Resampling Statistics

Any method that repeatedly draws new samples from existing data, such as bootstrapping trades or reshuffling their order, to gauge the variability of a result. It underpins Monte Carlo drawdown estimates and confidence intervals in backtesting.

Research workflow Core

The disciplined sequence from idea and hypothesis through data preparation, in-sample development, out-of-sample and robustness testing, to forward testing. A defined workflow limits how many times the same data is reused and keeps results honest. also: research process

Risk of ruin Risk

The probability that a sequence of losses reduces capital below a threshold at which trading can no longer continue. It rises sharply with larger position sizes and with a thin or negative expectancy, and it is a comparative guide rather than an exact figure.

Risk-adjusted return Metric

Return expressed relative to the risk taken to earn it, rather than in isolation. Ratios such as Sharpe, Sortino and Calmar are all risk-adjusted measures, and comparing strategies on them is fairer than comparing raw returns.

Risk-free rate Metric

The return available with essentially no risk, in India typically proxied by short-dated government treasury bills. It is subtracted from strategy returns to compute the excess return used in the Sharpe and Sortino ratios.

Robustness Robustness

The degree to which a strategy keeps working under conditions different from those it was built on, such as new data, higher costs or a changed regime. Robustness is prized over peak backtest performance because it is what tends to persist live.

Sample bias Bias

A distortion arising when the data sample used for testing is not representative of the conditions the strategy will face, for example a period containing only a bull market. Conclusions drawn from an unrepresentative sample do not generalise. also: sampling bias

Sample size Statistics

The number of independent trades or observations behind a metric. Small samples make win rate, Sharpe and profit factor unstable and easy to misread; a spectacular metric on a few dozen trades is usually noise or overfitting.

Scenario analysis Robustness

Evaluating how a strategy would behave under specific historical or hypothetical conditions, such as a 2008-style crash, a volatility spike or a liquidity freeze. Unlike random Monte Carlo, scenarios are chosen deliberately to probe known weaknesses.

Selection bias Bias

Distortion introduced when the instruments, period or trades studied are not representative of what would have been traded live. Cherry-picking a favourable index constituent or a calm time window are common forms.

Sharpe ratio Metric

A risk-adjusted return measure equal to excess return divided by the standard deviation of returns. It rewards consistency and penalises volatility, but treats upside and downside swings alike and assumes roughly normal returns.

Signal Core

A discrete instruction produced by strategy logic, such as go long, go short or flat, derived from data at a point in time. Signals are separate from execution, which decides how the resulting order is actually filled.

Slippage Execution

The difference between the price a strategy expected and the price at which the order actually filled. It grows with order size, thin liquidity and fast markets, and is a leading reason live results trail backtests.

Sortino ratio Metric

A variant of the Sharpe ratio that divides excess return by downside deviation only, ignoring upside volatility. It better reflects an investor's real concern with losses, and comparing it to the Sharpe shows how much of the volatility is favourable.

Stability testing Robustness

Checking that a strategy's behaviour holds across time periods, instruments and small perturbations to its inputs and data. Stable performance across subsamples is evidence of a real effect rather than a fit to one stretch of history. also: robustness testing

Standard deviation Statistics

A measure of how far returns typically spread around their mean. In backtesting it is the usual proxy for total risk and the basis of the Sharpe ratio, though it understates tail risk when returns are not normally distributed. also: sigma

Stationarity Statistics

The property of a series whose statistical characteristics, such as mean and variance, stay stable over time. Prices are typically non-stationary while returns are closer to stationary, and many backtesting methods implicitly assume some stability that markets violate.

Statistical significance Statistics

The judgement that a result is unlikely to have arisen by chance under a no-edge assumption. In backtesting it is necessary but far from sufficient, because significance can be manufactured by trying enough configurations.

Strategy lifecycle Core

The full arc of a systematic strategy: hypothesis, backtest, validation, paper and live deployment, ongoing monitoring, and eventual decay or retirement. Every strategy degrades as its edge is arbitraged away or the regime changes, so retirement is part of the process.

Stress testing Robustness

Deliberately subjecting a strategy to extreme but plausible conditions, for example doubling costs, widening spreads or replaying a crisis, to see how it degrades. A robust edge weakens gracefully; a fragile one collapses.

STT Execution

Securities Transaction Tax, a levy charged on trades in Indian securities and derivatives. Because it applies per transaction and differs by segment and side, ignoring it can make a high-frequency Indian backtest look profitable when it is not. also: Securities Transaction Tax

Survivorship bias Bias

A distortion caused by testing only instruments that still exist today, silently excluding those that were delisted, merged or went bankrupt. It flatters results because the failures are missing from the sample.

Survivorship-free data Data

A dataset that retains instruments which were later delisted, merged or removed from an index, reconstructed as the universe actually looked at each past date. Testing on it is the correct fix for survivorship bias. also: point-in-time universe

Tick data Data

The most granular market data, recording individual trades or quote updates with timestamps. It is essential for execution and microstructure research but is large, costly to store and demanding to process.

Tracking error Metric

The standard deviation of the difference between a strategy's returns and its benchmark's returns. It quantifies how tightly the strategy hugs the benchmark and is the denominator of the information ratio.

Trade log Metric

The complete record of every simulated or executed trade, with entry, exit, size, costs and profit or loss. It is the raw material from which all trade-based metrics such as win rate, expectancy and profit factor are computed.

Trade sequencing Robustness

The specific order in which winning and losing trades occurred in a backtest. Because the realised sequence is only one of many possible orderings, shuffling it in a Monte Carlo run reveals drawdowns the actual path happened to avoid.

Trading rules Core

The explicit, unambiguous specification of when a strategy enters, exits and sizes positions, together with its filters and risk controls. Rules must be precise enough that the same data always produces the same action, otherwise a backtest is not reproducible. also: strategy rules

Transaction costs Execution

The full cost of trading, including brokerage, exchange fees, taxes such as STT, the bid-ask spread and slippage. Realistic cost modelling in a backtest often turns an apparent edge into a loss, especially for high-turnover strategies. also: trading costs

Turnover Execution

How much a strategy trades over a period, often expressed as traded value relative to capital. High turnover multiplies transaction costs, so a strategy with a thin per-trade edge can be profitable on paper yet net-negative after realistic frictions.

Ulcer index Metric

A risk measure that computes the root-mean-square of drawdown depths over time, penalising deep and prolonged declines more than shallow ones. Because it only considers downside, it captures the discomfort of holding a strategy better than standard deviation.

Underfitting Bias

Building a model too simple or too constrained to capture the genuine structure in the data, so it performs poorly both in and out of sample. It is the opposite failure to overfitting and is a real, if less discussed, risk.

Underwater curve Metric

A plot of the current drawdown over time, sitting at zero at each new equity high and dipping below whenever equity is beneath its prior peak. It visualises both the depth and the duration of every drawdown episode. also: underwater plot, drawdown curve

Validation process Core

The staged battery of tests that separates a genuine edge from an overfit one: out-of-sample data, walk-forward analysis, parameter sensitivity, Monte Carlo and forward testing. Validation can only fail to reject an edge; it can never prove one exists. also: validation

Vectorised backtest Core

A backtest that computes signals and returns across the entire price series using array operations at once. It is fast for research but can hide look-ahead bias and struggles to model order-level details such as partial fills and queue position. also: vectorized backtest

Volatility Metric

The degree of variation in returns, usually measured by the annualised standard deviation of period returns. It is the denominator of the Sharpe ratio and a common input to position sizing, but it treats upside and downside movement symmetrically. also: return volatility

Volatility drag Metric

The reduction in compounded growth caused by the variability of returns, roughly half the variance per period. It is why the geometric mean falls below the arithmetic mean and why smoother equity curves compound faster for the same average return. also: variance drag

Walk-forward analysis Robustness

A validation method that repeatedly optimises on one window and tests on the next unseen window, rolling forward through history. It approximates how periodic re-optimisation would have performed and stitches the out-of-sample segments into one continuous equity curve. also: walk-forward optimisation, WFA

Why backtesting fails Core

The collection of reasons a strong backtest does not translate live: overfitting, look-ahead and survivorship bias, unrealistic costs, regime change and too small a sample. Most failures trace back to the researcher fooling themselves rather than to bad luck.

Win rate Metric

The fraction of trades that close profitable. A high win rate is neither necessary nor sufficient for profitability; it must be read together with the size of wins versus losses. also: hit rate, win percentage