Data-quality bias

Also seen as: Bad-data bias, data-error bias

What kinds of data problems distort a backtest?

Six problems account for most of it, and they fail in different directions:

• Unadjusted corporate actions. A split, reverse split, spin-off, or dividend changes the quoted price without changing what a holder owns. An unadjusted series reads that as a return.
• Missing or stale prices. A gap filled by repeating the last price makes an instrument look motionless, which understates its volatility and hides any fall that happened inside the gap.
• Currency mismatches. A price quoted in one currency compared with a value in another produces a difference that is an exchange rate, not a return.
• Coverage that starts later than you assumed. An instrument whose history begins part-way through the tested period contributes nothing before that point, so the early years of the test quietly describe a smaller portfolio than you specified.
• An instrument list that has already dropped the failures. This is survivorship bias reaching the test through the data rather than through your choices.
• Revised or restated figures. A value corrected after its first release, used as though the corrected version had been known on the original date, is look-ahead bias with a data cause.

For a backtest to mean anything, corporate actions such as splits and dividends have to be handled correctly, because they are the events where the quoted price and the holder's actual wealth come apart.

Why splits and dividends have to be adjusted

A split multiplies the number of shares and divides the price by the same factor. A holder's position is worth exactly what it was worth a moment before, so the correct return across a split is zero — which the data can only show if every price before the split is restated onto the post-split scale.

A dividend takes cash out of the company and hands it to the holder, so the price typically drops by roughly the dividend on the ex-dividend date. A price-only series records that drop as a loss and never records the cash, which systematically understates total return. The size of the omission compounds: on a stock yielding about 3% a year, reinvested dividends multiply the outcome by roughly 1.03²⁰ ≈ 1.8× over twenty years, so the dividend component accounts for something like 45% of the total return that a price-only series leaves out entirely.

Not every surprising number is a data error, though. A dividend that appears as a cost rather than income is the expected behavior on a short position, not a defect — see negative dividends.

Worked example: a stock split that looks like a 50% crash

A stock trades at $200. It carries out a 2-for-1 split, and the next session's quote prints at $100.

The holder lost nothing. But a series that keeps the $200 next to the $100 hands every downstream rule a one-day −50% move, and each of them reacts as designed:

• A 10% stop loss fires and sells a position that never lost money — and the strategy's history now contains a trade that would never have happened.
• The max drawdown records a −50% peak-to-trough fall that did not occur, making the strategy look far riskier than it was.
• A momentum or trend rule ranks the stock at the bottom of the universe and rotates out of it.

Correcting the data fixes all three at once, which is the point: the errors here are not in the rules. Reverse splits do the same thing in the other direction, and are worse, because a fabricated one-day gain attracts no suspicion at all.

What does Fincanva do about data quality?

• Backtests run on market and fundamental data sourced from multiple established data providers, updated daily, and a run goes to the latest available market close rather than to today's calendar date.
• Corporate actions such as splits and dividends are reflected in the price history a backtest reads, so a split does not appear as a price fall and dividends are not silently dropped from returns.
• Instruments that were delisted stay in the catalogue and remain searchable, so a strategy can include names that later failed instead of them disappearing from the universe.
• Each instrument carries a permanent identifier separate from its ticker symbol, so a price history stays joined when the symbol changes rather than splitting into two unrelated series — see permanent instrument identifier.
• Coverage varies by instrument: an instrument's coverage window — the span between its first and last available price dates — is the only history a backtest holding it can use, so a strategy's earliest usable date can be later than the simulation start year you set.

That last point is the one to check before reading a long backtest. The rest reduces data-quality bias; none of it removes the need to know what the data behind a particular run covers, and none of it prevents the two related failures where the data is fine and the reporting is not — see cherry-picking bias — or where the timing of otherwise-correct data is what leaks, which is what walk-forward replay addresses. See the nine biases Fincanva helps you avoid for the product behavior in context.

Backtests show what would have happened — not what will. Fincanva provides no financial advice — see Is this financial advice?.