17 Bugs in My Backtest Engine (And What They Taught Me)

Every trading strategy I run starts with a backtest: a simulator that replays years of market candles through the strategy and reports what would have happened. The bot behind it is a real system — ~26,000 lines of strict TypeScript, 272 tests, an LLM verifier that vets trade signals, a risk manager with hard limits.

Here is the uncomfortable part: the strategy code had tests, but the simulator was mostly trusted. And the simulator is the oracle. If the backtest lies, every strategy tuned against it is optimized for a market that does not exist. So I treated the backtest engine the way I would treat any production system under test — two full adversarial audit rounds, followed by three parallel external reviews. Result: 17 bugs found and fixed, two of them critical.

This post is a walkthrough of the most instructive ones, grouped by the failure pattern they represent — because the patterns generalize far beyond trading.

# The setup

The engine simulates an exchange: balances (free/locked per asset), limit and market orders, leverage with borrowed margin, funding payments, and liquidation. Three files carry the simulation: BacktestExchange.ts (order execution and balances), BacktestRunner.ts (the candle loop), BacktestReport.ts (metrics).

Audit one covered the main flows and fixed 8 bugs. Audit two was a deliberate second pass — same files, fresh eyes, looking for what the first pass missed. It found 9 more. That alone is lesson zero: the first audit of a complex stateful system never gets everything.

# Pattern 1: State that outlives itself

The liquidation handler had two bugs that compounded. First, it sold only free base balance — and only afterwards cancelled open orders, unlocking the rest. The unlocked base was never sold, leaving dead balance behind after a "full" liquidation. Second, and worse:

// after liquidation, none of these were reset:
positionQty      // funding kept being deducted
positionCost     // avgPrice = positionCost / positionQty → garbage
borrowedQuote    // net equity calculation → garbage

A liquidation mid-backtest poisoned every subsequent candle. The engine did not crash — it just quietly produced wrong numbers. That is the defining trait of oracle bugs: nothing throws, the report just lies.

A variant of the same pattern lived on the strategy side. executeDirectionalMode() recorded a new position in memory before the order was confirmed. If the exchange rejected the order, the position still existed — a ghost. The position manager then generated close signals for it, producing spurious orders that burned capital on nothing. The fix was structural: positions are created only in onOrderExecuted, on confirmed fill. Never record state for an action the world has not confirmed.

# Pattern 2: The silent clamp

The most expensive habit in the codebase was Math.max(0, ...) used as error handling:

// market sell with insufficient free balance:
bBal.free = Math.max(0, bBal.free - qty)   // silently sells less than asked

// funding deduction with empty free balance:
qBal.free = Math.max(0, qBal.free - funding)  // funding silently vanishes

// fee deduction on a second fill in the same candle:
qBal.free = Math.max(0, qBal.free - fee)   // money from thin air

Each clamp turned an invariant violation into a slightly wrong number, with no signal to the caller. The strategy believed a position was closed; the balance said otherwise. A silent clamp is a swallowed exception with extra steps. The fix everywhere was the same: make the shortfall explicit — partial fill reported back to the caller, deduction taken from locked, or an outright reject.

This pattern is everywhere in application code, not just trading. Any place a value is forced into range "to be safe" is a place where a real problem becomes invisible.

# Pattern 3: Accounting that double-counts or under-counts

Leverage broke equity math twice, in opposite directions.

Under-counting: equity was computed as quote + base × price, ignoring the borrowed margin entirely. A $10,000 position with $5,000 borrowed reported $15,000 in equity — max drawdown looked smaller than it was.

Double-counting: after the first fix, a second bug surfaced in the yearly runs. For market buys, margin was deducted from free but never returned after fill (limit orders unlocked it; market orders did not), and then getNetEquity() subtracted the same position cost again. Every directional entry "burned" its notional value from reported equity.

Together with the ghost-position bug, this produced my favorite number of the whole exercise: a backtest that started with $10,000 reported $795 in capital for 2025. Two bugs, each survivable alone, interacting into something absurd. The absurdity was the gift — $795 is obviously wrong, which is the only reason anyone dug in. The dangerous bugs are the ones that move your result by 8%.

# Pattern 4: Reused evidence

The P&L tracker matched sells to buys FIFO — but never marked buys as consumed. Sell 1 BTC against a 1 BTC buy, then sell another 0.5 BTC, and the engine happily matched it against the same buy again. Every buy could be sold multiple times; realized P&L was fiction. The fix was one field: consumedQty on each trade record.

A related one: grid buys and DCA sells shared a single FIFO pool, so a DCA sell could be matched against a grid buy, producing meaningless per-strategy P&L. Matching needs to respect the boundaries of the thing being measured.

# Pattern 5: Hardcoded context

Five bugs were the same bug: a constant baked in from whatever data the author had open that day.

The Sharpe one deserves emphasis: a headline quality metric, off by a factor of two, because of Math.sqrt(365 * 24). The fix in every case was to derive the constant from the data — candle gap from timestamps, base asset from symbol info — not to pick a better constant.

# Pattern 6: What the external reviewers caught

After the two internal audits, I ran three parallel external reviews with different angles: order execution, financial realism, market microstructure. They found a different class of problem — not broken code, but a broken model of reality:

• Fee structure. The reviewers assumed Binance-style maker/taker fees. Checking the actual exchange docs showed no maker/taker split at all in leverage mode — a flat 0.06%. Platform-specific research beats reasonable assumptions.
• Free leverage. The default funding rate was 0, meaning borrowed margin cost nothing. At 3× leverage held for 30 days, real funding is roughly −8% of margin; the backtest showed 0.
• Look-ahead bias. The strategy saw candle N's close and executed at candle N's close. Real strategies act on the previous close and fill at an unknown next price. This systematically inflates win rate, and no unit test will ever catch it — it is a modeling error, not a code error.

# What this taught me

Test the oracle before the system. My 272 tests validated the bot against the simulator; almost nothing validated the simulator against reality. Any system whose output drives decisions — a backtest, an eval harness, a metrics pipeline — deserves its own adversarial pass.

Absurd numbers are leads, not noise. $795 from $10,000 was a bug report written by the data itself. Build the habit of chasing outputs that look wrong instead of rationalizing them.

Adversarial review finds what authorship cannot. I wrote the engine; my first audit found the bugs I was capable of seeing. The second pass found 9 more, and the external reviewers found the assumptions I did not know I was making. This is the same reason LLM-as-judge evals use a different model than the one being judged.

The same patterns apply to AI systems. Silent clamps = guardrails that swallow failures. Ghost state = tool calls assumed successful before confirmation. Hardcoded context = eval thresholds copied from someone else's benchmark. Reused evidence = test cases leaked into training prompts. The domain changes; the failure patterns do not.