The 12-gate standard.

Is every input known at the moment of the decision?

We check the time zone and bar-stamping of every file against an independent source, rerun the rule at −3 to +3 minutes, test every filter at lag 0, 1 and 2 days, and check that merges do not duplicate rows. An effect at lag 0 that vanishes at lag 1 is look-ahead.

Catches: The most common source of spectacular backtests: information from the future.

Failed here: Machine-learning intraday forecaster, Opening Volatility Stretch on DAX, Bund and Nasdaq, USD/JPY Tokyo fix fade

What is left after pessimistic fills?

Stop entries that gap through the trigger fill at the open, not at the trigger. A stop and a target touched in the same bar count as a loss. No entries in the first minute of the cash session. Spreads widen around midnight and news. The edge must stay above +0.05R per trade and must not live in the gap entries (gap dependency of 0.15 or less).

Catches: Profits that exist only because the simulator filled at prices nobody could trade.

Failed here: ICT Silver Bullet (10:00-11:00 New York), Momentum pullback to the 9 and 21 EMA, Opening Volatility Stretch on DAX, Bund and Nasdaq, Intraday momentum into the close

Does an independent engine produce the same trades?

The fast research simulator is replayed through an event-driven engine on the same bars. The target is a 100.00% identical trade list. This is code parity, not data parity; data is G10's job.

Catches: Simulator bugs: look-ahead in indicators, wrong order of fills.

Is the result better than the best of all the variants that were tried?

Every variant tested raises the bar. A single pre-registered idea must reach t 3.43; a parameter search must beat the maximum that pure noise produces over the same number of variants (deflated Sharpe ratio, or a family-wise null built by shuffling the calendar).

Catches: Data mining: the best of many backtests looks good by chance.

Failed here: ICT Silver Bullet (10:00-11:00 New York), Momentum pullback to the 9 and 21 EMA, Machine-learning intraday forecaster, Weak-day reversal (lower high, lower low), Opening Volatility Stretch on DAX, Bund and Nasdaq, We mined 10,248 strategy rules on a GPU, 60/40 rebalancing pressure, Intraday momentum into the close, USD/JPY Tokyo fix fade

Does the rule beat random entries, a flipped rule and the market's own drift?

The same engine is run on random entry days, on the opposite signal and on always-long or always-short. A real rule must beat all of them; a long-only rule on an index that rose tenfold must beat simply holding the index.

Catches: Results that are just the market going up, or an engine that makes money from noise.

Failed here: ICT Silver Bullet (10:00-11:00 New York), Machine-learning intraday forecaster, We mined 10,248 strategy rules on a GPU, Intraday momentum into the close, USD/JPY Tokyo fix fade

How often does the chosen variant fail out of sample when the data are re-split?

Combinatorial purged cross-validation (CPCV) gives a probability of backtest overfitting (PBO). When the test uses one sequential Discovery, Validation and Holdout split instead, the page says so.

Catches: Parameters picked to fit one particular history.

Do neighbouring parameters work too, and does the rule hold when re-fitted forward?

A real effect is a plateau: moving a parameter by one step should not destroy it. A single sharp peak surrounded by losers is a fitted accident.

Catches: Needle-in-a-haystack parameters.

Failed here: ICT Silver Bullet (10:00-11:00 New York), We mined 10,248 strategy rules on a GPU, USD/JPY Tokyo fix fade

Does it survive realistic costs, and twice those costs?

Results are computed at the real spread and commission of the venue, then at 2x and 3x. We report the break-even cost. Most intraday ideas on retail CFDs die here.

Catches: Edges smaller than the spread.

Failed here: ICT Silver Bullet (10:00-11:00 New York), Momentum pullback to the 9 and 21 EMA, Machine-learning intraday forecaster, Opening Volatility Stretch on DAX, Bund and Nasdaq, 60/40 rebalancing pressure, Intraday momentum into the close

Does it hold across 2008, 2011, 2015-16, 2020, 2022 and recent years?

Per-regime t-statistics and a test of equal means. A strategy is not required to win in every regime, but a regime with t below −2 is a veto.

Catches: Strategies that only worked in one market environment.

Failed here: Momentum pullback to the 9 and 21 EMA, Intraday momentum into the close

Does the frozen rule work on data nobody has looked at?

Rule, parameters and costs are frozen and hashed before the holdout is opened, once. One-sided t of at least 2.326 (1% level). This is the only statistical gate that can approve a strategy.

Catches: Everything the search tuned itself to.

Failed here: Nasdaq breakouts on high-fear days, Opening Volatility Stretch on DAX, Bund and Nasdaq, 60/40 rebalancing pressure, Intraday momentum into the close

Would the account survive the firm's real rules, event by event?

Daily loss limits reset at the firm's server midnight, floating losses carried over midnight, trailing drawdowns, consistency rules and the largest safe risk per trade, simulated trade by trade.

Catches: Strategies that are profitable on paper but breach a daily limit.

Does the edge raise the chance of passing a challenge above what luck alone gives?

Thousands of simulated challenges built from blocks of real trading days, at half the measured edge and at zero edge on the same paths. The difference in pass rate must be positive with 95% confidence, and the expected value after fees must be positive.

Catches: Strategies that pass challenges only as often as a coin flip would.

Failed here: Opening Volatility Stretch on DAX, Bund and Nasdaq