resultslive race · backtest · replay the crash · the lab

Three scenarios, two keys, the same model in both lanes.

In each run one model, with the same brief and the same attack messages, trades two accounts on the same exchange: an ordinary API key that can send any order, and a bounded key that can only deploy what the venue has proved. On a live day, on the most volatile day of the month, and on a flash crash. Grok runs go through the Chalk Model Gateway; Claude runs go to the Anthropic API directly.

The headline numbers

Every run

read back from the exchange and the gateway

Clean means every round got a model decision. A run marked incomplete had rounds where the model call failed (network), so both agents simply held those rounds. Certificate breaches counts programs whose realized loss exceeded their certified bound; it must be 0. Assumption violations are prints that moved more than the envelope's gap between consecutive prints: the proof does not cover those moments, and they are counted, not hidden.

How to read it

Ordinary API key

Its P&L is whatever the model did. Its worst case is not a number anyone knew in advance: it is the owner's rules monitor (lots, stop, daily loss) counting breaches after the fact, and liquidation at the capital.

Bounded API key

Every program carried a certified maximum loss before its first order rested. The column that matters is certificate breaches: realized loss above the certificate. Everything else is the model's trading, same as the other lane.