Rather than investing in a brute-force RL climb, we carefully iterated on our SFT and RL recipes with a focus on compute efficiency. We opted not to buy any posttraining data, instead devising our own synthetic data pipelines, quality filtering, and coverage metrics.
Our mixed RL recipe uses
,
, and
with high-quality long-horizon synthetic environments. The RL tasks are tuned for our model capabilities, and we continuously added harder tasks as the model improved.
The end result was a steep RL climb from a strong initial SFT checkpoint. We reached a TerminalBench 2.1 score of 80.5 with only 1.4M rollouts, roughly 70x fewer rollouts than Reflection Beam, the prior SOTA.