Tinkerings

hero3

With the shared notes library and web tools. Updated 13:51 PST, 16 Nov 2026 ยท all runs

Step 1

RolloutRewardTestEdited accuracyHit capTensors editedTurnsMinutesForward passesPython callsEnded
1+1.2+0.68.6%92.6%176105.61555editor finished
2+9.4+11.116.8%82.4%486652.7754editor finished
3+18.8+16.726.2%78.5%68788.41173editor finished
4+15.6+15.223.0%75.4%67785.91163editor finished
5+33.6+32.841.0%39.8%488466.1972editor finished
6+0.0no editโ€“โ€“0278.6019no tool call in five replies
7+2.7+2.510.2%90.6%196106.21479editor finished
8+27.3+29.434.8%58.6%488659.91469editor finished
Mean+13.6+15.522.9%74.0%19.874.971.710.160.5

Step 2

RolloutRewardTestEdited accuracyHit capTensors editedTurnsMinutesForward passesPython callsEnded
1+32.0+31.439.5%55.1%487962.91362editor finished
2+20.7+22.028.1%64.5%486782.61451editor finished
3+13.3+13.420.7%77.7%484258.0732editor finished
4+25.8+24.233.2%59.0%483449.5527editor finished
5+24.2+22.431.6%65.6%487578.51260editor finished
6+27.3+28.334.8%53.9%4864110.81348editor finished
7+23.1+24.230.5%60.2%485249.4940editor finished
8+27.7+28.135.2%53.1%484249.6732editor finished
Mean+24.3+24.231.7%61.1%48.056.967.710.044.0

Reward: edited minus original accuracy on the reward set (256 answers, original 7.4%), in points. Test: the same on MATH-500 (original 6.6%), not used for training; scored after a step finishes, โ€“ until then. Hit cap: share of answers reaching 8,192 tokens (original 93%).