Tinkerings

cur4g16

With the shared notes library and web tools. Updated 13:51 PST, 16 Nov 2026 · all runs

Step 1

RolloutRewardTestEdited accuracyHit capTensors editedTurnsMinutesForward passesPython callsEnded
1+18.4–25.8%73.0%246899.91155editor finished
2+22.3–29.7%69.5%4883122.61368editor finished
3+2.3–9.8%90.6%7113135.7999time limit
4+0.4–7.8%93.8%179133.71364editor finished
5+12.5–19.9%80.9%1109110.91788editor finished
6+16.4–23.8%77.0%710275.9791editor finished
7-6.2–1.2%100.0%88629.4066editor finished
8+35.2–42.6%46.5%248745.71170editor finished
9+17.2–24.6%73.0%17964.31462editor finished
10+1.2–8.6%91.8%246930.1030editor finished
11+14.8–22.3%78.1%48588.61073editor finished
12+12.9–20.3%79.7%167103.31054editor finished
13+26.2–33.6%62.9%2466114.11153editor finished
14+34.8–42.2%37.1%1690106.11572editor finished
15+27.7–35.2%58.6%2278120.61262editor finished
16+22.3–29.7%67.6%2597110.21282editor finished
Mean+16.1–23.6%73.8%14.884.993.210.368.1

Step 2

RolloutRewardTestEdited accuracyHit capTensors editedTurnsMinutesForward passesPython callsEnded
1+4.7–12.1%85.9%4885107.0973editor finished
2+7.8–15.2%85.9%183103.11664editor finished
3+21.9–29.3%70.7%4878128.41361time limit
4+8.2–15.6%87.9%2466121.51153editor finished
5+19.1–26.6%71.9%3072118.11157editor finished
6+7.0–14.5%84.8%4870119.21058editor finished
7+13.7–21.1%77.3%10103125.72477editor finished
8+14.8–22.3%75.4%1378113.61360editor finished
9+25.4–32.8%58.2%17490.01060editor finished
10+27.3–34.8%61.7%4879111.71363editor finished
11+17.6–25.0%74.2%3075125.51559editor finished
12+5.5–12.9%87.9%163132.81249time limit
13+18.0–25.4%75.4%187124.11468editor finished
14+14.1–21.5%79.3%170127.2958editor finished
15+18.0–25.4%72.3%2490121.91770editor finished
16+19.5–27.0%73.0%572114.71258editor finished
Mean+15.2–22.6%76.4%20.877.8117.813.161.8

Step 3

RolloutRewardTestEdited accuracyHit capTensors editedTurnsMinutesForward passesPython callsEnded
1+6.6–14.1%86.3%4885112.21667editor finished
2+2.3–9.8%92.6%18592.51469editor finished
3+13.7–21.1%77.0%4108114.02184editor finished
4+10.2–17.6%87.5%810590.11387editor finished
5+13.7–21.1%77.0%486566.01053editor finished
6+16.8–24.2%77.0%66880.51451editor finished
7+14.8–22.3%78.1%77184.31356editor finished
8+24.2–31.6%65.2%486883.21352editor finished
9+12.9–20.3%80.5%37288.91852editor finished
10+13.3–20.7%77.3%4122121.22395editor finished
11+15.2–22.7%75.8%17972.61463editor finished
12+37.5–44.9%35.5%247347.41061editor finished
13+16.4–23.8%76.6%88991.11769editor finished
14+19.9–27.3%72.7%486761.31154editor finished
15+30.9–38.3%42.2%2410791.51787editor finished
16+23.4–30.9%71.1%18271.41763editor finished
Mean+17.0–24.4%73.3%17.784.185.515.166.4

Step 4

RolloutRewardTestEdited accuracyHit capTensors editedTurnsMinutesForward passesPython callsEnded
1+19.5–27.0%72.7%173117.41357editor finished
2+14.8–22.3%75.0%177118.21657editor finished
3+14.4–21.9%77.0%489113.81968editor finished
4+19.9–27.3%73.4%48104106.51586editor finished
5+19.1–26.6%73.4%38994.41275editor finished
6+3.9–11.3%89.8%18799.41570editor finished
7+24.6–32.0%69.1%485361.9941editor finished
8+18.8–26.2%72.3%308995.71473editor finished
9+23.4–30.9%68.4%55787.71342editor finished
10+18.4–25.8%73.8%47786.71063editor finished
11+27.0–34.4%62.1%27291.81358editor finished
12+22.3–29.7%58.2%487964.91067editor finished
13+30.1–37.5%53.5%1210095.61086editor finished
14+19.9–27.3%71.9%246295.81345editor finished
15+17.2–24.6%75.4%1104121.61884editor finished
16+21.5–28.9%71.1%4883112.41565editor finished
Mean+19.7–27.1%71.1%17.580.997.713.464.8

Step 5

RolloutRewardTestEdited accuracyHit capTensors editedTurnsMinutesForward passesPython callsEnded
1+0.0–44.5%37.1%32910251.91090editor finished
2+0.0–––06239.2753editor finished
3+0.8–45.3%40.2%107955.61362editor finished
4+3.1–47.7%17.2%116457.31447editor finished
5+0.0–––09241.3883editor finished
6+3.5–48.0%32.4%19759.61679editor finished
7+7.0–51.6%22.3%87337.51158editor finished
8+0.0–––09255.71377editor finished
9-2.3–42.2%45.3%248359.61565editor finished
10+0.0–––09246.51375editor finished
11+0.0–––09737.9887editor finished
12+2.7–47.3%17.2%486750.61451editor finished
13+0.0–––09841.51086editor finished
14+0.0–––07550.5964editor finished
15+0.0–––012631.48117editor finished
16-3.5–41.0%35.9%98951.51571editor finished
Mean+0.7–45.9%31.0%27.586.848.011.572.8

Step 6

RolloutRewardTestEdited accuracyHit capTensors editedTurnsMinutesForward passesPython callsEnded
1+0.8–45.3%30.1%2410258.31486editor finished
2+0.4–44.9%23.0%87573.11755editor finished
3+0.0–––010967.11790editor finished
4+0.0–––08750.31570editor finished
5+0.0–––09149.11177editor finished
6+0.0–––05033.2641editor finished
7+0.0–––08340.9774editor finished
8+2.7–47.3%30.5%249060.11669editor finished
9+1.9–46.5%38.3%19959.31083editor finished
10+1.6–46.1%36.7%248062.31365editor finished
11+0.0–––08747.0876editor finished
12+6.6–51.2%17.2%247762.11559editor finished
13+0.0–––08445.4973editor finished
14-0.8–43.8%33.6%97857.91462editor finished
15running––––19.9None–
16+0.0–––05549.41142editor finished
Mean of 15 finished+0.9–46.4%29.9%7.683.154.412.268.1

Step 7

RolloutRewardTestEdited accuracyHit capTensors editedTurnsMinutesForward passesPython callsEnded
1+0.0–––07640.9865editor finished
2+0.0–––08451.21269editor finished
3+0.0–––07939.6967editor finished
4+0.0–––09053.61673editor finished
5+0.0–––09669.61678editor finished
6+0.0–––09240.01080editor finished
7-1.9–42.6%39.1%247851.81758editor finished
8+0.0–––08524.3677editor finished
9+0.0–––06033.5752editor finished
10+0.0–––06439.61052editor finished
11+0.0–––07957.8968editor finished
12+1.2–45.7%30.1%249164.11374editor finished
13+0.0–––07647.51065editor finished
14+1.9–46.5%24.2%410653.31487editor finished
15+0.0–––09157.01376editor finished
16-1.9–42.6%36.3%2411451.51298editor finished
Mean-0.0–44.3%32.4%4.885.148.511.471.2

Step 8

RolloutRewardTestEdited accuracyHit capTensors editedTurnsMinutesForward passesPython callsEnded
1+3.1–47.7%33.6%48266.21564editor finished
2running––––6840.0None–
3-2.7–41.8%38.3%248768.81669editor finished
4+2.7–47.3%16.0%495844.21244editor finished
5+0.0–––011795.91993editor finished
6+0.0–––08545.01072editor finished
7+0.0–––07244.2960editor finished
8-1.2–43.4%34.0%47254.81554editor finished
9-1.9–42.6%36.3%246763.0857editor finished
10+3.9–48.4%18.0%488765.01470editor finished
11+3.9–48.4%28.5%247048.91454editor finished
12+0.0–––06741.3857editor finished
13+6.6–51.2%12.1%89672.21875editor finished
14+2.7–47.3%16.0%487360.21655editor finished
15+0.0–––07530.8667editor finished
16+0.0–––07944.71066editor finished
Mean of 15 finished+1.1–46.4%25.9%15.579.156.312.763.8

Reward: edited minus original accuracy on the reward set (256 answers, original 7.4%), in points. Test: the same on MATH-500 (original 6.6%), not used for training; scored after a step finishes, – until then. Hit cap: share of answers reaching 8,192 tokens (original 93%).