Weight-space programming
Can a model improve another model by editing its weights directly, with no gradients [1] [2]? We set this up as a teacher and a student.
- Student: Qwen3.5-0.8B. It solves math problems from MATH and GSM8K. This is the inner loop.
- Teacher: Claude Opus 5.5 or Inkling 2 Small. It works in many steps. It runs the student, reads its answers and activations, writes an edit to its weights in code, and tests the edit on problems. At the end it submits its best edit.
- Training the teacher: we train Inkling 2 Small with RL. The reward is how much its submitted edit improves the student's accuracy on problems it never saw.
The student fails at math mostly because it does not stop reasoning. It writes "Wait..." and rechecks its work until it runs out of tokens. This happens in about 95% of answers. Every edit below tries to fix this, in one of two ways, or both:
- Stop the loop. Lower the score of "Wait", so the model stops re-checking.
- Close the reasoning. Raise the score of "</think>", so the model ends its thinking and gives an answer.
A score is the share of correct final answers. Edits are scored on three sets of problems:
- Train: MATH and GSM8K training problems, used during the search. Some are set aside as validation, to check the final edit.
- Test: MATH-500, scored once on the submitted edit.
- Reward: 128 other MATH training problems, used to score each RL rollout. The teacher never sees them.
The localization and edit patterns the teacher models deploy resemble mechanistic interpretability methods, which I illustrate below. We now list these methods in the teacher's prompt. But the teachers used them before we did.
The page has three parts:
- How to diagnose: how the teacher finds what to change.
- How to update: how it changes the weights.
- Every edit tried so far, and what the edits show.
How to diagnose
A diagnosis finds the units that make the model write the wrong next token. Here that token is "Wait". The teachers used three methods from the interpretability literature:
- Direct logit attribution: which unit writes "Wait"?
- Difference of means: which unit fires differently just before "Wait"?
- Token directions: no measurement. Use the token's own direction.
Method 1 is built up below on a one-layer network.
1. Direct logit attribution [3] [4]
A small network
Start with a toy network. It has the same parts as the student, only smaller.
| Toy | Qwen3.5-0.8B | |
|---|---|---|
| Layers | 1 | 24 |
| Residual stream | 3 | 1024 |
| MLP units per layer | 4 | 3584 |
Because h is the residual plus what the MLP added, the "Wait" logit splits the same way, with u = U[:, Wait]:
Figure 2 opens up the MLP part. It has one term per unit.
Where a logit comes from
x [4]W [4 × 3]U [3 × 3]row
W for one weight, or a unit on the right for its row.x[j] × W[j][d] × U[d][Wait]. One row sums to one unit's contribution. All rows sum to the MLP part, 2.5.Across positions and layers
Figure 2 shows one position: one token whose next token is "Wait". A looping answer writes "Wait" many times. Let P be all those positions. Unit j is one MLP neuron: entry j of x, row j of W. For each layer L and unit j, add the unit's contribution over P:
rms(h[p])is the final norm. It scales every term at one position by the same number.Chas one number per unit per layer.- The unit with the largest
Cis the candidate to edit. Ccounts only what a unit writes straight into the final vector. So we confirm a candidate by editing it and generating answers.
Each teacher ran this search in a different way:
- Opus: layers 12 to 23. It summed the positions before "Wait" in 2 answers.
- Inkling: layers 22 and 23 only. It summed every position of 1 answer.
The five units with the largest C:
| Rank | Opus: layer, unit | C | Inkling: layer, unit | C |
|---|---|---|---|---|
| 1 | 16, 998 | 7.51 | 22, 11 | 55.55 |
| 2 | 23, 36 | 3.09 | 22, 3 | 50.07 |
| 3 | 22, 194 | 1.82 | 23, 46 | 26.83 |
| 4 | 23, 113 | 1.14 | 23, 7 | 19.95 |
| 5 | 22, 11 | 1.05 | 23, 240 | 19.85 |
- Opus found one unit far ahead of the rest: unit 998 in layer 16.
- Inkling never searched layer 16. Its top units were close to each other.
- The two scales differ because the sums cover different numbers of positions.
In our runs
Opus then edited unit 998 of layer 16. Both edits helped:
| Edit | Train accuracy |
|---|---|
| None | 7.0% |
| Zero the unit | 14.5% |
| Flip the unit | 17.6% |
2. Difference of means [6] [7]
Method 1 asks which unit writes "Wait". Method 2 asks which unit fires differently just before "Wait". It needs only the activations x. It compares two sets of positions:
P: positions whose next token is "Wait", as in method 1.Q: positions to compare against. These are either the other positions of the same answer, or positions in answers that finished cleanly.
Example: unit 2 fires 3 and 2 at the positions in P, and 0 and 1 at those in Q. Its means are 2.5 and 0.5, so the difference is 2. It fires mostly before "Wait".
The units with the largest d are the candidates. On the residual stream h, the same difference gives a direction instead of a unit. That direction can be removed from the output matrices [6].
3. Token directions
This method skips the measurement. Each token has a column in U. That column is a direction in the residual stream. We can remove it from the output matrices of every layer, or add it. Removing it from every layer is called directional ablation [6].
Inkling used this method in its first search. It saw that layers 15 to 21 push "</think>" down. So it added the "</think>" direction to the MLPs of layers 20 to 23. Test accuracy rose from 6.6% to 20.2%.
How to update
A diagnosis gives a unit or a direction. The teacher then changes the weights there. It only edits output matrices: the matrix W from How to diagnose, which writes an MLP's result back into the residual stream, and the same matrix in attention layers. No other weights are changed.
There are two kinds of edit:
- Change a few units. Each unit has its own slice of the matrix: what it writes. Changing it changes only that unit.
- Add a rank-one term
a·bᵀ[5]. This touches every unit. It changes what the whole matrix writes along one direction.
Opus's and Inkling's largest gains came from rank-one terms.
The figures in this section draw W as it is stored in the model file, [1024 × 3584]. Unit j is column j here. In How to diagnose we used the transpose, so unit j was row j. It is the same matrix.
W is down_proj. Unit j adds column j of W to the residual stream, scaled by its activation x[j]. The attention output matrix does the same job for attention. Only these two matrices are edited.The figure below shows a small W before and after an edit. Rows are residual dimensions. Columns are MLP units. positive weight, negative weight, zero. A stronger colour means a larger weight. marks the cells the edit changes. The rest are faded.
Kinds 1 to 4 change one column. The other columns stay the same. Kinds 5 to 7 add a rank-one term a·bᵀ, so they can change every cell. Cell (i, j) changes by a[i] × b[j].
Column 3 changes sign. 6 of 60 cells change.
Hover over a cell, tap it, or focus the figure and use the arrow keys to read its value.
W[:, j] = -W[:, j]Unit
j writes the negative of its old vector. Every token it raised, it now lowers.Example: Opus flipped unit 998 in layer 16. It kept this as one half of its final edit.
8. Change what a unit reads
Kinds 1 to 7 change output matrices: what a unit writes. Two later rollouts changed an input matrix instead: what layer 0's linear attention reads. They found query and key rows that were all zero. They filled each one with a copy of the same row from the next head.
rebase1 s2-r5 and rebase2 s3-r2. These are the only edits that aimed at neither "Wait" nor "</think>". Neither helped.Variants seen in later runs
- Redirect a unit, toward "</think>". One rollout took units that an earlier edit had zeroed and made them write the "</think>" direction. It never tested the edit, and it undid the earlier fix. Reward −32.4.
- Amplify a direction in one matrix only. Two rollouts raised "</think>" only in layer 23's attention output, at high strength. Attribution showed this one matrix gives almost all of the "</think>" logit where a real answer ends its reasoning. Reward +21.1 each.
Every edit tried so far
The tables list every edit, grouped by teacher and run. Most numbers are changes in accuracy, in points: edited minus original, on the same problems. Numbers with % are plain accuracies. α is the strength of an edit.
Claude Opus 5.5 1 search
Claude Opus 5.5 · trajectory
| Edit | Kind | Aimed at | Where | Strength | Measured | Outcome |
|---|---|---|---|---|---|---|
| Strengthen the end-of-thinking direction | amplify a direction | close the reasoning | attention and MLP, layers 14 to 22 | α 0.5 | train 7.4% → 7.0% | no effect, dropped |
| Zero unit 998 | remove a unit | stop the loop | MLP, layer 16 | train 7.0% → 14.5% | replaced by the flip | |
| Flip unit 998 | flip a unit | stop the loop | MLP, layer 16 | train 17.6% | kept as one half | |
| Flip + direction | flip + amplify | both | layers 14 to 22 | α 2.0 | train 37.1% | rejected: a quarter of finished answers skipped the reasoning |
| Flip + direction on nine chosen matrices | flip + amplify | both | attention, layers 17 to 21; MLP, layers 15, 16, 19, 22 | α 2.5 | validation +26 to +29 on three sets | submitted. Test 7.6% → 34.8% |
Each half alone, on train problems:
- direction only: 15.6%
- flip only: 17.6%
Inkling 2 Small 7 runs
first search
Inkling 2 Small, first search · trajectory
Strengthen the end-of-thinking direction in the MLPs of layers 20 to 23. Aimed at closing the reasoning. One row per strength:
| α | Train | Outcome |
|---|---|---|
| 0.5 | +4.5 | |
| 0.7 | +8.9 | |
| 0.85 | +21.4 | submitted. Test 6.6% → 20.2% |
| 1.0 | +30.4 | skipped the reasoning in about 30% of answers |
On validation the two strongest settings gave +22.7 and +20.3.
second search
Inkling 2 Small, second search · trajectory
| Edit | Kind | Aimed at | Where | Measured | Outcome |
|---|---|---|---|---|---|
| Zero 11 units that write "Wait" | remove units | stop the loop | MLP, layers 5, 15, 16, 18, 19, 20, 22, 23 | validation: GSM8K +7 to +10, MATH 0 to +1.5 | kept as one half |
| Also double 8 units that fire where finished answers close their reasoning | amplify units | close the reasoning | MLP, layers 16, 18, 19, 22, 23 | validation MATH +6.7 | submitted. Test 6.6% → 11.8%. Tripling was worse. |
first RL test (smoke3)
Inkling 2 Small, first RL test (smoke3), 8 forward passes per rollout · trajectories 1 2 3 4
| Edit | Kind | Aimed at | Where | Measured | Outcome |
|---|---|---|---|---|---|
| Rollout 1: zero 10 units with a large share of the "Wait" score, including unit 998 of layer 16 | remove units | stop the loop | MLP, layers 16, 19, 22, 23 | train +12.1, +11.3 | submitted. Reward +13.3 |
| Rollout 2: add a trigger-and-write term toward end-of-thinking. Then add a stronger one | trigger and write | close the reasoning | MLP, layer 23, then layers 21 to 23 | train −0.8, then −5.5 | The stronger one made the model stop thinking at once and give no answer. Nothing submitted. Reward 0. |
| Rollout 3: remove a "looping" direction: looping text minus clean text | remove a direction | stop the loop | attention, layer 23 | train +6.3, +6.3, +5.1 | submitted. Reward +4.7 |
| Rollout 4: a trigger-and-write term. Then zero 6 units, including unit 998 of layer 16 | trigger and write, then remove units | stop the loop | MLP, layers 16, 18, 20, 22, 23 | train −10.6, then +25.0 | Dropped the second edit because there was no confirmation run. Nothing submitted. Reward 0. |
second RL test (b32-1)
Inkling 2 Small, second RL test (b32-1), 32 forward passes per rollout. Stopped before scoring, so no reward · trajectories 1 2 3 4
| Edit | Kind | Aimed at | Where | Measured | Outcome |
|---|---|---|---|---|---|
| Rollout 1: strengthen a direction. Then a trigger-and-write term | amplify a direction; trigger and write | stop the loop | attention, layer 23 | train −2.0; then α 3: +5.1, α 5: −4.3 | still testing when stopped |
| Rollout 2: trigger and write. When units fire as they do at the end of a finished answer's reasoning, write the end-of-thinking direction | trigger and write | close the reasoning | MLP, layer 23 | train +7.1, then +32.7, +39.0 | still testing when stopped |
| Rollout 3: strengthen the end-of-thinking direction in every attention output matrix | amplify a direction | close the reasoning | attention, all 24 layers | α 1: +9.4 · α 1.2: +13.8 · α 1.5: +21.5 | still testing when stopped |
| Rollout 4: strengthen the end-of-thinking direction in every output matrix | amplify a direction | close the reasoning | attention and MLP, all 24 layers | α 0.25 to 0.4: +6.6 to +13.7 · α 0.5: +23.4 | still testing when stopped |
main RL run (hero3), step 1
Inkling 2 Small, main RL run (hero3), step 1, before any training
- 32 forward passes per rollout.
- Reward set: 256 hidden answers. Original weights: 7.4%.
- Test set: MATH-500. Original weights: 6.6%.
- Mean reward: +13.6.
| Edit | Kind | Aimed at | Where | Test | Reward |
|---|---|---|---|---|---|
| Rollout 5: Remove the "Wait" direction. Add the end-of-thinking direction twice. First, each direction loses the part it shares with the 600 most frequent tokens, so it is specific to its own token | remove a direction + amplify a direction | both | all 48 output matrices | +32.8 | +33.6 |
| Rollout 8: Remove the "Wait" direction from every output matrix. Add the end-of-thinking direction at 0.6 in the MLPs of layers 18 to 23 | remove a direction + amplify a direction | both | all output matrices; MLP, layers 18 to 23 | +29.4 | +27.3 |
| Rollout 3: Zero 16 units that write "Wait", including unit 998 of layer 16. Amplify 5 units that write against it | remove units + amplify units | stop the loop | MLP, layers 16 to 23 | +16.7 | +18.8 |
| Rollout 4: Multiply by 5 the 18 units that are active where the model is ready to end its reasoning. Zero 4 others | amplify units + remove units | close the reasoning | MLP, layers 15, 16, 18, 19, 22, 23 | +15.2 | +15.6 |
| Rollout 2: Strengthen the end-of-thinking direction at α 0.45 | amplify a direction | close the reasoning | all 48 output matrices | +11.1 | +9.4 |
| Rollout 7: Remove a "looping" direction: activations inside a loop minus outside | remove a direction | stop the loop | attention, layer 23 | +2.5 | +2.7 |
| Rollout 1: Zero 7 units that write "Wait" | remove units | stop the loop | MLP, layer 23 | +0.6 | +1.2 |
| Rollout 6: No edit: five malformed tool calls in a row | no edit | 0 |
main RL run (hero3), step 2
Inkling 2 Small, main RL run (hero3), step 2, after one training update · analysis
- All 8 rollouts strengthened the end-of-thinking direction in all 48 output matrices, at α 1.0 to 2.0.
- 3 rollouts also removed the "Wait" direction. In 2 of them (rollouts 2 and 3), "Wait" was one of the tokens projected out of that direction. So the removal did nothing.
- Every rollout read step 1's notes, which held this recipe. So the gain mixes training and copying. A run without notes (nolib1) measures training alone.
- Mean reward: +24.3. Test scoring is in progress.
| Rollout rewards, best to worst |
|---|
| 32.0, 27.7, 27.3, 25.8, 24.2, 23.1, 20.7, 13.3 |
RL run without notes (nolib1), steps 2 and 3
Inkling 2 Small, RL run nolib1. It starts from the same checkpoint as hero3 step 2, but the teacher gets no shared notes and no web search · analysis
Step 2. Mean reward +13.5.
| Edit | Kind | Aimed at | Where | Test | Reward |
|---|---|---|---|---|---|
| Rollout 1: Remove the "Wait" direction from every output matrix. Add "</think>" at 1.0 in layer 23's attention output | remove a direction + amplify a direction | both | all 48 output matrices | +19.0 | +21.1 |
| Rollout 2: Flip 6 units that write "Wait", including unit 998 of layer 16 | flip units | stop the loop | MLP, layers 16, 20, 22, 23 | +12.3 | +11.3 |
| Rollout 3: Remove the "Wait" direction from every output matrix | remove a direction | stop the loop | all 48 output matrices | +18.7 | +16.8 |
| Rollout 4: Raise "</think>" at strength 6 in one matrix | amplify a direction | close the reasoning | attention, layer 23 | +21.6 | +21.1 |
| Rollout 5: Double one unit | scale a unit | not specified | MLP, layer 19 | +0.4 | +0.4 |
| Rollout 6: Zero 7 units that write "Wait", flip unit 998 of layer 16 at 1.5, and remove a looping direction | remove units + flip a unit + remove a direction | stop the loop | 18 matrices, layers 15 to 23 | +10.0 | +13.3 |
| Rollout 7: Raise "</think>" at strength 6 in one matrix | amplify a direction | close the reasoning | attention, layer 23 | +21.4 | +21.1 |
| Rollout 8: Remove a looping direction found by difference of means | remove a direction | stop the loop | all 48 output matrices | +4.4 | +3.1 |
- Without notes, the teachers tried many kinds of edit, as in step 1. With notes (hero3 step 2), all of them copied one recipe.
- Step 3. Mean reward +18.7. Its edits are not catalogued yet.
| Step 3 rollout rewards, best to worst |
|---|
| 32.8, 27.7, 25.0, 18.4, 16.8, 14.4, 14.4, 0.0 |
For reference: the published edit
The published edit from arXiv 2611.09382, found by GPT-6 Astra, applied by us
| Edit | Kind | Aimed at | Where | Measured | Outcome |
|---|---|---|---|---|---|
| Subtract from unit 507 the direction "doubt words minus progress words" | redirect a unit | stop the loop | MLP, layer 23 | Test 7.1% → 28.9% |
What the edits show
- The teachers found only two error modes. The model re-checks its work in a loop ("Wait"), and it never ends its reasoning ("</think>"). Both end the same way: the answer is cut off at the token limit. Only two later rollouts aimed at something else: query and key rows in layer 0 that were all zero. That did not help.
- Shared notes make the teachers copy. With notes, every hero3 step-2 rollout used the same recipe. Without notes (nolib1), the teachers tried many kinds of edit.
- The best edit does both. Opus tested each half alone and both together. Together was about twice as good as either half.
- One unit keeps turning up. Unit 998 in the layer-16 MLP. Opus found it, and so did several RL rollouts. Two checkpoints in arXiv 2611.09382 also edit it.
- Pushing too hard is the common failure. If the end-of-thinking score rises too much, the model stops reasoning. Opus, Inkling and one RL rollout all hit this. The teachers that tested several strengths backed off.
- The largest gains edit many layers. In hero3 step 1, the two best rollouts removed the "Wait" direction from every output matrix and added the end-of-thinking direction. Edits to a few units earned less.
- No edit tried anything else, such as arithmetic mistakes or wrong answers in finished solutions. A likely reason: the teacher starts with a summary of the student's answers, and that summary points straight at "Wait".
References
- Takehi, Teoh, Zhang and Khattab. Sparks of Weight-Space Programming. blog post, 2026.
- Hu et al.. Exploring Weight-Space Programming: Capabilities, Generalization, and Limitations. arXiv 2611.09382, 2026.
- Elhage et al.. A Mathematical Framework for Transformer Circuits. Transformer Circuits Thread, 2021.
- nostalgebraist. interpreting GPT: the logit lens. LessWrong, 2020.
- Meng et al.. Locating and Editing Factual Associations in GPT. arXiv 2202.05262, 2022.
- Arditi et al.. Refusal in Language Models Is Mediated by a Single Direction. arXiv 2406.11717, 2024.
- Turner et al.. Steering Language Models With Activation Engineering. arXiv 2308.10248, 2023.