Tinkerings

Weight-space programming

Karthik Chandrasekar, Aryan Keluskar, Charles Weill, Deniz Zorlu

State on 14 November 2026, 19:10

Can a model improve another model by editing its weights directly, with no gradients [1] [2]? We set this up as a teacher and a student.

The student fails at math mostly because it does not stop reasoning. It writes "Wait..." and rechecks its work until it runs out of tokens. This happens in about 95% of answers. Every edit below tries to fix this, in one of two ways, or both:

A score is the share of correct final answers. Edits are scored on three sets of problems:

The localization and edit patterns the teacher models deploy resemble mechanistic interpretability methods, which I illustrate below. We now list these methods in the teacher's prompt. But the teachers used them before we did.

The page has three parts:

How to diagnose

A diagnosis finds the units that make the model write the wrong next token. Here that token is "Wait". The teachers used three methods from the interpretability literature:

  1. Direct logit attribution: which unit writes "Wait"?
  2. Difference of means: which unit fires differently just before "Wait"?
  3. Token directions: no measurement. Use the token's own direction.

Method 1 is built up below on a one-layer network.

1. Direct logit attribution [3] [4]

A small network

Start with a toy network. It has the same parts as the student, only smaller.

ToyQwen3.5-0.8B
Layers124
Residual stream31024
MLP units per layer43584
The forward pass of one layer: the residual stream h0 passes through; the MLP computes x from h0, then m = x @ W, which is added back; h @ U gives the logits … so the answer is 12.x is read here, where the next token is "Wait"h0 [3][0.5, 0, 0]residual: passes through unchanged+h [3][2, 2, 2.25]@ Ulogits [3][3, 2, −0.875]Wait, </think>, Sogate, upx [4][1, 2, 1.5, 0.5]@ Wm [3][1.5, 2, 2.25]MLP: up to 4 units, then back down to 3 through Wfigure 2 zooms into this box
1. The forward pass at one position.

Because h is the residual plus what the MLP added, the "Wait" logit splits the same way, with u = U[:, Wait]:

$$\text{logit}_{\text{Wait}} = h \cdot u = (h_0 + xW)\cdot u = \underbrace{h_0 \cdot u}_{\text{residual: }0.5} \;+\; \underbrace{\sum_j x_j \,(W_j \cdot u)}_{\text{MLP, one term per row of }W:\ 2.5} = 3$$

Figure 2 opens up the MLP part. It has one term per unit.

Where a logit comes from

x [4]
*
W [4 × 3]
*
U [3 × 3]
=
contribution to "Wait" [4 × 3]
sum
row
row contribution [4]
Hover a cell of W for one weight, or a unit on the right for its row.
2. Each weight adds x[j] × W[j][d] × U[d][Wait]. One row sums to one unit's contribution. All rows sum to the MLP part, 2.5.

Across positions and layers

Figure 2 shows one position: one token whose next token is "Wait". A looping answer writes "Wait" many times. Let P be all those positions. Unit j is one MLP neuron: entry j of x, row j of W. For each layer L and unit j, add the unit's contribution over P:

$$C_L[j] = \sum_{p \in P} x_L[p, j]\,\bigl(W_L[j] \cdot u\bigr) \,/\, \text{rms}(h[p])$$

Each teacher ran this search in a different way:

The five units with the largest C:

RankOpus: layer, unitCInkling: layer, unitC
116, 9987.5122, 1155.55
223, 363.0922, 350.07
322, 1941.8223, 4626.83
423, 1131.1423, 719.95
522, 111.0523, 24019.85

In our runs

Opus then edited unit 998 of layer 16. Both edits helped:

EditTrain accuracy
None7.0%
Zero the unit14.5%
Flip the unit17.6%

2. Difference of means [6] [7]

Method 1 asks which unit writes "Wait". Method 2 asks which unit fires differently just before "Wait". It needs only the activations x. It compares two sets of positions:

Example: unit 2 fires 3 and 2 at the positions in P, and 0 and 1 at those in Q. Its means are 2.5 and 0.5, so the difference is 2. It fires mostly before "Wait".

$$d[j] = \text{mean}_{p \in P}\, x[p, j] \;-\; \text{mean}_{q \in Q}\, x[q, j]$$

The units with the largest d are the candidates. On the residual stream h, the same difference gives a direction instead of a unit. That direction can be removed from the output matrices [6].

3. Token directions

This method skips the measurement. Each token has a column in U. That column is a direction in the residual stream. We can remove it from the output matrices of every layer, or add it. Removing it from every layer is called directional ablation [6].

Inkling used this method in its first search. It saw that layers 15 to 21 push "</think>" down. So it added the "</think>" direction to the MLPs of layers 20 to 23. Test accuracy rose from 6.6% to 20.2%.

How to update

A diagnosis gives a unit or a direction. The teacher then changes the weights there. It only edits output matrices: the matrix W from How to diagnose, which writes an MLP's result back into the residual stream, and the same matrix in attention layers. No other weights are changed.

There are two kinds of edit:

Opus's and Inkling's largest gains came from rank-one terms.

The figures in this section draw W as it is stored in the model file, [1024 × 3584]. Unit j is column j here. In How to diagnose we used the transpose, so unit j was row j. It is the same matrix.

Where the edited matrix sits: the residual stream h feeds the MLP units x, and W maps x back into the residual streamresidual stream h [1024]h + mgate_proj,up_projMLP unitsx [3584]1024 residual dimensionsW [1024 × 3584], edited3584 MLP unitsm =W @ x
One MLP at one position. W is down_proj. Unit j adds column j of W to the residual stream, scaled by its activation x[j]. The attention output matrix does the same job for attention. Only these two matrices are edited.

The figure below shows a small W before and after an edit. Rows are residual dimensions. Columns are MLP units. positive weight, negative weight, zero. A stronger colour means a larger weight. marks the cells the edit changes. The rest are faded.

Kinds 1 to 4 change one column. The other columns stay the same. Kinds 5 to 7 add a rank-one term a·bᵀ, so they can change every cell. Cell (i, j) changes by a[i] × b[j].

Flip a unit: W[:, j] = -W[:, j]beforeafterjj

Column 3 changes sign. 6 of 60 cells change.

Hover over a cell, tap it, or focus the figure and use the arrow keys to read its value.

2. Flip a unit
W[:, j] = -W[:, j]
Unit j writes the negative of its old vector. Every token it raised, it now lowers.
Example: Opus flipped unit 998 in layer 16. It kept this as one half of its final edit.

8. Change what a unit reads

Kinds 1 to 7 change output matrices: what a unit writes. Two later rollouts changed an input matrix instead: what layer 0's linear attention reads. They found query and key rows that were all zero. They filled each one with a copy of the same row from the next head.

Fix what a unit reads: zero rows of layer 0 query/key input matrix filled with copies from the next head dim 0dim 1dim 2dim 3dim 0dim 1dim 2dim 3head hhead h+1copy the samedimension fromthe next headlayer 0 query/key input matrix: one row per dimensionempty rows: all zero, so these dimensions read nothing
8. Rollouts rebase1 s2-r5 and rebase2 s3-r2. These are the only edits that aimed at neither "Wait" nor "</think>". Neither helped.

Variants seen in later runs

Every edit tried so far

The tables list every edit, grouped by teacher and run. Most numbers are changes in accuracy, in points: edited minus original, on the same problems. Numbers with % are plain accuracies. α is the strength of an edit.

Claude Opus 5.5 1 search

Claude Opus 5.5 · trajectory

EditKindAimed atWhereStrengthMeasuredOutcome
Strengthen the end-of-thinking directionamplify a directionclose the reasoningattention and MLP, layers 14 to 22α 0.5train 7.4% → 7.0%no effect, dropped
Zero unit 998remove a unitstop the loopMLP, layer 16train 7.0% → 14.5%replaced by the flip
Flip unit 998flip a unitstop the loopMLP, layer 16train 17.6%kept as one half
Flip + directionflip + amplifybothlayers 14 to 22α 2.0train 37.1%rejected: a quarter of finished answers skipped the reasoning
Flip + direction on nine chosen matricesflip + amplifybothattention, layers 17 to 21; MLP, layers 15, 16, 19, 22α 2.5validation +26 to +29 on three setssubmitted. Test 7.6% → 34.8%

Each half alone, on train problems:

Inkling 2 Small 7 runs
first search

Inkling 2 Small, first search · trajectory

Strengthen the end-of-thinking direction in the MLPs of layers 20 to 23. Aimed at closing the reasoning. One row per strength:

αTrainOutcome
0.5+4.5
0.7+8.9
0.85+21.4submitted. Test 6.6% → 20.2%
1.0+30.4skipped the reasoning in about 30% of answers

On validation the two strongest settings gave +22.7 and +20.3.

second search

Inkling 2 Small, second search · trajectory

EditKindAimed atWhereMeasuredOutcome
Zero 11 units that write "Wait"remove unitsstop the loopMLP, layers 5, 15, 16, 18, 19, 20, 22, 23validation: GSM8K +7 to +10, MATH 0 to +1.5kept as one half
Also double 8 units that fire where finished answers close their reasoningamplify unitsclose the reasoningMLP, layers 16, 18, 19, 22, 23validation MATH +6.7submitted. Test 6.6% → 11.8%. Tripling was worse.
first RL test (smoke3)

Inkling 2 Small, first RL test (smoke3), 8 forward passes per rollout · trajectories 1 2 3 4

EditKindAimed atWhereMeasuredOutcome
Rollout 1: zero 10 units with a large share of the "Wait" score, including unit 998 of layer 16remove unitsstop the loopMLP, layers 16, 19, 22, 23train +12.1, +11.3submitted. Reward +13.3
Rollout 2: add a trigger-and-write term toward end-of-thinking. Then add a stronger onetrigger and writeclose the reasoningMLP, layer 23, then layers 21 to 23train −0.8, then −5.5The stronger one made the model stop thinking at once and give no answer. Nothing submitted. Reward 0.
Rollout 3: remove a "looping" direction: looping text minus clean textremove a directionstop the loopattention, layer 23train +6.3, +6.3, +5.1submitted. Reward +4.7
Rollout 4: a trigger-and-write term. Then zero 6 units, including unit 998 of layer 16trigger and write, then remove unitsstop the loopMLP, layers 16, 18, 20, 22, 23train −10.6, then +25.0Dropped the second edit because there was no confirmation run. Nothing submitted. Reward 0.
second RL test (b32-1)

Inkling 2 Small, second RL test (b32-1), 32 forward passes per rollout. Stopped before scoring, so no reward · trajectories 1 2 3 4

EditKindAimed atWhereMeasuredOutcome
Rollout 1: strengthen a direction. Then a trigger-and-write termamplify a direction; trigger and writestop the loopattention, layer 23train −2.0; then α 3: +5.1, α 5: −4.3still testing when stopped
Rollout 2: trigger and write. When units fire as they do at the end of a finished answer's reasoning, write the end-of-thinking directiontrigger and writeclose the reasoningMLP, layer 23train +7.1, then +32.7, +39.0still testing when stopped
Rollout 3: strengthen the end-of-thinking direction in every attention output matrixamplify a directionclose the reasoningattention, all 24 layersα 1: +9.4 · α 1.2: +13.8 · α 1.5: +21.5still testing when stopped
Rollout 4: strengthen the end-of-thinking direction in every output matrixamplify a directionclose the reasoningattention and MLP, all 24 layersα 0.25 to 0.4: +6.6 to +13.7 · α 0.5: +23.4still testing when stopped
main RL run (hero3), step 1

Inkling 2 Small, main RL run (hero3), step 1, before any training

  • 32 forward passes per rollout.
  • Reward set: 256 hidden answers. Original weights: 7.4%.
  • Test set: MATH-500. Original weights: 6.6%.
  • Mean reward: +13.6.
EditKindAimed atWhereTestReward
Rollout 5: Remove the "Wait" direction. Add the end-of-thinking direction twice. First, each direction loses the part it shares with the 600 most frequent tokens, so it is specific to its own tokenremove a direction + amplify a directionbothall 48 output matrices+32.8+33.6
Rollout 8: Remove the "Wait" direction from every output matrix. Add the end-of-thinking direction at 0.6 in the MLPs of layers 18 to 23remove a direction + amplify a directionbothall output matrices; MLP, layers 18 to 23+29.4+27.3
Rollout 3: Zero 16 units that write "Wait", including unit 998 of layer 16. Amplify 5 units that write against itremove units + amplify unitsstop the loopMLP, layers 16 to 23+16.7+18.8
Rollout 4: Multiply by 5 the 18 units that are active where the model is ready to end its reasoning. Zero 4 othersamplify units + remove unitsclose the reasoningMLP, layers 15, 16, 18, 19, 22, 23+15.2+15.6
Rollout 2: Strengthen the end-of-thinking direction at α 0.45amplify a directionclose the reasoningall 48 output matrices+11.1+9.4
Rollout 7: Remove a "looping" direction: activations inside a loop minus outsideremove a directionstop the loopattention, layer 23+2.5+2.7
Rollout 1: Zero 7 units that write "Wait"remove unitsstop the loopMLP, layer 23+0.6+1.2
Rollout 6: No edit: five malformed tool calls in a rowno edit0
main RL run (hero3), step 2

Inkling 2 Small, main RL run (hero3), step 2, after one training update · analysis

  • All 8 rollouts strengthened the end-of-thinking direction in all 48 output matrices, at α 1.0 to 2.0.
  • 3 rollouts also removed the "Wait" direction. In 2 of them (rollouts 2 and 3), "Wait" was one of the tokens projected out of that direction. So the removal did nothing.
  • Every rollout read step 1's notes, which held this recipe. So the gain mixes training and copying. A run without notes (nolib1) measures training alone.
  • Mean reward: +24.3. Test scoring is in progress.
Rollout rewards, best to worst
32.0, 27.7, 27.3, 25.8, 24.2, 23.1, 20.7, 13.3
RL run without notes (nolib1), steps 2 and 3

Inkling 2 Small, RL run nolib1. It starts from the same checkpoint as hero3 step 2, but the teacher gets no shared notes and no web search · analysis

Step 2. Mean reward +13.5.

EditKindAimed atWhereTestReward
Rollout 1: Remove the "Wait" direction from every output matrix. Add "</think>" at 1.0 in layer 23's attention outputremove a direction + amplify a directionbothall 48 output matrices+19.0+21.1
Rollout 2: Flip 6 units that write "Wait", including unit 998 of layer 16flip unitsstop the loopMLP, layers 16, 20, 22, 23+12.3+11.3
Rollout 3: Remove the "Wait" direction from every output matrixremove a directionstop the loopall 48 output matrices+18.7+16.8
Rollout 4: Raise "</think>" at strength 6 in one matrixamplify a directionclose the reasoningattention, layer 23+21.6+21.1
Rollout 5: Double one unitscale a unitnot specifiedMLP, layer 19+0.4+0.4
Rollout 6: Zero 7 units that write "Wait", flip unit 998 of layer 16 at 1.5, and remove a looping directionremove units + flip a unit + remove a directionstop the loop18 matrices, layers 15 to 23+10.0+13.3
Rollout 7: Raise "</think>" at strength 6 in one matrixamplify a directionclose the reasoningattention, layer 23+21.4+21.1
Rollout 8: Remove a looping direction found by difference of meansremove a directionstop the loopall 48 output matrices+4.4+3.1
  • Without notes, the teachers tried many kinds of edit, as in step 1. With notes (hero3 step 2), all of them copied one recipe.
  • Step 3. Mean reward +18.7. Its edits are not catalogued yet.
Step 3 rollout rewards, best to worst
32.8, 27.7, 25.0, 18.4, 16.8, 14.4, 14.4, 0.0
For reference: the published edit

The published edit from arXiv 2611.09382, found by GPT-6 Astra, applied by us

EditKindAimed atWhereMeasuredOutcome
Subtract from unit 507 the direction "doubt words minus progress words"redirect a unitstop the loopMLP, layer 23Test 7.1% → 28.9%

What the edits show

References

  1. Takehi, Teoh, Zhang and Khattab. Sparks of Weight-Space Programming. blog post, 2026.
  2. Hu et al.. Exploring Weight-Space Programming: Capabilities, Generalization, and Limitations. arXiv 2611.09382, 2026.
  3. Elhage et al.. A Mathematical Framework for Transformer Circuits. Transformer Circuits Thread, 2021.
  4. nostalgebraist. interpreting GPT: the logit lens. LessWrong, 2020.
  5. Meng et al.. Locating and Editing Factual Associations in GPT. arXiv 2202.05262, 2022.
  6. Arditi et al.. Refusal in Language Models Is Mediated by a Single Direction. arXiv 2406.11717, 2024.
  7. Turner et al.. Steering Language Models With Activation Engineering. arXiv 2308.10248, 2023.