Tinkerings
Photo of the author

Notes to build mental models for how things work.

Weight-space programming

interpretability
rl

How Claude Opus 5.5 and Inkling 2 Small diagnose and edit Qwen3.5-0.8B’s weights to stop it looping on math: direct logit attribution, difference of means, token directions, and every edit tried so far.

Nov 14, 2026

What kappa does

evals

Agreement minus luck, divided by the room above luck. Why a judge is gated on kappa but not picked on it, from the Legal Research Bench calibration study.

Nov 7, 2026

Score centering

rl

Why a mismatched inference engine pushes the trainer toward itself, why ratios cannot fix it cleanly, and the one-line change that does: subtract the engine’s distribution instead of the trainer’s.

Nov 6, 2026

Bias and variance of the policy gradient

rl

Every RL-for-LLM weighting rule, from REINFORCE to DPPO, measured on a toy where the true gradient is exact: bias, variance, the cap, and why a trust region is needed.

Nov 6, 2026

Triton Kernels

gpus
Modern GPUs have massive compute but limited memory bandwidth. The key metric is arithmetic intensity (AI):
Sep 8, 2025

Context Parallelism

distributed-training
Context Parallelism (CP) is a distributed training technique that enables training of LLMs with extremely long sequences (up to 1M tokens) by sharding the sequence dimension…
Sep 3, 2025

Expert Parallelism

distributed-training
Let’s distill how to run a Mixture-of-Experts (MoE) model with expert parallelism with an example.
Sep 1, 2025
No matching items

Tinkerings