Tonon Journey https://lucastononro.github.io/ Lucas Tonon builds tools for coding agents and writes down what happened. Skills, experiments, and the occasional evening lost to a shell script. en tonon@vetto.ai (Lucas Tonon) Wed, 28 Oct 2026 13:23:59 GMT how you could have invented muon from memory https://lucastononro.github.io/how-you-could-have-invented-muon-from-memory/ https://lucastononro.github.io/how-you-could-have-invented-muon-from-memory/ Sun, 25 Oct 2026 12:00:00 GMT The matrix trick hiding in plain sight that made AdamW look less inevitable. research AdamW is a comfortable default. Then someone remembers that a matrix is a transformation, and suddenly there is a suspiciously simple optimizer to explain.

Let's try to invent it. We have a model, some data, and enough calculus and linear algebra to be dangerous. How should we change the weights?

We'll start with the obvious answer, gradient descent. Then we'll notice that a matrix does more than store a list of numbers. Taking that seriously will lead us to Muon's update. The aim is to make each choice follow from the previous one, so the formula becomes something we can reconstruct.

The optional explanations are folded away. Open whichever bits you need; there is no entrance exam.

§A function, some data, and a loss

A model is a mathematical function fθ:X→Yf_\theta:\mathcal X\to\mathcal Y, with adjustable parameters θ\theta. We choose those parameters to approximate a relationship we do not know. In regression, call that relationship f∗f_*; after fitting, our estimate is f^=fθ^≈f∗\hat f=f_{\hat\theta}\approx f_*. The hat means "estimated from data".

This covers a line fθ(x)=ax+bf_\theta(x)=ax+b, a vision network, or an LLM. For an LLM, the output is a probability distribution over possible next tokens. We are estimating a conditional distribution, rather than assuming every input has one inevitable answer.

A loss turns "how well are we doing?" into a number to minimize. It doesn't have to be a distance in the formal mathematical sense. Here, it measures what we intuitively call distance or difference: how far the model's predictions are from what we want.

For regression, our training data consists of pairs (xi,yi)(x_i,y_i). The input is xix_i, the desired output is yiy_i, and the model predicts fθ(xi)f_\theta(x_i). For a language model, xtx_t is the observed token at position tt, and x<tx_{<t} is the preceding context. Two common losses are

Lreg(θ)=1N∑i=1N(fθ(xi)−yi)2,LLM(θ)=−1T∑t=1Tlog⁡pθ(xt∣x<t).\begin{aligned} L_{\mathrm{reg}}(\theta) &=\frac1N\sum_{i=1}^N\bigl(f_\theta(x_i)-y_i\bigr)^2,\\[10pt] L_{\mathrm{LM}}(\theta) &=-\frac1T\sum_{t=1}^T\log p_\theta(x_t\mid x_{<t}). \end{aligned}

The first is mean squared error over NN input/output pairs. The second is token cross-entropy over TT predicted tokens: it penalizes assigning low probability to the token that actually occurred. A lower loss means improvement on this objective; choosing a useful objective is our responsibility.

Why does low token probability give a large loss?

For a target distribution yy and predicted probabilities pp, cross-entropy is −∑jyjlog⁡pj-\sum_j y_j\log p_j. A one-hot target puts all its mass on the observed token, so only −log⁡p(observed token)-\log p(\text{observed token}) remains.

Giving the correct token probability 0.80.8 costs about 0.2230.223 nats. Giving it 0.20.2 costs about 1.6091.609. The negative logarithm makes confident mistakes expensive. Averaging this quantity over tokens gives the loss above.

Training moves θ\theta through a space with one dimension per parameter. We want a useful direction and a sensible distance. We get local information about the loss, not a map showing the best route to the bottom.

§First, take a step downhill

So far, θ\theta has meant all the model's adjustable parameters. To work out one update, focus on the weight matrix of a single layer, W∈Rm×nW\in\mathbb R^{m\times n}, and hold the other parameters fixed. We will write the loss as L(W)L(W) to show which parameters we are changing. It is the same training objective, viewed as a smaller optimization problem.

Once we have a rule for this matrix, we can apply it to other weight matrices too. Biases and other parameter shapes may use a different update rule. For now, assume L(W)L(W) is differentiable: small weight changes have a first-order effect that we can predict from the local slopes.

An optimizer adds a step DD, with the same shape as WW:

Wnew=W+D.W_{\mathrm{new}}=W+D.

Each DijD_{ij} says how much to change the corresponding weight WijW_{ij}. We keep uppercase WW and DD for the whole matrices throughout this derivation. In the small examples below, lowercase w1,w2w_1,w_2 denote individual weights.

The gradient matrix GG collects the partial derivatives, Gij=∂L/∂WijG_{ij}=\partial L/\partial W_{ij}. Each entry measures the local change in loss per tiny change in that weight, with the others held fixed. A positive slope says increasing the weight raises the loss locally; a negative slope says the opposite.

The gradient gives us slopes. To turn them into an actual step, choose a positive scale α\alpha, called the learning rate, and define

D=−αG.D=-\alpha G.

The minus sign reverses the slopes; α\alpha controls how far we move along them. This is gradient descent. Whether it lowers the actual loss still depends on how large a step we take.

Derivatives and partial derivatives, without the ceremony

For f(w)=w2f(w)=w^2, changing w=1w=1 by hh changes the output by 2h+h22h+h^2. Divide by hh to get the average slope 2+h2+h. As hh approaches zero, this approaches the derivative f′(1)=2f'(1)=2.

A shrinking secant interval reveals the derivativeSecant slope 2.020 for h=0.020 approaches the tangent slope 2.A derivative is a limiting slopeShrink the interval; keep the starting point fixed.11 + hw
f(1+h)−f(1)h=2+h\frac{f(1+h)-f(1)}h=2+h
h = 0.020secant slope = 2.020
f′(1)=lim⁡h→0(2+h)=2f'(1)=\lim_{h\to0}(2+h)=2
Blue: secant · dashed: tangent · curve: f(w) = w²
The moving point approaches w = 1. The blue secant measures the slope over a finite interval; the dashed tangent shows its limit. A partial derivative does the same experiment while holding the other weights fixed.
Shrink the interval and watch the slope settle.

A derivative measures slope. A second derivative measures how that slope changes, which is curvature. If every partial derivative is zero, we have a stationary point. It could be a minimum, a maximum, or a saddle. Zero does not come with a "best possible answer" certificate.

Work out a two-weight loss and its gradient

Let two predictions be y^1=w1\hat y_1=w_1 and y^2=2w2\hat y_2=2w_2, both with target zero. Half the sum of squared errors gives

L(w1,w2)=12[(y^1−0)2+(y^2−0)2]=12[w12+(2w2)2]=12(w12+4w22).\begin{aligned} L(w_1,w_2) &=\tfrac12\bigl[(\hat y_1-0)^2+(\hat y_2-0)^2\bigr]\\[10pt] &=\tfrac12\bigl[w_1^2+(2w_2)^2\bigr]\\[10pt] &=\tfrac12(w_1^2+4w_2^2). \end{aligned}

The 1/21/2 tidies the derivatives. The 44 comes from squaring the second prediction's factor of 22. In this two-weight example, collect the derivatives in a vector gg:

g=(∂L∂w1,∂L∂w2)=(12⋅2w1,12⋅8w2)=(w1,4w2).\begin{aligned} g&=\left(\frac{\partial L}{\partial w_1},\frac{\partial L}{\partial w_2}\right)\\[12pt] &=\left(\tfrac12\cdot2w_1,\tfrac12\cdot8w_2\right)\\[10pt] &=(w_1,4w_2). \end{aligned}

A real network chains layers together. An early weight affects later activations, then the prediction, then the loss. Backpropagation applies the chain rule through those layers to compute the same kind of partial derivatives.

Choosing DD gets harder when the loss has curved valleys, flat regions, saddles, or several minima. A downhill step can settle in a shallow basin even when a deeper one exists elsewhere.

The surface below has two minima. The first run settles in the higher one. Try the other basin to see a different result. You can also drag the graph to look around and zoom with the buttons.

Rotate a loss surface with two basins and follow computed gradient descentStep 15. Weights −0.816, 0.000. Loss 1.1010. A computed gradient path descends a three-dimensional surface with two basins. The floor trail shows parameter positions.w₁w₂localglobalStep 15 / 15 · loss 1.1010w = (−0.816, 0.000)D = −0.09 G
Height is loss; the floor axes are two weights. Orange follows computed gradient steps. The left basin contains a local minimum with positive loss; the right basin reaches the global minimum, zero. This small example shows why local slope information cannot promise the best solution.
Drag to rotate. Arrow keys also rotate; + and − zoom.
The exact loss behind the picture

The surface uses

Ldraw(x,y)=(x2−1)2+0.3(x−1)2+8y2.\begin{aligned} L_{\mathrm{draw}}(x,y)&=(x^2-1)^2\\[6pt] &\quad+0.3(x-1)^2+8y^2. \end{aligned}

Its gradient is (4x(x2−1)+0.6(x−1), 16y)(4x(x^2-1)+0.6(x-1),\,16y). The global minimum is (1,0)(1,0): all three nonnegative terms vanish there. There is also a strict local minimum at x=(−1−0.4)/2≈−0.816x=(-1-\sqrt{0.4})/2\approx-0.816, y=0y=0, separated from the deeper basin by a saddle.

The animation recomputes the gradient at each step, using α=0.09\alpha=0.09. The same function drives the surface and the path. Moving between iterates is only a visual aid; the optimization steps are discrete. This is a two-weight illustration, not a claim that shallow local minima are the main difficulty in every large network.

§Turn many slopes into one score

We have a candidate step DD, but its entries only tell us how much to change the weights. They do not tell us whether those changes help. For that, we need to know how the loss responds. That is the information in GG.

Hold every weight except WijW_{ij} fixed. Recall that Gij=∂L/∂WijG_{ij}=\partial L/\partial W_{ij} is the local slope: loss change per unit change in this weight, in the limit of a tiny move. Near our current weights, we approximate the curve by a straight line. On a straight line, change in height is slope times horizontal movement. Here that becomes

predicted loss change≈GijDij.\text{predicted loss change}\approx G_{ij}D_{ij}.

For example, if Gij=3G_{ij}=3 and we increase that weight by Dij=0.01D_{ij}=0.01, the predicted loss change is 3×0.01=0.033\times0.01=0.03. Moving it by −0.01-0.01 instead predicts −0.03-0.03, a decrease. A negative gradient entry reverses which direction helps. The same weight change can help or hurt depending on its slope.

When several weights change together, differentiability tells us that their first-order effects add. This does not assume the weights act independently: the slopes depend on the whole current matrix, and the linear approximation leaves a remainder. We will make that approximation explicit in the next section. For now, our combined score is

∑i,jGijDij=⟨G,D⟩F.\sum_{i,j}G_{ij}D_{ij}=\langle G,D\rangle_F.

This is the ordinary dot product of the entries, keeping them arranged as matrices. Its name is the Frobenius inner product. For any two matrices AA and BB of the same shape, we write

⟨A,B⟩F=∑i,jAijBij=tr⁡(A⊤B).\langle A,B\rangle_F=\sum_{i,j}A_{ij}B_{ij} =\operatorname{tr}(A^\top B).

Here A⊤A^\top is the transpose, which swaps rows and columns. tr⁡\operatorname{tr} denotes the trace, the sum of a square matrix's diagonal entries. The whole expression returns one number.

What does the trace notation mean?

For a square matrix CC, the trace is the sum of its diagonal entries:

tr⁡(C)=∑jCjj.\operatorname{tr}(C)=\sum_j C_{jj}.

For example,

C=[302−2],tr⁡(C)=3+(−2)=1.\begin{aligned} C&=\begin{bmatrix}3&0\\2&-2\end{bmatrix},\\[10pt] \operatorname{tr}(C)&=3+(-2)=1. \end{aligned}

The entry 22 below the diagonal contributes nothing. Trace takes a square matrix and returns one scalar.

Why does it appear in our score? The jjth diagonal entry of A⊤BA^\top B is the dot product of column jj of AA with column jj of BB:

(A⊤B)jj=∑iAijBij,tr⁡(A⊤B)=∑j∑iAijBij.\begin{aligned} (A^\top B)_{jj}&=\sum_i A_{ij}B_{ij},\\[10pt] \operatorname{tr}(A^\top B)&=\sum_j\sum_i A_{ij}B_{ij}. \end{aligned}

Adding those diagonal entries counts every matching pair once. That gives the same inner product we started with.

We will also need a way to measure the size of a step. Using the same entries, define the Frobenius norm:

∥A∥F=∑i,jAij2.\|A\|_F=\sqrt{\sum_{i,j}A_{ij}^2}.

It is the Euclidean length we would get by listing all the entries in a vector. This length and our score are related: the difference between the squared lengths of A+BA+B and A−BA-B isolates their inner product.

⟨A,B⟩F=∥A+B∥F2−∥A−B∥F24.\boxed{\langle A,B\rangle_F =\frac{\|A+B\|_F^2-\|A-B\|_F^2}{4}.}

This is the real polarization identity.

Where does the polarization formula come from?

Start with the norm's definition. Squaring it removes the square root, and each entry of A+BA+B is Aij+BijA_{ij}+B_{ij}. Apply the ordinary identity (a+b)2=a2+2ab+b2(a+b)^2=a^2+2ab+b^2 to each entry:

∥A+B∥F2=∑i,j(Aij+Bij)2=∑i,jAij2+∑i,jBij2+2∑i,jAijBij=∥A∥F2+∥B∥F2+2⟨A,B⟩F.\begin{aligned} \|A+B\|_F^2&=\sum_{i,j}(A_{ij}+B_{ij})^2\\[10pt] &=\sum_{i,j}A_{ij}^2+\sum_{i,j}B_{ij}^2\\[6pt] &\quad+2\sum_{i,j}A_{ij}B_{ij}\\[10pt] &=\|A\|_F^2+\|B\|_F^2\\[6pt] &\quad+2\langle A,B\rangle_F. \end{aligned}

The last line simply recognizes the two squared norms and the inner product from their definitions. Replacing BB by −B-B changes the sign of the cross term:

∥A−B∥F2=∥A∥F2+∥B∥F2−2⟨A,B⟩F.\begin{aligned} \|A-B\|_F^2&=\|A\|_F^2+\|B\|_F^2\\[6pt] &\quad-2\langle A,B\rangle_F. \end{aligned}

Subtract the second expansion from the first. The squared norms cancel, and the cross terms leave 4⟨A,B⟩F4\langle A,B\rangle_F. Dividing by four gives the formula above.

§Predict whether a step will lower the loss

Our sum of slopes times changes looks like a useful score. Now we need to connect it to the quantity we actually care about, the loss after taking the step.

A differentiable function is locally approximated by its value plus a linear change. Applied to the weights, that linear change is exactly ⟨G,D⟩F\langle G,D\rangle_F. This is the first-order Taylor expansion:

L(W+D)≈L(W)+⟨G,D⟩F.\boxed{L(W+D)\approx L(W)+\langle G,D\rangle_F.}

The first term is our current loss. The second is the predicted change. A more negative inner product means a larger predicted decrease. The approximation is local, so the step size still matters.

Why does the local slope predict a change in loss?

Taylor approximates a function near a point using derivatives there. For a sufficiently smooth scalar function,

f(x+h)=f(x)+f′(x)h+f′′(x)2!h2+⋯+f(N)(x)N!hN+RN(h).\begin{aligned} f(x+h)&=f(x)+f'(x)h\\[8pt] &\quad+\frac{f''(x)}{2!}h^2+\cdots\\[8pt] &\quad+\frac{f^{(N)}(x)}{N!}h^N+R_N(h). \end{aligned}

The remainder RNR_N records the error. If the next derivative is continuous and bounded in magnitude by BB along the interval, ∣RN(h)∣≤B∣h∣N+1/(N+1)!|R_N(h)|\leq B|h|^{N+1}/(N+1)!. Keeping only the value and slope gives the first-order approximation.

For f(x)=x2f(x)=x^2 at x=1x=1, the tangent predicts 1+2h1+2h and the exact value is 1+2h+h21+2h+h^2. Watch the terms appear with their geometric meaning.

A parabola, its tangent line and the Taylor approximation errorAt x=1.15, the actual value is 1.322, the tangent predicts 1.300, and their gap is 0.0225. Halving h quarters the gap.Value, slope, then the missing pieceHalve h. The error drops to one quarter.1hx
f(1+h)=f(1+h)=11+2h+2h+h2+h^2
h = 0.150tangent = 1.300actual = 1.322gap h² = 0.0225curvetangenterror
First locate the value, then build the tangent prediction, then reveal the gap to the curve. Halving h halves the linear change but quarters the error h².
Value, slope, and the part the tangent misses.

For matrices, restrict the loss to a line, ϕ(t)=L(W+tD)\phi(t)=L(W+tD). The chain rule gives

ϕ′(0)=∑i,j∂L∂WijDij=⟨G,D⟩F.\phi'(0)=\sum_{i,j}\frac{\partial L}{\partial W_{ij}}D_{ij} =\langle G,D\rangle_F.

That is where Frobenius enters: it packages the sum of slopes times changes. Differentiability gives the precise statement

L(W+D)=L(W)+⟨G,D⟩F+R(D).\begin{aligned} L(W+D)&=L(W)+\langle G,D\rangle_F\\[6pt] &\quad+R(D). \end{aligned}

The remainder becomes small relative to the step:

R(D)∥D∥F⟶0as D→0.\frac{R(D)}{\|D\|_F}\longrightarrow0\quad\text{as }D\to0.

A quadratic error bound needs extra smoothness, such as a continuous Hessian nearby. For our earlier quadratic, the remainder is exactly 12(d12+4d22)\tfrac12(d_1^2+4d_2^2). An oversized step can let this positive remainder overwhelm the predicted decrease.

§Decide how far we are willing to move

We can now compare steps by their predicted change in loss. But the prediction only uses local slopes. We choose a size limit so we stay close enough for it to be useful. A huge downhill step can overshoot, raise the actual loss, and send training bouncing around. The tangent line will happily recommend it anyway.

Call the limit η>0\eta>0, our step budget. We choose both its size and how to measure it; a limit alone does not guarantee convergence. Measuring size with the Frobenius norm gives

min⁡∥D∥F≤η⟨G,D⟩F.\min_{\|D\|_F\leq\eta}\langle G,D\rangle_F.

Read this as: among all steps within the budget, choose the one with the most negative predicted loss change. The gradient GG is fixed at our current weights; the step DD is what we get to choose.

To see the geometry, imagine listing the matrix entries in vectors. For nonzero GG and DD, call the angle between those vectors φ\varphi:

⟨G,D⟩F=∥G∥F∥D∥Fcos⁡φ.\langle G,D\rangle_F=\|G\|_F\|D\|_F\cos\varphi.

The quantity ∥D∥Fcos⁡φ\|D\|_F\cos\varphi is the signed projection of the step onto the gradient direction. Multiplying by ∥G∥F\|G\|_F turns that displacement into a predicted loss change. Moving with the gradient gives a positive score; moving against it gives a negative one. A perpendicular step scores zero, which predicts no first-order improvement. We want the most negative score our budget allows.

Cauchy–Schwarz says ∣⟨G,D⟩F∣≤∥G∥F∥D∥F|\langle G,D\rangle_F|\leq\|G\|_F\|D\|_F. Taking the negative side of that bound and applying our budget,

⟨G,D⟩F≥−∥G∥F∥D∥F≥−η∥G∥F.\langle G,D\rangle_F \geq-\|G\|_F\|D\|_F \geq-\eta\|G\|_F.

For G≠0G\neq0, choose DF=−ηG/∥G∥FD_F=-\eta G/\|G\|_F. Substitution gives ⟨G,DF⟩F=−η∥G∥F\langle G,D_F\rangle_F=-\eta\|G\|_F, exactly the lower bound. There is our minus sign and our best direction.

Ordinary SGD uses −αG-\alpha G: the same direction, with step length α∥G∥F\alpha\|G\|_F instead of a fixed η\eta.

Our budget so far measures the combined size of the changes to the matrix entries. There is another useful question: how much will this update change what the layer outputs?

To see the difference, feed the same input vector xx into the layer before and after the update. Here xx means the activations entering this layer. For the linear part of the layer, call the output zz:

zbefore=Wx,zafter=(W+D)x,=Wx+Dx,zafter−zbefore=Dx.\begin{aligned} z_{\mathrm{before}}&=Wx,\\[8pt] z_{\mathrm{after}}&=(W+D)x,\\[8pt] &=Wx+Dx,\\[8pt] z_{\mathrm{after}}-z_{\mathrm{before}}&=Dx. \end{aligned}

The old output WxWx cancels. An unchanged bias would cancel too. What remains, DxDx, is exactly the change caused by updating the weights. Even in one dimension, this distinction matters: changing a weight from 2 to 2.1 changes the output for input 3 from 6 to 6.3. The weight changed by 0.1; the output changed by 0.1×3=0.30.1\times3=0.3.

So we can choose a budget based on the update's effect on the layer's outputs. We need it to cover different inputs, not just one example. That leads to the next question: which input direction does DD change the most?

§Measure the largest output change

Keep the same loss prediction, but measure the update by its largest output change per unit of input length.

We must fix the input length before comparing directions. Otherwise, simply doubling xx doubles DxDx, even though the update DD has not changed. Try every input with length 1 and measure the resulting change DxDx. The largest length we find is the spectral norm, also called the Euclidean operator norm:

∥D∥op=max⁡∥x∥2=1∥Dx∥2.\|D\|_{\mathrm{op}}=\max_{\|x\|_2=1}\|Dx\|_2.

Here ∥x∥2\|x\|_2 is Euclidean vector length (the size of the vector, x12+⋯+xn2\sqrt{x_1^2+\cdots+x_n^2}). Fixing it at 1 lets us compare directions fairly.

Sweep unit inputs to find the spectral normUnit input (0.707,0.707) maps to (1.414,1.414), with output length 2.000. The maximum is 2.Which unit input gets stretched the most?The longest output has length 2.input x · length 1output DxD
D=[1.50.50.51.5]D=\begin{bmatrix}1.5&0.5\\0.5&1.5\end{bmatrix}
‖Dx‖₂ = 2.0002
x∗=12[11],∥D∥op=2x_*={1\over\sqrt2}\begin{bmatrix}1\\1\end{bmatrix},\quad\|D\|_{\rm op}=2
A unit input sweeps around the circle while its image sweeps an ellipse. For this D, the longest image has length 2, attained at x = (1, 1)/√2 and its negative. The animation then settles on one maximizing direction.
Keep the input length fixed. Search over its direction.

A spectral budget ∥D∥op≤η\|D\|_{\mathrm{op}}\leq\eta guarantees ∥Dx∥2≤η∥x∥2\|Dx\|_2\leq\eta\|x\|_2 for every input. It controls this layer's worst output change per unit input. It does not, by itself, control the whole network's loss curvature.

For a diagonal update, the two budgets are easy to see:

D=diag⁡(d1,d2)=[d100d2],D=\operatorname{diag}(d_1,d_2) =\begin{bmatrix}d_1&0\\0&d_2\end{bmatrix},
∥D∥F=d12+d22,∥D∥op=max⁡(∣d1∣,∣d2∣).\begin{aligned} \|D\|_F&=\sqrt{d_1^2+d_2^2},\\[10pt] \|D\|_{\mathrm{op}}&=\max(|d_1|,|d_2|). \end{aligned}

Set η=1\eta=1. Each point in the animation is a candidate update (d1,d2)(d_1,d_2). Shading marks allowed steps; the outline is the budget limit. Frobenius gives a disk, and the spectral norm gives a square.

Find the best allowed step by sliding an equal-score line to the budget boundaryExample 3. 4. Last contact: this is the best allowed update. The blue arrow is minus the gradient: a direction of decreasing predicted loss, with arbitrary drawn length. The orange dot is the candidate update; its matrix and score change as it moves. The dashed line joins equal scores.Example 3 · budget η = 1
G=[200−1]G=\begin{bmatrix}2&0\\0&-1\end{bmatrix}
4. Last contact: this is the best allowed updateFrobenius
d12+d22≤1\sqrt{d_1^2+d_2^2}\leq1
d₁d₂−gBest update D*−0.8940.0000.0000.447score = −2.24Spectral
max⁡(∣d1∣,∣d2∣)≤1\max(|d_1|,|d_2|)\leq1
d₁d₂−gBest update D*−1.0000.0000.0001.000score = −3.00−g: direction, not step lengthD: the update being considereddashed: same predicted change
Read the stages in order: allowed steps, the −g direction, decreasing score, then the last contact. The blue arrow is a direction guide. The orange point is the candidate update; its matrix and score change together. Both panels use the same gradient and budget η = 1.
Start at score zero. Follow the blue arrow toward lower scores.

Write the score as c(d1,d2)=g1d1+g2d2c(d_1,d_2)=g_1d_1+g_2d_2. The blue arrow points along −g=(−g1,−g2)-g=(-g_1,-g_2), where this score decreases. Its drawn length is just for readability. The orange dot is a candidate update DD, and the dashed line passes through all updates with the same score as that dot.

Start at the origin: no weight change, score zero. Now slide the dashed line in the arrow's direction. More negative scores mean larger predicted decreases. Stop when the line is about to lose contact with the allowed region. The final orange dot is the best allowed update. The arrow tells us which way is better; the boundary decides how far we can go.

Keep our loss prediction in view:

L(W+D)≈L(W)+⟨G,D⟩F⏟the change we minimize.\begin{aligned} L(W+D)&\approx L(W)\\[8pt] &\quad+\underbrace{\color{#b54328}{\langle G,D\rangle_F}}_{\text{the change we minimize}}. \end{aligned}

We changed which steps are allowed. The highlighted score is still the Frobenius inner product.

§Choose a step in each independent direction

The diagonal examples show what the new budget allows. We now need the best step for a general matrix, whose useful directions may not line up with the coordinate axes:

min⁡∥D∥op≤η⟨G,D⟩F.\min_{\|D\|_{\mathrm{op}}\leq\eta}\langle G,D\rangle_F.

The useful fact is that we can choose perpendicular input and output coordinates in which the matrix only stretches directions. This is the singular value decomposition, or SVD. It gives unit input directions viv_i, unit output directions uiu_i, and stretch factors σi\sigma_i, called singular values. In these coordinates, Gvi=σiuiGv_i=\sigma_i u_i.

We will use the SVD theorem without proving it here. It says that every real matrix, square or rectangular, can be written as UΣV⊤U\Sigma V^\top. You can accept that result for this derivation, or follow the construction in Gilbert Strang's SVD lecture. What matters for our optimizer is what the factors do.

For an m×nm\times n matrix, the full factorization has U∈Rm×mU\in\mathbb R^{m\times m}, Σ∈Rm×n\Sigma\in\mathbb R^{m\times n}, and V∈Rn×nV\in\mathbb R^{n\times n}. The middle factor has the same rectangular shape as the original matrix, with nonnegative entries on its diagonal and zeros everywhere else. UU and VV are orthogonal, so they preserve lengths.

Read UΣV⊤U\Sigma V^\top right to left: V⊤V^\top changes input coordinates, Σ\Sigma stretches each coordinate, and UU changes to output coordinates. Those coordinate changes can include reflections. The original matrix need not look diagonal at all.

For G=[3001]G=\begin{bmatrix}3&0\\0&1\end{bmatrix}, the useful directions are already the axes. It triples one and leaves the other unchanged, so U=V=IU=V=I and Σ=G\Sigma=G. A less conveniently aligned matrix needs the coordinate changes too.

SVD rotates and stretches a circle; the polar factor removes the stretchReplace both singular values with ones. A two-dimensional SVD with U a 30 degree rotation, singular values 2 and one half, and V transpose a minus 45 degree rotation. The current lengths of the two marked directions are 1.000 and 1.000. At the end both lengths are one and the polar factor preserves lengths.From a circle to an ellipse, and backReplace both singular values with onesu₁u₂
G=UΣV⊤ ⟶ Q=UV⊤G=U\Sigma V^\top\ \longrightarrow\ Q=UV^\top
Qvi=uiQv_i=u_i
Σ⟶I\Sigma\longrightarrow I
Lengths of the marked directions1.001.0012Dashed circle: length 1
Follow the marked input directions through Vᵀ, Σ, and U. This full-rank 2 × 2 example has stretches 2 and ½, with input and output rotations of 45° and 30°. The final stage previews what happens when both stretches become 1.
Follow the factors from right to left.

The animation gives us the directions; now we can write the score in those coordinates. The compact SVD keeps only the rr directions with positive stretches.

For nonzero GG, its compact SVD is

G=UrΣrVr⊤=∑i=1rσiuivi⊤.G=U_r\Sigma_rV_r^\top =\sum_{i=1}^r\sigma_i u_i v_i^\top.

Here rr counts the independent active directions, and Σr\Sigma_r stores the positive stretches on its diagonal. The columns of UrU_r and VrV_r are the output and input directions.

The largest σi\sigma_i is ∥G∥op\|G\|_{\mathrm{op}}. Combining all their squared lengths gives ∥G∥F=∑iσi2\|G\|_F=\sqrt{\sum_i\sigma_i^2}.

Unlike eigenvalue diagonalization, SVD uses separate input and output coordinates. That is why the same construction works for rectangular matrices too.

Now the score separates into those directions:

⟨G,D⟩F=∑i=1rσi ui⊤Dvi.\langle G,D\rangle_F=\sum_{i=1}^r\sigma_i\,u_i^\top Dv_i.

Read one term before adding them all. The vector DviDv_i is the output change caused by the update when the input is viv_i. The scalar ui⊤Dviu_i^\top Dv_i is its signed component along the unit output direction uiu_i. Positive means along uiu_i; negative means against it. The positive number σi\sigma_i weights that component in our score.

Our spectral budget limits the output change for every unit input. Each viv_i is one of those inputs, so

∥Dvi∥2≤∥D∥op∥vi∥2≤η.\|Dv_i\|_2\leq\|D\|_{\mathrm{op}}\|v_i\|_2 \leq\eta.

A signed component cannot have greater magnitude than the vector itself. A vector of length at most η\eta therefore cannot project farther than −η-\eta in the opposite direction. Written as inequalities,

ui⊤Dvi≥−∥Dvi∥2≥−η.u_i^\top Dv_i\geq-\|Dv_i\|_2\geq-\eta.

The most negative value occurs when Dvi=−ηuiDv_i=-\eta u_i: use the full allowed length and point exactly against uiu_i. Multiplying each bound by σi>0\sigma_i>0 preserves its direction. Adding gives

⟨G,D⟩F=∑i=1rσiui⊤Dvi≥∑i=1rσi(−η)=−η∑i=1rσi.\begin{aligned} \langle G,D\rangle_F &=\sum_{i=1}^r\sigma_i u_i^\top Dv_i\\[10pt] &\geq\sum_{i=1}^r\sigma_i(-\eta)\\[10pt] &=-\eta\sum_{i=1}^r\sigma_i. \end{aligned}

That is a lower bound. To prove it is the best attainable score, we still need one matrix that sends every active viv_i to −ηui-\eta u_i without breaking the budget. Choose

D∗=−ηUrVr⊤.\boxed{D_*=-\eta U_rV_r^\top.}

To check this, multiply the proposed update by one input direction viv_i. Let eie_i be the coordinate vector with a 1 in position ii and zeros elsewhere. The columns of VrV_r have unit length and are perpendicular to one another, so their dot products with viv_i give

Vr⊤vi=ei.V_r^\top v_i=e_i.

Multiplying a matrix by eie_i selects its iith column, so Urei=uiU_re_i=u_i. Substitute those two facts, one at a time:

D∗vi=(−ηUrVr⊤)vi=−ηUr(Vr⊤vi)=−ηUrei=−ηui.\begin{aligned} D_*v_i&=(-\eta U_rV_r^\top)v_i\\[8pt] &=-\eta U_r(V_r^\top v_i)\\[8pt] &=-\eta U_re_i\\[8pt] &=-\eta u_i. \end{aligned}

Thus this single update matrix sends every active input direction to the opposite output direction with length η\eta. Every term in the score reaches its lower bound.

It also respects the budget for combinations of inputs. Vr⊤V_r^\top extracts the coordinates in the active input directions without increasing length; UrU_r maps those coordinates into orthonormal output directions without changing their length. Multiplication by η\eta scales that length by η\eta. So ∥D∗x∥2≤η∥x∥2\|D_*x\|_2\leq\eta\|x\|_2 for every xx, with equality on each active viv_i.

We have both pieces: no allowed update can score lower, and this update achieves the bound. After factoring out the minus sign and step size, its effect is to replace the positive stretches (the singular values stored in Σr\Sigma_r) by ones:

UrΣrVr⊤⟼UrIrVr⊤=UrVr⊤.U_r\Sigma_rV_r^\top\longmapsto U_rI_rV_r^\top=U_rV_r^\top.

Call UrVr⊤U_rV_r^\top the polar factor, written polar⁡(G)\operatorname{polar}(G). It keeps the active input and output directions and replaces their positive stretches by 1. We have derived the matrix transformation at the center of Muon. Next we need to decide which gradient estimate to feed into it, and how to compute it cheaply.

§Give the direction some memory

So far we have treated GG as the direction to trust. In training, we usually compute it from a minibatch, a small sample of the data. Different samples give different gradients. Before applying the polar transformation, combine recent gradients into a steadier estimate MtM_t:

Mt=βMt−1+(1−β)Gt.M_t=\beta M_{t-1}+(1-\beta)G_t.

Start with M0=0M_0=0 and choose 0≤β<10\leq\beta<1. With β=0.8\beta=0.8, keep 80% of the old estimate and add 20% of the incoming gradient. The animation builds that sum.

β = 0.8, starting from M₀ = 0.
Momentum builds a weighted sum, with formulas synchronized to the arrowsBatch 12. Use the result as the next memory. Previous estimate 0.923, 0.128 times 0.8, plus incoming gradient 1.000, −0.900 times 0.2, gives 0.939, −0.077. Formula values are rounded to three decimals.Batch 12 · β = 0.8Use the result as the next memory
MtM_t==βMt−1\beta M_{t-1}++(1−β)Gt(1-\beta)G_t
≈\approx0.8[0.9230.128]0.8\begin{bmatrix}0.923\\0.128\end{bmatrix}++0.2[1.000−0.900]0.2\begin{bmatrix}1.000\\-0.900\end{bmatrix}
≈\approx[0.7390.103]\begin{bmatrix}0.739\\0.103\end{bmatrix}++[0.200−0.180]\begin{bmatrix}0.200\\-0.180\end{bmatrix}
MtM_t≈\approx[0.939−0.077]\begin{bmatrix}0.939\\-0.077\end{bmatrix}
entry 1entry 2memorythis batchnew estimateSecond entry over successive batches012 batches
Blue is the retained estimate; dark ink is the new contribution. Add them head to tail to get orange. The vectors list two diagonal entries; values are rounded to three decimals. The lower trace follows the second entry across batches.

Now put that memory on a loss surface. The dark path follows each noisy gradient; the blue path averages them before stepping. Both start at the same weights, use the same learning rate, and receive the same sequence of batch perturbations. Each computes its gradient at its own current position.

Compare noisy gradient descent and momentum on a three-dimensional loss surfaceStep 48 of 48. Raw gradients and momentum on the same quadratic loss. Noise on, momentum beta 0.8. Both use learning rate 0.14. Current losses 0.334 and 0.018.Same valley. Different memory.Step 48 / 48 · α = 0.14 · β = 0.8 · noise on
L(w1,w2)=12w12+6w22L(w_1,w_2)=\tfrac12w_1^2+6w_2^2
Height = lossminimumw₁w₂raw gradientmomentumMemory carries the path past w₁ = 0.Smoothing does not remove overshoot.Position along the valley, w₁0048 stepsLoss: raw 0.334 · memory 0.018Sideways travel: 16.68 vs 3.63
Height is loss; the two floor coordinates are weights. Sideways travel counts the total movement across the narrow valley. The lower plot shows the position along the valley: crossing below zero means passing the minimum in that direction. Both methods take 48 computed steps with α = 0.14. Smooth motion between steps is only a visual aid.
Drag to rotate. Compare noisy steps with their moving average.

Watch the dark path bounce across the valley while the blue path filters those changing directions. Then remove the noise and try longer memory. The blue path can pass the minimum because its stored direction takes time to reverse. Momentum reduces jitter in this example; it does not guarantee that we avoid overshooting. The learning rate and memory length both matter. Why Momentum Really Works explores this behavior in detail.

Use this momentum estimate in our update:

Wt+1=Wt−ηt polar⁡(Mt).\boxed{W_{t+1}=W_t-\eta_t\,\operatorname{polar}(M_t).}

Momentum followed by orthogonalization gives the idealized core of Muon. Its step optimizes the local score built from MtM_t, so it need not decrease the current minibatch's loss.

How much history is in that average, and why this order?

Expanding the recurrence gives Mt=(1−β)∑s=1tβt−sGsM_t=(1-\beta)\sum_{s=1}^t\beta^{t-s}G_s. Older gradients fade geometrically. The weights sum to 1−βt1-\beta^t, so starting at zero pulls early estimates toward zero.

Average first, then orthogonalize. These operations do not commute. In one dimension, averaging 1010 and −1-1 gives a positive number whose sign is +1+1. Averaging their signs gives zero. The sign function is the scalar version of the polar transformation.

So far, our optimizer has a clean formula and an SVD bill. Let's deal with the bill.

§Change the stretches without finding the directions

Computing a full decomposition at every training step would be expensive. But look at what we actually need from the momentum matrix:

M=UrΣrVr⊤,Q=UrIrVr⊤=UrVr⊤.\begin{aligned} M&=U_r\Sigma_rV_r^\top,\\[10pt] Q&=U_rI_rV_r^\top=U_rV_r^\top. \end{aligned}

The step from Σr\Sigma_r to IrI_r is the important choice here. Let's work through why we made it, starting with two numbers instead of the whole factorization.

Suppose the momentum matrix is diagonal, with stretches (singular values on the diagonal of Σr\Sigma_r) equal to 10 and 1. Its input and output directions are just the two coordinate axes. Set the allowed output-change budget to η=1\eta=1. We are choosing an update, so we may choose its length in each of those directions.

Recall our score: multiply each slope by its proposed change, then add. Here we use the momentum values 10 and 1 as averaged slopes. A more negative total is better according to that score. Consider these two choices:

Update along the two axesLargest output change for a unit inputScore using the momentum
−1-1 and −0.1-0.11110(−1)+1(−0.1)=−10.110(-1)+1(-0.1)=-10.1
−1-1 and −1-11110(−1)+1(−1)=−1110(-1)+1(-1)=-11

The first choice preserves the momentum's 10-to-1 ratio. The second moves farther along the weaker direction. Both obey the same budget, and the second has the more negative score. Keeping that ratio would give up an improvement we are allowed to take.

Why doesn't moving in both directions break the budget? Take any unit input x=(x1,x2)x=(x_1,x_2), so x12+x22=1x_1^2+x_2^2=1. The second update sends it to (−x1,−x2)(-x_1,-x_2), whose length is still 1. The input cannot have length 1 along both axes at once. That is why the spectral budget permits an update length of 1 along each axis. A Frobenius budget would instead measure the entries together as 12+12=2\sqrt{1^2+1^2}=\sqrt2 and reject this update when η=1\eta=1. These are different definitions of a small update.

Now connect those numbers to the general matrix. Recall that the SVD describes each input direction viv_i and its corresponding output direction uiu_i:

Mvi=σiui.Mv_i=\sigma_i u_i.

Here σi>0\sigma_i>0 is the original stretch. It tells us what MM does; it does not require our update DD to use that same length. Introduce ai≥0a_i\geq0 as the update length we choose in direction ii. To move against the momentum, we ask for

Dvi=−aiui.Dv_i=-a_i u_i.

This is our proposed update rule, not another property of the SVD. It means: when the input points along viv_i, the change in the layer's output points opposite uiu_i, with length aia_i. One matrix that does this in every active direction is

D=−Urdiag⁡(a1,…,ar)Vr⊤.D=-U_r\operatorname{diag}(a_1,\ldots,a_r)V_r^\top.

As in the multiplication we checked earlier, Vr⊤V_r^\top selects a coordinate, the diagonal matrix scales it by aia_i, and UrU_r turns it into uiu_i. For this choice of DD, the largest output stretch is max⁡iai\max_i a_i. The budget therefore says each aia_i may be at most η\eta.

The score also comes from earlier: we used ⟨G,D⟩F\langle G,D\rangle_F to predict the loss change. After introducing momentum, we use ⟨M,D⟩F\langle M,D\rangle_F to choose a step from the averaged slopes. This is a momentum-based score, not an exact prediction from the current gradient. Expanding it in the singular directions gives the same calculation as before:

⟨M,D⟩F=∑iσiui⊤Dvi=∑iσiui⊤(−aiui)=−∑iσiai.\begin{aligned} \langle M,D\rangle_F &=\sum_i\sigma_i u_i^\top Dv_i\\[8pt] &=\sum_i\sigma_i u_i^\top(-a_i u_i)\\[8pt] &=-\sum_i\sigma_i a_i. \end{aligned}

The last equality uses ui⊤ui=1u_i^\top u_i=1. Each positive σi\sigma_i multiplies a negative term, so increasing aia_i makes the score smaller. We can increase every aia_i to η\eta without exceeding the spectral budget. The earlier lower-bound argument shows this choice is optimal even among updates outside this proposed form.

All chosen lengths are now equal. Factoring out their common size η\eta leaves ones on the diagonal:

D∗=−ηUrIrVr⊤,Q=UrIrVr⊤.\begin{aligned} D_*&=-\eta U_rI_rV_r^\top,\\[8pt] Q&=U_rI_rV_r^\top. \end{aligned}

That is why we replace the positive stretches (the singular values stored in Σr\Sigma_r) by 1 in QQ. We keep the directions and choose new update lengths to fit our budget. The learning rate supplies the common size afterward. Only the middle factor becomes an identity; QQ still maps the input directions to their corresponding output directions and generally is not the identity matrix.

Newton–Schulz offers a way to approach this target using repeated matrix multiplications. Before deriving its update, we need two tools: a multiplication that changes the stretches without changing their directions, and a scalar rule that brings each stretch toward 1 and keeps it there.

Start with the multiplication. Let XX be our current approximation. The following identity changes its stretches without computing the directions explicitly.

If X=UrΣrVr⊤X=U_r\Sigma_rV_r^\top, then

XX⊤X=UrΣr3Vr⊤.XX^\top X=U_r\Sigma_r^3V_r^\top.

Multiplying by the transpose and then itself cubes every singular value while retaining the singular directions. That means

aX+bXX⊤X=Ur(aΣr+bΣr3)Vr⊤.aX+bXX^\top X=U_r(a\Sigma_r+b\Sigma_r^3)V_r^\top.

We can apply p(s)=as+bs3p(s)=as+bs^3 to all singular values using matrix multiplications alone. We never have to compute UrU_r or VrV_r. That leaves a scalar design problem: choose pp so repeating it brings positive stretches (singular values stored in Σr\Sigma_r) to 1.

Show the factors cancelling

The compact SVD has Vr⊤Vr=IrV_r^\top V_r=I_r and Ur⊤Ur=IrU_r^\top U_r=I_r. Expand the product:

XX⊤X=(UrΣrVr⊤)(VrΣrUr⊤)(UrΣrVr⊤)=UrΣrVr⊤Vr⏟IrΣrUr⊤Ur⏟IrΣrVr⊤=UrΣr3Vr⊤.\begin{aligned} &XX^\top X\\[8pt] &=(U_r\Sigma_rV_r^\top)(V_r\Sigma_rU_r^\top) (U_r\Sigma_rV_r^\top)\\[10pt] &=U_r\Sigma_r\underbrace{V_r^\top V_r}_{I_r} \Sigma_r\underbrace{U_r^\top U_r}_{I_r}\Sigma_rV_r^\top\\[10pt] &=U_r\Sigma_r^3V_r^\top. \end{aligned}

The same argument produces higher odd powers of Σr\Sigma_r. If a polynomial produces a negative value, that sign can be absorbed into a singular vector; the actual singular value is its magnitude. Our classical cubic below stays nonnegative on the interval we use.

§Make one the destination, and keep it there

Now we can work on one stretch at a time. If all the positive diagonal entries approach 1, then

Σr,k⟶Ir,Xk=UrΣr,kVr⊤⟶UrVr⊤=Q.\begin{aligned} \Sigma_{r,k}&\longrightarrow I_r,\\[10pt] X_k=U_r\Sigma_{r,k}V_r^\top&\longrightarrow U_rV_r^\top=Q. \end{aligned}

That is why 1 is the destination. Reaching it in the scalar calculation gives us the polar direction in the matrix calculation. We still need a rule for getting there.

Take one positive stretch s0s_0 and repeatedly feed the result of a function back into that same function:

s1=p(s0),s2=p(s1),…s_1=p(s_0),\qquad s_2=p(s_1),\qquad \ldots

In general, sk+1=p(sk)s_{k+1}=p(s_k). Here kk counts these inner iterations, not training steps. We want the numbers to approach 1. Once a number gets there, another iteration should leave it there, so we require p(1)=1p(1)=1. A value that a function leaves unchanged is called a fixed point.

Staying at the destination is only half the job. Nearby values also need to move toward it. For example, p(s)=12s+12p(s)=\tfrac12s+\tfrac12 sends 0.90.9 to 0.950.95, then 0.9750.975, halving the distance to 1 each time. But p(s)=2s−1p(s)=2s-1 sends 0.90.9 to 0.80.8, then 0.60.6. Both fix 1; only the first attracts nearby values. A fixed point can be a place you approach or a place you get kicked away from.

Watch a value go into the function, then come back as its next input

A fixed point is where y=p(s)y=p(s) meets y=sy=s. Orange goes vertically to evaluate the function, then horizontally to the diagonal to reuse the output. The number also moves into the formula above the graph, so the geometry and substitution tell the same story.

Plug in the number. Reuse the result.
An animated staircase compares attracting and repelling fixed pointsRepelling fixed point at 1. The error doubles at every step. Input 0.6000 moves into the fixed formula, which produces 0.2000. The output moves to the diagonal and is reused as the next input.Same fixed point, opposite behaviorThe error doubles at every step
p(s)=2s−1p(s)=2s-1
2×2\times
−1-1
p(0.6000)=p(0.6000)=
0.200000.50.511p(s)y = s
p(1)=1alwaysp(1)=1\quad\text{always}
s2=0.6000s_2=0.6000
s3=0.2000s_{3}=0.2000
∣ek+1∣=2∣ek∣|e_{k+1}|=2|e_k|
Error: eₖ = sₖ − 10.6000
Both maps fix 1. Starting at 0.9, p(s) = ½s + ½ halves the error; p(s) = 2s − 1 doubles it. Watch the input travel into the function, the arithmetic appear, and the result feed the next iteration.

We want the attracting behavior, with fast error reduction near the target. Write the current stretch as s=1+es=1+e, where ee is its error. The local approximation we used for the loss now tells us how the next error behaves. Assuming p(1)=1p(1)=1,

p(1+e)−1=p′(1)e+O(e2).p(1+e)-1=p'(1)e+O(e^2).

So p′(1)p'(1) multiplies the leading part of the error. A slope with magnitude below 1 shrinks nearby errors; choosing p′(1)=0p'(1)=0 removes that linear term altogether. For the cubic p(s)=as+bs3p(s)=as+bs^3 that our matrix multiplication can compute, the two requirements are

p(1)=1keeps the target fixed,p′(1)=0removes the linear error.\begin{aligned} p(1)=1&\quad\text{keeps the target fixed},\\[10pt] p'(1)=0&\quad\text{removes the linear error}. \end{aligned}

Substituting the cubic and its derivative gives

a+b=1,a+3b=0.\begin{aligned} a+b&=1,\\[8pt] a+3b&=0. \end{aligned}

Subtract the first equation from the second: 2b=−12b=-1. So b=−1/2b=-1/2 and a=3/2a=3/2. Putting these coefficients back into the scalar rule and its matrix counterpart gives the classical Newton–Schulz iteration:

p(s)=32s−12s3,Xk+1=32Xk−12XkXk⊤Xk.\begin{aligned} p(s)&=\tfrac32s-\tfrac12s^3,\\[10pt] X_{k+1}&=\tfrac32X_k-\tfrac12X_kX_k^\top X_k. \end{aligned}

Normalize a nonzero momentum matrix with X0=M/∥M∥FX_0=M/\|M\|_F. Since ∥M∥op≤∥M∥F\|M\|_{\mathrm{op}}\leq\|M\|_F, its singular values start in [0,1][0,1]. Repeated cubic steps bring each positive one toward 1 and keep zero at zero. Consequently, the middle factor approaches IrI_r, and XkX_k approaches Q=UrVr⊤Q=U_rV_r^\top. The scalar fixed point gives us the matrix direction we wanted. Zero stretches stay zero, so this also preserves the active subspace when the original matrix has missing directions.

Why do the iterates actually reach one?

For 0<s<10<s<1,

p(s)−s=12s(1−s2)>0,1−p(s)=12(1−s)2(s+2)≥0.\begin{aligned} p(s)-s&=\tfrac12s(1-s^2)>0,\\[10pt] 1-p(s)&=\tfrac12(1-s)^2(s+2)\geq0. \end{aligned}

The value increases without crossing 1. Its limit must be a fixed point; the only positive one in this interval is 1. The squared error factor gives quadratic convergence near 1. Very small starting values can still need many iterations to get close.

More generally, ∣p′(1)∣<1|p'(1)|<1 gives local attraction. Setting it to zero improves the local error reduction; it does not, on its own, prove that arbitrary starting values converge. That is why we checked the interval.

§Get a useful approximation in five steps

We now have an iteration that converges, but a tiny initial stretch can take many steps to get close to 1. Every training step contains this inner matrix calculation. We need to stop it somewhere, or we will spend the training budget polishing one update. The original Muon experiments found five inner steps sufficient for the models tested. Five is a compute-budget choice, not a convergence guarantee. They use this tuned quintic:

q(s)=3.4445s−4.7750s3+2.0315s5.q(s)=3.4445s-4.7750s^3+2.0315s^5.

Its slope at zero is 3.44453.4445, versus the cubic's 1.51.5, so small values initially grow faster. There is a catch: q(1)=0.701q(1)=0.701. One is not even a fixed point.

The bargain is fast early progress toward unit-sized stretches, with some error left over. Running this same quintic forever does not finish the job; it can keep oscillating. Our two cubic conditions do not derive these tuned coefficients. The Newton–Schulz notes explain this choice and the resulting oscillation.

Substitute each value into the cubic and quintic, then return the result to the graphStarting at 0.1, iteration 12. The cubic receives 1.000000 and returns 1.000000. The quintic receives 0.703558 and returns 1.110680. Both substitutions and their numerical terms are shown beside the trace.Start 0.1 · iteration 12 of 12Return each result to the graph1value sₖ0512 · k5-step budgetcubicquinticBand: 0.7 ≤ s ≤ 1.3quintic: in band at step 2cubic: in band at step 6Past 5: the quintic oscillates.
p(s)=1.5s−0.5s3p(s)=1.5s-0.5s^3
s11=s_{11}=
1.0000
≈1.5(1.0000)−0.5(1.0000)3\approx1.5(1.0000)-0.5(1.0000)^{3}
≈1.50000−0.50000\approx1.50000-0.50000
s12≈s_{12}\approx
1.0000
q(s)=3.4445s−4.775s3+2.0315s5q(s)=3.4445s-4.775s^3+2.0315s^5
s11=s_{11}=
0.7036
≈3.4445(0.7036)−4.775(0.7036)3\approx3.4445(0.7036)-4.775(0.7036)^{3}
+2.0315(0.7036)5+2.0315(0.7036)^{5}
≈2.42340−1.66292+0.35020\approx2.42340-1.66292+0.35020
s12≈s_{12}\approx
1.1107Substitutions are rounded; the recurrence uses full precision.
Starting at 0.1, the quintic first enters the shaded 0.7–1.3 band at step 2; the cubic takes 6. The band is an illustrative tolerance around 1. Rings mark the first arrival, and the vertical line marks the five-step compute budget. Each value travels into its polynomial and back to the graph. Keep going to see the difference between getting close early and converging. Try starts at 1 and 0 too. Arithmetic is displayed rounded and computed at full precision.
Substitute, calculate, and reuse each result.

A quintic q(s)=as+bs3+cs5q(s)=as+bs^3+cs^5 has three coefficients. We could impose q(1)=1q(1)=1 and q′(1)=0q'(1)=0, then use the remaining freedom to shape the interval. Muon's original coefficients make a different tradeoff. Polynomial design includes choosing the conditions, not only solving for the coefficients.

Show the matrix code and the practical optimizer details

Here is the five-step direction calculation in float32. The reference implementation uses bfloat16 for speed.

import torch

@torch.no_grad()
def muon_direction(M, steps=5, eps=1e-7):
    """Approximate polar direction for one finite 2D matrix."""
    assert M.ndim == 2
    X = M.float()
    transposed = X.shape[0] > X.shape[1]
    if transposed:
        X = X.T
    X = X / (X.norm() + eps)  # Frobenius norm; zero stays zero.
    a, b, c = 3.4445, -4.7750, 2.0315
    for _ in range(steps):
        A = X @ X.T
        X = a * X + (b * A + c * (A @ A)) @ X
    return X.T if transposed else X

Transposing a tall matrix makes XX⊤XX^\top smaller and cheaper to multiply; the return restores its shape. The quintic uses the same SVD algebra, adding a Σr5\Sigma_r^5 term.

The reference optimizer also uses a Nesterov-style blend Ht=(1−β)Gt+βMtH_t=(1-\beta)G_t+\beta M_t by default, applies matrix-shape scaling, and supports decoupled weight decay. The finite quintic output need not have spectral norm exactly 1.

The reference setup uses Muon for hidden weight matrices and another optimizer, such as AdamW, for embeddings, the output layer, and scalar or vector parameters. The function above computes the direction, not the full training recipe.

§Put the pieces together: one Muon step

We have enough parts. Time to assemble the optimizer.

At training step tt, backpropagation gives the gradient GtG_t. Blend it into the momentum, normalize that matrix, and run five polynomial steps. Then scale the result by the learning rate and subtract it from the weights:

Mt=βMt−1+(1−β)Gt,X0=Mt∥Mt∥F+ε,Ak=XkXk⊤,Xk+1=aXk+(bAk+cAk2)Xk,Wt+1=(1−ηtλ)Wt−ηtX5.\begin{aligned} M_t&=\beta M_{t-1}+(1-\beta)G_t,\\[10pt] X_0&=\frac{M_t}{\|M_t\|_F+\varepsilon},\\[10pt] A_k&=X_kX_k^\top,\\[10pt] X_{k+1}&=aX_k+(bA_k+cA_k^2)X_k,\\[10pt] W_{t+1}&=(1-\eta_t\lambda)W_t-\eta_tX_5. \end{aligned}

Run the recurrence for k=0,…,4k=0,\ldots,4, with a=3.4445a=3.4445, b=−4.7750b=-4.7750, and c=2.0315c=2.0315. The tiny ε\varepsilon prevents division by zero. The optional decay λ\lambda shrinks the old weights separately from the gradient calculation. Set it to zero to recover the update we derived.

This is a minimal Muon step. Practical variants can blend in the current gradient again with Nesterov momentum, scale the update for the matrix shape, or use different polynomial schedules. The reference implementation makes those choices explicit. The central operation stays the same: turn the momentum matrix into an approximate polar direction.

Follow one complete Muon matrix update from gradient to new weightsOperation 10: Add the step to the weights. Displayed matrix 0.9319, 0.5515, −0.2841, 0.9198. The unit circle is mapped by exactly this matrix. Dotted arrows show the exact SVD target, with its scale or weight addition applied at the final operations.10 / 10 · Add the step to the weightsOne matrix through the complete Muon stepSolid: images of v₁ and v₂Dotted: W − 0.1Q
Wt+1=Wt+DW_{t+1}=W_t+D
Wt=W_t=
1.0000.500−0.2501.000
Wt+1=W_{t+1}=
0.9320.551−0.2840.920stretches: 1.150, 0.882gap to dotted target: 0.0252
Q=UV⊤=[3/2−1/21/23/2]Q=UV^\top=\begin{bmatrix}\sqrt3/2&-1/2\\1/2&\sqrt3/2\end{bmatrix}
Exact SVD reference · gap measured with the Frobenius normUpdated weights need not be orthogonal.
Follow one matrix through memory, normalization, five quintic iterations, and the weight update. Solid arrows show what the current matrix does to the momentum’s two singular input directions v₁ and v₂. Dotted arrows show the exact target Qvᵢ = uᵢ, and the printed gap measures the remaining error. At the last two operations the reference becomes −0.1Q, then W − 0.1Q, so we compare like with like. The ellipse is the current image of the unit circle; the dashed circle has radius 1. Settings: β = 0.8, η = 0.1, λ = 0, no Nesterov blend.
Follow one matrix through the whole update.

Here Q=UV⊤Q=UV^\top is the exact polar factor of the momentum matrix, computed as a reference for the illustration. Muon computes X5X_5 instead. The solid arrows approach the dotted ones without landing exactly on them, because five quintic steps leave residual stretch.

Momentum chooses which matrix we trust. Normalization puts its stretches in range. The polynomial reshapes them, and the minus sign turns the result into a weight update. The updated weights themselves need not be orthogonal.

§Same gradients, different optimizers

We have assembled a complete Muon step. To see what its matrix transformation changes, give the same gradient sequence to three update rules.

SGD uses the gradient directly. Adam keeps moving averages of both gradients and their elementwise squares, then divides each coordinate by its own estimated scale. AdamW adds weight decay separately. Muon rescales the momentum matrix along its singular directions.

Let RtR_t denote the gradient-driven part before the learning rate and decay:

RtSGD=Gt,RtAdamW=m^t⊘(v^t+ε),RtMuon=X5≈polar⁡(Mt).\begin{aligned} R_t^{\mathrm{SGD}}&=G_t,\\[10pt] R_t^{\mathrm{AdamW}}&=\widehat m_t\oslash \left(\sqrt{\widehat v_t}+\varepsilon\right),\\[10pt] R_t^{\mathrm{Muon}}&=X_5\approx\operatorname{polar}(M_t). \end{aligned}

An entry is one number in the matrix, such as the number in row 1, column 2. The symbol ⊘\oslash means dividing each number by the number in the same position in another matrix. AdamW keeps a separate running scale for each position. For example, if two momentum entries are 4 and 1, and their estimated scales are 4 and 1, their contributions become 4/4=14/4=1 and 1/1=11/1=1. Those scales come from past squared gradients; this is an example of the arithmetic, not a fixed rule that all entries become 1.

That is entrywise scaling: rescaling each matrix entry separately. Muon instead uses the matrix's singular directions, which can involve several entries moving together. The next comparison asks whether that distinction changes the path through a loss function.

Compute SGD, AdamW and Muon updates from six identical gradientsSupplied gradient 6 of 6. Three computed steps. Different shapes.. SGD, AdamW and Muon compute different updates from the same gradient sequence. Ellipses use unit Frobenius norm for display; raw norms and matrix entries are shown.Gradient 6 / 6Three computed steps. Different shapes.
Gt=G_t=
1.0000.400−0.2001.300η = 0.1same inputsSGD
Rt=GtR_t=G_t
D =−0.100−0.0400.020−0.130‖D‖F = 0.170No momentum · no decayAdamW
Rt=m^tv^t+εR_t={\widehat m_t\over\sqrt{\widehat v_t}+\varepsilon}
D =−0.0270.015−0.023−0.103‖D‖F = 0.110Momentum + decay λ = 0.1Muon
Rt=X5R_t=X_5
D =−0.0750.008−0.015−0.102‖D‖F = 0.127Momentum + decay λ = 0.1Shapes show D / ‖D‖F; the numbers show the actual step.Blue: image of (1, 0) · orange: image of (0, 1)
Replay the same six supplied gradient matrices through all three methods. Each panel shows the actual step D and its image of the unit circle. The images are divided by each step's Frobenius norm so their shapes fit the same scale; the unscaled norm is printed below. Watch the blue and orange basis arrows as well as the outline. This compares update rules, not training speed or final loss.
Same supplied gradients. Watch how each rule reshapes the update.

The first gradient has four positive entries. SGD keeps their relative sizes. AdamW's first bias-corrected gradient contribution is nearly the sign of each entry, so it is nearly rank one here. Muon's result keeps both matrix directions active. That difference comes from the rules, before we have trained anything.

The useful conclusion is that Muon gives weaker singular directions room to contribute instead of letting the strongest one dominate the update. That is what the rounder shape shows. It does not mean every step is smaller or gentler. These pictures normalize away the overall size; use the printed norms to compare size. The learning rate still decides how far we move, and a large rate can overshoot.

§Now give them a loss to minimize

The replay showed how the same gradients produce different steps. During training, those steps also change where the next gradient is evaluated. To see that feedback, give all three optimizers an actual loss and let each follow its own path.

Here is a quadratic loss with a narrow valley. In the two matrix directions shown below, it is

L(a,b)=12(40a2+b2).L(a,b)=\tfrac12(40a^2+b^2).

The bottom is at (0,0)(0,0). The loss curves forty times more sharply along aa than along bb. All three methods start at (1.5,1.5)(1.5,1.5), then recompute their gradients at their own new weights. Each curve is an actual optimization run.

Why compare two ways of writing this same valley? AdamW gives each matrix entry its own scale. We want to see whether its behavior changes when one useful movement is spread across several entries instead of stored in just one. Muon's rule uses singular directions, so this is a concrete way to compare what the two rules pay attention to.

Here aa measures movement across the steep side of the valley, and bb measures movement along its shallow side. The optimizer updates the entries of WW, not the letters aa and bb directly. We can encode those two coordinates in WW in either of these ways.

Separate entries. Put aa in the top-left entry and bb in the bottom-right entry. Hold a=1a=1 and increase only bb from 0 to 0.2:

[1000]⟶[1000.2].\begin{bmatrix}1&0\\0&0\end{bmatrix} \quad\longrightarrow\quad \begin{bmatrix}1&0\\0&0.2\end{bmatrix}.

Only the bottom-right number changes. AdamW's scale for that entry is based on gradients for the shallow movement alone.

Mixed entries. Store the same coordinates using W11=W22=(a+b)/2W_{11}=W_{22}=(a+b)/2 and W12=W21=(a−b)/2W_{12}=W_{21}=(a-b)/2. Make exactly the same change, a=1a=1 and b:0→0.2b:0\to0.2:

[0.50.50.50.5]⟶[0.60.40.40.6].\begin{bmatrix}0.5&0.5\\0.5&0.5\end{bmatrix} \quad\longrightarrow\quad \begin{bmatrix}0.6&0.4\\0.4&0.6\end{bmatrix}.

Now the shallow movement needs all four numbers to change: add 0.1 on the diagonal and subtract 0.1 off it. Increasing aa would instead add to all four entries. Each position therefore participates in both the steep and shallow movements. Its gradient history mixes their contributions, and AdamW's separate scale for that position cannot assign an independent scale to each movement.

The button Use separate entries for a and b switches from the second encoding to the first. It changes the formula for WW and reruns all three optimizers. It does not change L(a,b)L(a,b), so the bowl itself stays the same. Watch whether the optimizer follows the same path through that bowl. The formula and example matrix above the plots show the selected encoding; the matrices below show the actual weights during the run.

Both encodings start from a=b=1.5a=b=1.5, which gives W=1.5IW=1.5I in either case. That is why the starting weights look identical even though subsequent updates can differ. With the initial mixed encoding and learning rates, SGD bounces across the steep direction, AdamW makes slow progress along the shallow direction, and Muon gets farther down the valley in 80 steps. Use the switch to test how much of that result depends on the encoding.

Follow SGD, AdamW and Muon down a computed quadratic matrix lossActual matrix-loss optimization, step 80 of 80. The matrix basis mixes the two directions by 45 degrees. Current losses: SGD 0.0304436; AdamW 0.145701; Muon 0.0000156281. Every method recomputes its gradient from its own current weights.Training step 80 / 80Mixed matrix entries · basis rotated 45°How valley coordinates become matrix entries
W=12[a+ba−ba−ba+b]W=\frac12\begin{bmatrix}a+b&a-b\\a-b&a+b\end{bmatrix}
Same example point: a = 1, b = 00.5000.5000.5000.500The switch changes this map, not the valley axes.
L(a,b)=12(40a2+b2)L(a,b)=\tfrac12(40a^2+b^2)
-1-10011abstartminimum: L = 0Contours join points with equal loss.Loss · logarithmic scale10²110⁻⁴10⁻⁸04080training stepsSGD0.03044initial η = 0.04AdamW0.14570initial η = 0.08Muon1.56e−5initial η = 0.08Fresh gradients at every point.Try the basis and rate controls.λ = 0 · same rate schedule · different cost per stepThe same paths on the loss surfaceHeight = loss · drag to rotate · arrows also rotateminimumSGDAdamWMuonActual weight matrices at step 80SGD0.123−0.123−0.1230.123AdamW0.270−0.270−0.2700.270Muon−0.0030.0020.002−0.003
The left plot shows paths over equal-loss contours. The right plot tracks the actual loss on a logarithmic scale, so small values remain visible. Every point uses the same loss function shown above. Play the run, then try separating the two directions into different matrix entries or raising AdamW’s initial learning rate. Both changes can reverse the ranking. The third plot lifts these same paths onto the loss surface; height is the actual loss. Drag to rotate it, or use the arrow keys. Smooth motion connects computed training steps.
Follow three optimizers. Drag the surface to rotate.

Now press Use separate entries for a and b. The valley has the same shape, but its directions line up with the individual entries AdamW rescales. AdamW becomes much faster; SGD and Muon follow the same paths as before. Or keep the mixed basis and raise AdamW's learning rate to 0.320.32. It catches up and finishes below Muon. AdamW has declined our invitation to be deprecated.

Muon responds to the matrix's coupled directions, so rotating this example does not change its path in valley coordinates. That is a precise sense in which its update follows the matrix geometry. Calling it "more natural" is less useful than seeing which changes of coordinates leave its behavior intact.

There is only one minimum in this quadratic bowl. None of these paths demonstrates escaping a local minimum. At a point with zero gradient and no stored momentum, Muon also has no direction to follow. Momentum can carry an optimizer through a flat or turning region, but replacing singular values by one does not by itself guarantee escape.

The test answers the question we started with. Here, changing how the two movements are distributed across matrix entries changes AdamW's path, while SGD and Muon keep the same paths in the valley. AdamW scales those entries separately; Muon rescales the singular directions that combine them. The learning-rate control also shows that this difference does not establish a universal winner. We count training steps here; a Muon step does more work, so these curves do not measure wall-clock speed.

The matrix loss, gradients, and exact settings

The full problem has a 2×22\times2 weight matrix:

L(W)=12tr⁡(W⊤HW),G=HW,H=B[40001]B⊤.\begin{aligned} L(W)&=\tfrac12\operatorname{tr}(W^\top H W),\\[10pt] G&=HW,\\[10pt] H&=B\begin{bmatrix}40&0\\0&1\end{bmatrix}B^\top. \end{aligned}

The mixed basis uses B=12[1−111]B=\tfrac1{\sqrt2}\begin{bmatrix}1&-1\\1&1\end{bmatrix}; the aligned basis uses B=IB=I. All methods start at W0=1.5IW_0=1.5I. For these two choices of BB, their iterates stay in the slice W=Bdiag⁡(a,b)B⊤W=B\operatorname{diag}(a,b)B^\top. Substitution gives the two-dimensional loss plotted above. The code updates all four matrix entries, then reads off aa and bb.

SGD has no momentum and starts with learning rate 0.040.04. AdamW starts at 0.080.08 or 0.320.32, with β1=0.8\beta_1=0.8, β2=0.9\beta_2=0.9 and ε=10−8\varepsilon=10^{-8}. Muon starts at 0.080.08, with β=0.8\beta=0.8, five quintic iterations and ε=10−7\varepsilon=10^{-7}, without Nesterov or a shape multiplier. Every method has zero weight decay, so AdamW is equivalent to Adam in this example. Momentum histories start at zero; AdamW applies bias correction.

Each method scales its initial rate by the same schedule over 80 training steps:

ηt=η0[0.1+0.45(1+cos⁡πt80)].\eta_t=\eta_0\left[0.1+0.45\left(1+\cos\frac{\pi t}{80}\right)\right].

Here t=0,…,79t=0,\ldots,79. These settings make the geometric effect visible. The basis and learning-rate controls let us test how fragile any claim about the winner would be. The five inner polynomial iterations belong to each Muon training step; they are a different count from the 80 steps on the horizontal axis.

That is the optimizer we set out to invent. Use the local loss prediction to choose a direction, decide what "small" means for a matrix, remember recent gradients, and approximate the resulting polar transform cheaply. The remaining question is how well those choices work on the model we actually want to train.

§Where these choices came from

We reconstructed the update by choosing a loss prediction, an output-change budget, a gradient average, and a short polynomial calculation. That is a route to understanding the method. For its historical development and experimental results:

If I had to rebuild the argument, I'd start with DxDx: how much should one update be allowed to change the layer's output?

]]> after the white rabbit: an Unlock! adventure from tabletop to 3D https://lucastononro.github.io/after-the-white-rabbit/ https://lucastononro.github.io/after-the-white-rabbit/ Sat, 05 Sep 2026 12:00:00 GMT A browser adventure inspired by Unlock!, built with Codex, Blender and Three.js. How we adapted the puzzles and fixed the models that got in their way. agents I wanted a version of this game I could walk around in. Follow the rabbit through Wonderland, explore between puzzles and get close enough to inspect a clue. A room was fine when it fit the story. I just didn't want the whole thing stuck in one.

Fresh gameplay at normal speed, with game audio and continuous diagonal walking. Recorded from chapter checkpoints; pop-up menus are omitted. Contains puzzle spoilers. Download the MP4 to share.

§What is Unlock!?

Unlock! is a series of cooperative escape-room card games from Space Cowboys. You play alone or with other people around a table, using a deck and a free companion app. The cards show locations, objects and clues. You look for hidden details, work out which objects belong together and solve puzzles to advance the story. The app checks codes, runs interactive puzzles and gives you hints. It also keeps time, with adventures usually designed for about an hour.

In Pursuit of the White Rabbit is one of the three adventures in Unlock! Heroic Adventures. It takes Alice through Wonderland and its strange inhabitants. That gave this project both its story and its puzzle progression. I wanted to keep the deductions, then give you a world where you could make them by looking around and handling objects.

I worked with Codex from the French solution booklet. Save that link for after playing. It contains the answers. We made original environments, models and an interface for a single-player, first-person browser game. The puzzle logic follows the tabletop adventure closely. This is an independent fan adaptation with no affiliation to Space Cowboys.

§A place between the puzzles

The hall stays indoors because its puzzles depend on scale. A table can be out of reach, and a door can be too small. Beyond it, you follow paths through a wooded valley to the bridges, the pepper-filled cottage and the tea clearing. The Queen's grounds have separate areas to explore before you reach the final construction.

I wanted a reason to walk somewhere. Small objects become discoverable when you get close, and some clues need a different viewing angle. The Explore menu helps you return to landmarks, while the journal records what you've found. I also wanted time to look around, so this version has no countdown or penalty for a wrong answer.

§The rabbit had to behave like a rabbit

The rabbit was one of my first complaints. Bouncing the whole model up and down wasn't enough. We made separate animations in Blender for standing idle, hopping and checking his watch. His ears move with his head, and the pocket watch stays attached to his paw. He follows a route and waits when you fall behind.

The model is stylized, but the movement still has to make sense. Getting the ears and watch to follow the right body parts took attention.

§The first fidelity pass

When we checked the puzzles against the original, some were giving too much away. An early sign diagram drew most of its own solution. The journal explained the clock puzzle before you had to make the connection. We removed those explanations and revised the clues so you have to compare the messages and clocks yourself. The sign puzzle now asks you to trace its colored arrows.

The croquet puzzle had a different problem. It accepted the intended sequence even though the hedgehog rolled through an obstacle. We changed the lawn and made the shot checks use the visible geometry. A blocked shot now stops at the obstacle.

The guards were harder to model than they looked. An accidental difference between their uniforms became an extra clue, so we had to check every variation. At the tart, the pastry hid part of the crumb outline you needed to recognize. We cut the missing slice into the model and moved the crumbs into view. A sign also covered the gardeners' tunic numbers. These were small modeling mistakes that made the puzzles unfair.

For the final house, we replaced a list of selected pieces with a construction you assemble. You place each part, turn the bridges over, remove mistakes and test whether it holds. This version of the puzzle needed another pass, described in the study below.

§The first release still had bugs

We published too early. Codex reported passing checks, and I still ran into overlapping doors before leaving the opening hall. I hadn't solved the game yet. The earlier review checked answers and selected checkpoints, but missed problems in the route between them.

The doors made that gap obvious. Their panels overlapped, and parts of the arches looked broken. Door 6 looked big enough to walk through at a size the game refused. Its triangular lock was a tiny symbol, and a separate height condition could stop a collected key from working. Knowing the intended solution wasn't enough to tell whether I was stuck on a puzzle or a bug.

The first release, with overlapping door panels and broken-looking arches.
The first release, with overlapping door panels and broken-looking arches.

Before. This is the original published build, captured again without changing its models.

The original adventure asks Alice to shrink for this passage. The size rule was faithful, but our model gave the wrong evidence. We separated the doors and rebuilt the small opening to match the player's body. The triangular keyway is now part of a brass plate attached to the door. Once unlocked, the leaf swings open and you walk into the passage to leave the hall. The geometry and the interaction need to agree.

The repaired hall doors, with separate frames and a small passage at floor level.
The repaired hall doors, with separate frames and a small passage at floor level.

After. The opening has to explain why shrinking helps before the game asks you to do it.

§The wall gave away the answer

Then I noticed something even simpler. Door 6 had a hole behind it. Door 9 had a solid wall. I could pick the right door by looking at the unfinished geometry, without comparing the key and locks at all.

My screenshot of Door 6, with the rectangular wall opening visible above and behind its arch.
My bug report. The gap behind Door 6 gives away which door leads somewhere.
My comparison screenshot of Door 9 against an uninterrupted wall.
Door 9 has no matching gap. Comparing the scenery was enough to skip the deduction.

We rebuilt the casings in Blender. All five doors now have the same fitted jambs and arched interior, with solid wood covering the corners of the rectangular opening. Door 6 still opens into a passage after you unlock it.

The repaired Door 6 viewed from above and to one side, with a solid casing covering the wall cutout.
The repaired model in the game, viewed from above and to one side. The casing covers the cutout while the leaf remains hinged.

This was another accidental clue we had introduced. A correct answer check cannot catch that. We added checks from several player heights and side angles, including one through the passage after the leaf opens. The screenshots belong in the bug log too, because the first repair had left this problem behind.

The cake and bottle had another basic mistake. Both menus offered every size, so cake could shrink you and a drink could make you grow. Cake now offers larger sizes; the bottle offers smaller ones. Both stay available, so choosing the wrong size doesn't leave you stuck. This went into the bug log with the door failures and the rest of the repair pass.

§Reading and moving on a phone

The first phone layout was hard to read and use. The book had small controls and a scrollable list inside a scrollable popup. That cost space in landscape and made it easy to scroll the wrong part of the page.

The first phone popup, with a narrow reading panel and small book controls.
The first phone popup, with a narrow reading panel and small book controls.

Before, in the browser's 844 by 390 phone layout.

We gave phone puzzles the full width, enlarged the buttons and removed the extra scroll region. Close stays in a pinned toolbar. A separate Look at the clue button lets you switch back to the scene without losing your place in the puzzle.

The revised phone popup, with a pinned toolbar and larger book controls.
The revised phone popup, with a pinned toolbar and larger book controls.

After, at the same game viewport size.

The keyboard needs room too. We added a check with the available height reduced to 210 pixels, then made the answer field and submit button reachable in that space. This is a browser layout check, not evidence from a physical phone.

The answer field and submit button visible in a shortened phone viewport.
The answer field and submit button visible in a shortened phone viewport.

Reduced-height check with sample input. The keyboard itself is not being emulated.

Walking also felt twitchy and slow. We smoothed the joystick response, reduced look sensitivity and added a Run toggle. Small Alice still enters the second chapter because that follows the source, and the mushroom belongs to that part of the story. We removed the outdoor speed penalty. Looking small shouldn't turn the trip to the next clue into a crawl.

The residents needed another Blender pass as well. The frog and caterpillar were mostly static shapes, and the caterpillar got in the way of its appointment clock. The new characters have moving joints and facial animation. The cook's spoon stays in the cauldron, the Hatter raises his cup, and the Queen gestures while holding her scepter. We preserved the clue props and left the gardeners' numbers and guards' uniform differences alone.

The cook was a useful lesson in checking animation inside the scene. Her new face sat behind the old kitchen sign, and her stirring hand dipped below the pot's solid top. We moved the sign onto the counter front, raised the hand and lengthened the spoon. A spare hat also hid the Hatter's face. We moved that prop to the other end of the table and kept its price tag.

The frog messenger rebuilt in Blender, with separate limbs and facial features.
The frog messenger rebuilt in Blender, with separate limbs and facial features.

Blender model view of the revised frog. His speaking motion uses the mouth and throat; his body no longer has to bounce to suggest life.

The cook's raised stirring hand and face clear the cauldron in the Blender inspection render.
The cook's raised stirring hand and face clear the cauldron in the Blender inspection render.

Blender inspection render. The spoon still reaches inside the pot while the hand stays above its rim.

§What the recorded run caught

Recording the solution became another test. The fan could clear smoke while I stood beside a solid kitchen wall. We restricted it to the open front and made assisted walking approach that entrance. Closing the little passage's popup also used to reopen it immediately. It now responds when you enter, so closing it lets you move again.

Slow rendering exposed a separate walking bug. The game counted at most 50 milliseconds per frame, so a slow frame also slowed the player's travel. We corrected the timing and split movement into short collision steps. Assisted walking now keeps its remaining movement when it reaches a waypoint. This fixes lost travel time; it doesn't promise a particular frame rate on a phone.

The court's approach had a wall across part of the road. We pulled that wall back and widened the hedge opening. The source of this one was embarrassingly familiar. The visible world and the walking rules had been built separately.

A castle wall covering part of the road into the court during the repair playthrough.
A castle wall covering part of the road into the court during the repair playthrough.

Caught during the recording. This is an intermediate build in the repair pass, not the first published release.

The repaired approach, with the road visible through the court entrance.
The repaired approach, with the road visible through the court entrance.

The final entrance checked in the game. The path continues around the low maze.

The earlier repair run reaches the ending through all 25 puzzles. It uses known answers and resumes ordinary saved progress after the fixes found along the way. I kept the video and solution screenshots in the walkthrough below. It is useful evidence that the route works, but a first-time player may still find a clue confusing in a way this run couldn't reveal.

I put too much weight on the answer checks. A player also has to recognize the opening, turn the key, reach the input field and walk to the next scene. Those actions need a playthrough with the actual controls. The first release hadn't earned the confidence we gave it.

§Keeping the game while changing the medium

Changing your height and comparing uniforms both work in 3D. Other puzzles rely on the cards themselves, so we had to adapt them. Matching the gardeners becomes a tool chest with matching seals. Recoloring a card header becomes painting the roses in front of you. French sound puzzles have English equivalents.

The fidelity audit records those changes and compares the puzzles with the original. It contains spoilers too.

On a phone, turn it sideways. Walk with the left joystick and look around with the right pad. Interact and Explore sit beside it, and both thumbs can work at once. Phone mode starts with reduced graphics. Your progress saves in the browser.

The automated checks cover puzzle answers, progression, saves, walking routes, animation attachments, construction rules and touch input. They missed the first-release failures above. The repair log records what we found and the checks for each fix. We still haven't tested on a physical phone.

Play After the White Rabbit. The study and walkthrough below contain the solutions.

Solutions from here onward.

The study compares every deduction with the original. The full walkthrough follows it on this page. Jump to the walkthrough or return to the game.

§Study appendix: puzzle by puzzle

The last puzzle made the limits of our first adaptation obvious. Clicking a floor ring moved the camera to an exact view and recorded its number. A separate set of painted strips appeared after construction. You could finish without doing much looking.

The earlier final puzzle, with a floating amber number and an automatic viewpoint.
The earlier final puzzle, with a floating amber number and an automatic viewpoint.

Before. Captured from the earlier repair build. The ring supplied the view and recorded the answer without asking you to read it.

We rebuilt the house in Blender with folded, ribbed surfaces and put the paint directly on those surfaces. Moving a piece now moves its part of the image. You choose a piece and a socket, swap mistakes, turn the bridges over, then adjust your viewing angle and eye height. Nothing records a sightline until you enter the number you read.

Our first attempt at the replacement used flatter folds. The number stayed too readable from an oblique view, so we added depth to break the strokes apart. We also had to give each fold an unpainted underside. Otherwise the opposite color showed through from behind.

The ribs are our own architectural choice. They give the paint changes in depth and let it break apart as you move. We still use fixed sockets, so this does not reproduce the dexterity of balancing paper. The booklet's written sequence has nine items while its numbered diagram has eight; we follow the written sequence.

The original solution's two views of the folded house.
Original perspective diagram, cropped from the supplied solution, PDF page 7. Artwork credited to Space Cowboys.
The rebuilt house with its amber paint aligned by the player.
Our rebuilt surfaces in the browser. The paint belongs to the pieces and obeys the scene's depth.
The painted surfaces viewed before alignment.
The painted surfaces viewed before alignment.

Before adjusting the viewpoint. Dragging the scene or using the two sliders changes where you look from.

These comparisons are grouped by deduction and interaction, so their count differs from the game's completion counter. Each pair shows a crop from the original solution beside a screenshot of our adaptation. Click either image to enlarge it. The PDF links open the source page. Original artwork is credited to Space Cowboys; the browser screenshots come from our recorded playthroughs and repair checks. The house images show the latest rebuild.

Some original tasks run in the companion app. For those, the crop shows the relevant card or instruction in the solution booklet, rather than an app screen. The notes describe where our version changes the task.

01. The storybook

Original · PDF p. 1
The torn story fragments in the solution booklet.
The torn story fragments in the solution booklet.

Order ten memories; read highlighted numbers.

Our 3D adaptation
The story fragments in our book interface.
The story fragments in our book interface.

We kept the chronological deduction in a book on the hall table. Dragging fragments is close to the paper task; the arrow buttons make it possible on a phone. The scene gives the book a place, but this is still mostly a reading puzzle.

02. The sideways letter and its shadow

Original · PDF p. 1
The rotated M and its second shape.
The rotated M and its second shape.

Rotate M; include its shadow.

Our 3D adaptation
The brass letter after rotation, with the second shape behind it.
The brass letter after rotation, with the second shape behind it.

A brass shape rotates in the room, with a second shape behind it. This transfers well because the player changes an object's orientation and looks again. The second shape is deliberately modeled; it is not a physically simulated shadow.

03. The cat's arithmetic

Original · PDF p. 2
The cat and the multiplication clue.
The cat and the multiplication clue.

Continue the unusual multiplication sequence.

Our 3D adaptation
The same sequence on the hall blackboard.
The same sequence on the hall blackboard.

The equations live on a blackboard. The reasoning is almost unchanged. Giving a written puzzle a 3D wall does not add much by itself, so the board stays a short stop within the hall.

04. The missing letter

Original · PDF p. 2
The doors form the missing A.
The doors form the missing A.

Door arrangement completes LICE.

Our 3D adaptation
The repaired doors viewed together across the hall.
The repaired doors viewed together across the hall.

The doors form the visual clue together. Our first frames overlapped and obscured it. Separating the openings mattered more than adding decoration, and examination now includes the whole wall.

05. Growing to the key

Original · PDF p. 2
The scale, table and cake explain how to reach the key.
The scale, table and cake explain how to reach the key.

Compare Alice's height with the table.

Our 3D adaptation
Alice at the high table after eating cake.
Alice at the high table after eating cake.

The high table, height scale and key share a room. Eating raises the camera and makes the key reachable. This replaces card-number arithmetic with a physical reach requirement. Cake can no longer shrink Alice.

06. The triangular lock

Original · PDF p. 2
The triangular key combines with door 6.
The triangular key combines with door 6.

Match the key to door six.

Our 3D adaptation
The matching triangular keyway in the door.
The matching triangular keyway in the door.

You compare the tooth with a brass keyway and turn the key in a hinged door. A previous extra height check blocked a key already in the satchel. Removing that check lets the visible lock explain the interaction.

07. Shrinking through the opening

Original · PDF p. 2
The bottle and height scale show the small-door clearance.
The bottle and height scale show the small-door clearance.

Drink to fit the small door.

Our 3D adaptation
The opening at the smaller player size.
The opening at the smaller player size.

The bottle lowers Alice's size and the player crosses a real opening. We rebuilt its dimensions after the first doorway looked passable at sizes the code refused. The bottle remains reusable.

08. The missing feet

Original · PDF p. 3
The two sides of the card give 89 and 12.
The two sides of the card give 89 and 12.

Subtract the visible measure from 89.

Our 3D adaptation
The missing-feet subtraction inside the passage.
The missing-feet subtraction inside the passage.

The small passage presents the same subtraction. The act of crossing replaces combining cards. The popup now waits for a new entry instead of reopening every frame after being closed.

09. Looking up

Original · PDF p. 3
The booklet tells the player to point the device upward.
The booklet tells the player to point the device upward.

Point the phone toward the ceiling.

Our 3D adaptation
Looking upward inside the passage triggers the move to the woods.
Looking upward inside the passage triggers the move to the woods.

We use the camera's upward look to leave the hall. That preserves the change in attention but replaces the original device gesture. Desktop dragging and the phone look pad both work without motion-sensor permission.

10. The caterpillar's smoke

Original · PDF p. 3
The caterpillar hides two digits in smoke.
The caterpillar hides two digits in smoke.

Find the digits in smoke.

Our 3D adaptation
The smoke rings above our caterpillar.
The smoke rings above our caterpillar.

Smoke rings form the visual clue above the caterpillar. We had to include them in the examination frame and keep the animated body clear of the clock. A good character animation can still ruin a puzzle if it hides the evidence.

11. The solitary mushroom

Original · PDF p. 3
The unique mushroom on a card back.
The unique mushroom on a card back.

Find the unique mushroom; recover scale.

Our 3D adaptation
The spotted mushroom among the trees.
The spotted mushroom among the trees.

A separate mushroom sits among roots, with countable spots. Alice reaches the woods small and regains normal size here. We removed the outdoor speed penalty so this story beat does not make exploration drag.

12. The messenger

Original · PDF p. 4
The frog appointment and its message.
The frog appointment and its message.

Hear the repeated number homophone.

Our 3D adaptation
Our frog with the English dialogue and transcript.
Our frog with the English dialogue and transcript.

The frog uses won/one in English, with optional speech and a transcript. This preserves listening for a repeated sound rather than translating the French dialogue word for word. His mouth and throat now move during speech.

13. The fan and pepper kitchen

Original · PDF p. 4
The fan clears the smoke from the kitchen card.
The fan clears the smoke from the kitchen card.

Find the fan; clear the smoke.

Our 3D adaptation
Using the fan at the open kitchen front.
Using the fan at the open kitchen front.

You collect a fan in the valley and sweep it at the cottage's open front. Card combination becomes a tool action. The first version accepted the side wall as an approach, which broke the scene's logic even though progression worked.

14. Tea with the Hatter

Original · PDF p. 4
The hat distraction and the tea-to-T deduction.
The hat distraction and the tea-to-T deduction.

Ignore the hat; hear tea/T.

Our 3D adaptation
The Hatter, his price tag and the tea table.
The Hatter, his price tag and the tea table.

The tea-to-letter deduction survives in English. The price tag stays visible as a distraction. We moved the spare hat because it concealed the new Hatter's face, while keeping the clue props.

15. The four appointments

Original · PDF p. 4
Four messages above their appointment clocks.
Four messages above their appointment clocks.

Order messages; read minute hands.

Our 3D adaptation
The ordered messages beside our clock faces.
The ordered messages beside our clock faces.

Four visits give messages, and the player orders them beside the clocks. This keeps the cross-reference between text and hands. An early journal entry explained that connection too soon; we removed it.

16. The colored crossing

Original · PDF p. 5
The four traced paths in the solution.
The four traced paths in the solution.

Trace arrows in EXIT color order.

Our 3D adaptation
Tracing one colored route on our sign table.
Tracing one colored route on our sign table.

The mixed signs occupy a table and draw trails only as you trace them. A previous diagram showed the completed shapes before any deduction. The revised table preserves the tracing task but uses a new layout.

17. The three hedge paths

Original · PDF p. 5-6
The hedge map and its three size modifiers.
The hedge map and its three size modifiers.

Apply three operations to Alice's size.

Our 3D adaptation
The three paths and their arithmetic controls.
The three paths and their arithmetic controls.

The same calculations open gates to the garden, tribunal and croquet lawn. Separate paths make the court explorable. We also repaired a wall across its approach; correct gate arithmetic was not enough to make the route work.

18. Recognizing the gardeners

Original · PDF p. 5
The gardeners match symbols on a card back.
The gardeners match symbols on a card back.

Match gardeners with their card-back symbols.

Our 3D adaptation
The numbered gardeners beside the tool chest.
The numbered gardeners beside the tool chest.

Their tunic numbers identify seals on a tool chest. This is a larger translation because there is no deck to search. The chest gives the observation a use, but loses the original familiarity of noticing a previously seen card back.

19. Painting the roses

Original · PDF p. 5
Painting the white card header makes a new combination.
Painting the white card header makes a new combination.

Recolor the white card header.

Our 3D adaptation
The brush applies red paint to the white blooms.
The brush applies red paint to the white blooms.

We turn the card-header trick into painting five roses. The story action fits the garden and gives the player a tool, but it changes the deduction substantially. This is an adaptation choice, not an equivalent mechanism.

20. The stolen tart

Original · PDF p. 6
The crumb outline points to the heart suit.
The crumb outline points to the heart suit.

Recognize the suit in crumbs.

Our 3D adaptation
The cut pastry and its visible crumb trail.
The cut pastry and its visible crumb trail.

The missing pastry wedge and crumbs sit together in the world. We cut the slice from the actual model and moved crumbs out from under it. The player should be able to see the shape without guessing what the artist intended.

21. The guards' disguise

Original · PDF p. 6
The two guards and seven annotated differences.
The two guards and seven annotated differences.

Find seven differences between guards.

Our 3D adaptation
Two cropped examination views, with the guard on duty at left and the disguised guard at right.
Two cropped examination views, with the guard on duty at left and the disguised guard at right.

Two inspectable uniforms retain seven deliberate differences. We removed an accidental eighth change. This puzzle needs a controlled comparison more than character variety; even a decorative change can become false evidence.

22. The Queen's hearing

Original · PDF p. 6
The hearing questions and the Queen in the solution.
The hearing questions and the Queen in the solution.

Answer questions; remember the second one.

Our 3D adaptation
The Queen asks about the evidence gathered earlier.
The Queen asks about the evidence gathered earlier.

The hearing uses evidence gathered at the kitchen, tart and guards. The memory trick remains. The Queen's gestures add performance, while the response choices keep the questions usable on a phone.

23. Croquet

Original · PDF p. 6
The three rebound cards and the flag.
The three rebound cards and the flag.

Find three rebounds and the flag.

Our 3D adaptation
The hedgehog on our lawn after the first rebound.
The hedgehog on our lawn after the first rebound.

An animated hedgehog follows individual shots across a lawn. We test those segments against the visible obstacles. Earlier code accepted the intended answer while the hedgehog crossed a block, so we rebuilt the route and collision checks together.

24. The wooden witnesses

Original · PDF p. 7
Overlapping the figures joins their markings.
Overlapping the figures joins their markings.

Overlap figures; remove their heads.

Our 3D adaptation
The wooden figures overlap with their heads lowered.
The wooden figures overlap with their heads lowered.

Three wooden rewards can be stacked, then their heads lowered. Their permanent paint joins into a number. We removed an unrelated number overlay so the answer comes from the objects the player rearranged.

25. The Queen's decree

Original · PDF p. 7
The decree and the missing-letter cards.
The decree and the missing-letter cards.

Identify three unused letters.

Our 3D adaptation
Our English decree with the letter-marking tool.
Our English decree with the letter-marking tool.

The English decree has the required missing letters and a letter-marking tool. This changes the writing while retaining the letter-coverage puzzle. A literal translation would not preserve the constraint.

26. The kings' construction

Original · PDF p. 7
The numbered folded house and the marked construction pieces.
The numbered folded house and the marked construction pieces.

Fold and arrange the marked pieces.

Our 3D adaptation
The rebuilt bench with physical pieces and selectable sockets.
The rebuilt bench with physical pieces and selectable sockets.

The rebuilt bench lets you select actual painted pieces and choose their sockets, in any order. Occupied slots can be swapped and pieces returned. The two engraved bridges turn over. This keeps a structured placement puzzle while replacing paper balancing with fixed supports.

27. Reading the finished house

Original · PDF p. 7
Both viewing angles and the cat's eye-color clue.
Both viewing angles and the cat's eye-color clue.

Align both sides; order by eye color.

Our 3D adaptation
The amber strokes align across our folded surfaces.
The amber strokes align across our folded surfaces.

The old rings moved the camera to the answer and recorded it. The rebuild puts paint on the folded, ribbed surfaces themselves. You adjust your viewpoint and eye height, then enter your reading. The ribs make our architectural object differ from folded paper; they give its paint depth instead of a floating overlay.

§Getting the recording wrong, twice

The first short video was laggy. It came from a 15 FPS recording made in a software-rendered browser. Exporting it at 30 FPS repeated the existing frames, so the movement still looked choppy.

We recorded again in a native browser at 60 FPS. Resizing the browser panel changed the video's dimensions halfway through a take, so we fixed the capture at 1280 by 720 and let only the preview resize. That solved a recording problem. The next video still looked wrong.

§GPT-6 Astra could finish a recording and miss how bad the walking looked

I asked for something I could share. GPT-6 Astra used the Explore menu's assisted walking, then treated the frame-rate and playback checks as evidence that the video was ready. The camera kept zigzagging. I rejected it.

The route planner only moved between four grid neighbors, alternating short forward and sideways steps. The camera turned toward each step. The game already accepted W and D together, but the agent had chosen assisted travel for filming instead of holding both directions. It also failed to catch the result in its review. Smoothly playing a bad route was still a bad video.

We changed assisted walking to allow diagonals and join visible stretches into straight paths, while checking the route against obstacles. Turning also takes the shorter direction when an angle crosses the wraparound point.

For the replacement at the top of this post, the agent used visible local recording buttons that hold movement keys together through the game's normal input handlers. These are synthetic key events, not a human playthrough. The outdoor takes use those held keys; puzzle shots use the game's interaction controls. Chapter checkpoints set up each take. No camera positions were scripted during the walks.

The replacement keeps the original pace and game sound. We checked the actual movement as well as the video: both keys remain down throughout the diagonal, both position coordinates change together, and the camera holds its direction until the deliberate turn. The old clip stays here because it shows a failure that passing frame-rate checks did not catch.

§Full walkthrough

The 24-minute repair playthrough and every chapter's written solution are here on the same page. The recording documents that earlier repair pass.

§Earlier full repair playthrough

Recorded desktop playthrough with on-screen captions and no audio. Pauses and repeated travel are cut. Open the video.

We started with a fresh save and played every puzzle through the ordinary controls. The run found more bugs, so we repaired them and resumed that saved game after rebuilding. This is an edited demonstration across that repair pass. The kitchen, Hatter and court entrance received further visual fixes after their first appearances in the video. This video predates the visual finale rebuild. Its last chapter shows the earlier interaction. The written steps below cover the current version of the finale.

The recording uses Low graphics and software rendering. It is not a phone performance test or a timed first attempt. The written steps below explain each deduction; read the chapter you need.

On a phone, play sideways. The left stick moves; the right pad turns the camera. Tap Interact to use the object in front of you, or Explore to choose something nearby. Run off switches to Run on for longer walks. In a puzzle, Look at the clue reveals the world and Back to the puzzle returns to the answer. The fan and paintbrush have a Hide instructions button so their panel can leave your view clear.

§Chapter I · Down the rabbit hole

§Put the story back together

Inspect the open book on the low table near the entrance. Use the arrows to put its ten memories in this order:

  1. By the river, you glance at your sister's book TWO times.
  2. A white rabbit interrupts the quiet afternoon.
  3. Nothing about a passing rabbit seems very strange.
  4. Until he cries, "ONE moment late! Oh, my ears!"
  5. He draws a little watch from his waistcoat.
  6. Its hands show THREE o'clock.
  7. You follow him through the brambles.
  8. He disappears into a hole beneath a tree.
  9. After FIVE metres, the tunnel drops straight down.
  10. You land beside a book. Your story is waiting inside.

Read the highlighted number words in that order and enter 2135.

§Turn the brass letter

Inspect the brass M and turn it a quarter turn with Turn left or Turn right. The sideways letter reads as a 3, and its shadow supplies another 3. Choose Read the lock and enter 33. Entering only 3 leaves out the shadow.

§Continue the cat's multiplication table

The blackboard reads 4 × 5 = 12, 4 × 6 = 13, then 4 × 7 = 14. Each step raises the second number and the result by one. Six more steps take the result from 14 to 20 and the second number from 7 to 13. Enter 13.

§Recover the missing letter

Look at the doors together. Their arrangement forms an A. Add it to the plaque's LICE and enter A at The incomplete name. The recovered instruction tells you that eating makes you grow and drinking makes you shrink.

§Reach the key and open door 6

  1. Inspect the cake, listed as The biscuit in Explore. Choose 55 to grow to the highest mark.
  2. Walk to the high table and collect the triangular key. Growing alone does not collect it.
  3. Find door 6, whose lock has a triangular opening. Inspect it and choose Turn the triangular key. The leaf swings open.
  4. Return to the blue bottle and choose 11 to shrink.
  5. Go back to door 6. Walk through the opening, or inspect it and choose Walk through door 6.

The key opens the lock; the bottle gives you enough clearance to pass through. Only size 11 fits this doorway. The cake can only raise your size and the bottle can only lower it. You can revisit both if you choose an intermediate mark by mistake.

The repaired door with its triangular keyway.
The repaired door with its triangular keyway.
Alice at the small opening after shrinking.
Alice at the small opening after shrinking.

§Find your missing measure

After you pass through the doorway, the full measure is 89 and the dial by your head is 12. Subtract 89 − 12 and enter 77. Close the puzzle and tilt the camera upward toward the ceiling opening. That takes you into the woods.

§Chapter II · An afternoon out of time

§The caterpillar and the mushroom

Keep the 3:10 appointment with the caterpillar. The smoke forms two pairs of stacked rings. Enter 88.

The caterpillar examination includes the stacked smoke rings.
The caterpillar examination includes the stacked smoke rings.

Follow his advice to the solitary violet mushroom among the roots west of his grove. Inspect it and choose Taste both sides of the mushroom. This restores your size to 33 and gives you the first message. You arrive in the woods small because this discovery completes the size puzzle.

§The messenger's invitation

At the 4:25 appointment, listen to the frog messenger or open Read the conversation. He repeats "won," which sounds like "one." Enter 1. Keep the message from his invitation.

§Clear the pepper kitchen

Find the blue silk fan near the ruins and the garden entrance. Approach it and inspect it to put it in your satchel. Cross to the pepper cottage for the 5:40 appointment.

With the fan collected, inspect the kitchen. Stand at the open front of the cottage, then press Sweep the fan three times. If you approach a side or back wall, the game walks you around to the entrance. On a phone, the right-hand Interact button also sweeps the fan. Hide instructions clears the middle of the screen while you work.

The smoke clears and the cook gives you the next message. Remember that the cook bakes the Queen's tarts. You will need that evidence later.

The final cook model after clearing the kitchen.
The final cook model after clearing the kitchen.

§Take tea with the Hatter

Go to the 6:00 appointment. The cup accepts one letter. Say "tea" aloud and enter T. The hat's 10/6 price tag is a distraction. Take the last message.

The revised Hatter, with his face and cup visible.
The revised Hatter, with his face and cup visible.

§Read the four clock hands

After all four meetings, inspect The rabbit's lost minutes. Arrange the messages as follows:

OrderMessageAppointmentLong hand points to
1The hours follow each other…3:102
2…but only the numeral on the clock…4:255
3…where the minute hand points…5:408
4…at each appointment, counts.6:000

The sentence tells you to read the numeral under the minute hand at each appointment. On these clocks the top mark is 0. Enter 2580.

The four messages in appointment order.
The four messages in appointment order.

§Trace the Cheshire crossing

At the crossing, select one color and start at its ring. Follow the next arrow of that same color. You can click the signs in the world or use the position buttons in the panel.

The complete button routes are:

ColorRoute through its signsShape
Green · EStart sign → Upper left → Middle left → Middle right → Lower right → Lower left5
Brown · XStart sign → Upper right → Middle right → Middle left → Lower left → Lower right2
Amber · IStart sign → Lower center1
Pink · TStart sign → Upper right → Lower right → Lower left → Start sign0

Pink returns to its starting ring to close the loop. Read the shapes in the colored E X I T order: green, brown, amber, pink. Choose Read the crossing lock and enter 5210.

The completed green trail draws a five.
The completed green trail draws a five.

The Queen's road is open. Follow the path east to her grounds.

§Chapter III · Her Majesty's peculiar justice

§Open the three hedge paths

Inspect The three hedge paths. Each gate applies its operation to your normal size, 33:

PathCalculationAnswer
Garden33 − 429
Tribunal33 × 399
Croquet33 + 1649

Open all three paths. You can visit their activities in any order.

§Help the gardeners

Inspect the three gardeners and read their tunic numbers: 2, 5 and 7. Go to their tool chest, select spade seals ♠ 2, ♠ 5 and ♠ 7, then choose Open the tool chest. This collects the red paint and brush.

Return to the rose bed. Paint all five white blooms red. Walk around the bed as needed and use the brush on each white rose. Interact and Paint nearest white rose both work while the brush is equipped. The remaining count tells you how many are left.

The gardener gives you a wooden effigy when every white bloom is red.

§Examine the tart and both guards

On the tribunal path, inspect the tart. The missing slice and crumbs outline a heart. Choose Hearts ♥ to record the clue.

Compare Guard on duty with Suspicious guard. There are seven differences:

DetailGuard on dutySuspicious guard
Hat bandRedGold
Halberd bladeDeeply notched edgeStraighter edge
MoustacheAbsentPresent
GlovesWhiteGold
Heart emblem colorRedGold
Heart emblem directionUprightInverted
Bottom jacket trimRedGold
The original guard.
The original guard.
The disguised guard, with seven altered details.
The disguised guard, with seven altered details.

Choose I have compared them when you finish. The game needs both the tart observation and the uniform inspection before the hearing.

The tart and heart-shaped trail of crumbs.
The tart and heart-shaped trail of crumbs.

§Answer the Queen

Give these four answers in order:

  1. Who bakes the Queen's tarts? The cook.
  2. Who stole the tarts? The Knave of Hearts.
  3. How many differences expose the disguise? Seven.
  4. What was my second question? Who stole the tarts?

The last answer is the wording of her earlier question. The hearing gives you the knave's wooden effigy.

§Win at croquet

Inspect the croquet lawn. Choose Cushion 4, wait for the hedgehog to stop, then Cushion 7, Cushion 6, and finally Flag 1. The route is 4 → 7 → 6 → 1, or 4761.

Those straight shots avoid the orange obstacles and use exactly three rebounds. If you hit a block, start the sequence again. Choose Collect your prize after reaching the flag to receive your own wooden effigy.

The hedgehog at the flag after three rebounds.
The hedgehog at the flag after three rebounds.

§Join the three wooden figures

Bring all three rewards to The wooden witnesses. Stack the figures from top to bottom by selecting:

  1. You
  2. Knave
  3. Gardener

Choose Lower the heads. With the heads out of the way, the painted fragments join into 54. Choose Read the joined number and enter 54.

The stacked figures after lowering their heads.
The stacked figures after lowering their heads.

§Find the letters missing from the decree

Read the Queen's decree and mark each letter that appears. The three unused letters are R, K and Z. Enter RKZ. Their order does not matter.

The route behind the thrones opens. Follow it north to the impossible house.

§Chapter IV · The house of impossible things

§Build the kings' house

Inspect the kings' instructions. Select a piece from the physical bench or its button, then choose a socket on the model or its position button. You can build in any order. Place the pieces as follows:

PositionPiece
Lower left♠ 6
Lower middle♠ 4
Lower right♥ 6
Lower bridgeBlack bridge, face down
Middle left♥ 4
Middle right♥ 2
Upper bridgeRed bridge, face down
Top left♥ 5
Top right♥ 7

Select each bridge and turn it until its engraved face points down. Selecting a placed piece and choosing an occupied socket swaps the two pieces. Return a selected piece to the bench if needed. Choose Test the structure when the arrangement is ready.

The new construction bench and position controls.
The new construction bench and position controls.

§Find the two viewing angles

Approach the amber floor mark and inspect that side. The camera begins away from the aligned view. Drag across the visible house, or use Move around the house and Raise or lower your eye. Bring the horizontal control near its middle, then lower your eye until the colored strokes join across the folded surfaces.

Read the upper digit before the lower digit. The amber side reads 78. Enter it and choose Record this reading. Merely opening this view does not record an answer.

The amber side aligned by adjusting the viewing position.
The amber side aligned by adjusting the viewing position.

Repeat from the ivory side. Its upper and lower digits read 32. Record 32.

The ivory side after alignment.
The ivory side after alignment.

The cat's left eye is amber. Use that color's reading first. Go to The way home and enter 7832 to finish.

The ending after all 25 puzzles.
The ending after all 25 puzzles.

The recorded ending shows 25 puzzles solved. Its in-game minutes are not a wall-clock benchmark.

]]> pr-review-evidence: a desktop nobody can see https://lucastononro.github.io/pr-review-evidence/ https://lucastononro.github.io/pr-review-evidence/ Wed, 29 Jul 2026 12:00:00 GMT Agents can describe a change but never show it working. So give one its own screen and record it. skill Start with the output, because the output is the argument (you'd have skipped to it anyway). This is a pull request on my own repo, and everything under the Evidence heading was produced and posted by an agent:

PR #12's evidence section: the release-asset links the uploader printed, then the stills rendering inline
PR #12's evidence section: the release-asset links the uploader printed, then the stills rendering inline

And this is the video it attached — 72 seconds, three chapters, served here from the same file GitHub is serving there:

The evidence on PR #12. A real cursor, real keystrokes, a desktop that never existed.

Watch the cursor. It travels, it hovers, the badge renders, a filter chip highlights. That's not a screencast of a browser tab. It's a recording of a whole desktop that came into existence for seventy-two seconds inside a container. It never touched my screen.

§Why this is worth having

A good engineer does two things in a PR: describes the change, and shows it working. Agents have always been fine at the first and structurally incapable of the second, for the dull reason that they have no screen. So the second half arrived as prose — "I verified the modal comes up clean" — and I either took that on faith or opened the browser and repeated the work myself. At that point the claim had bought me nothing.

pr-review-evidence closes that specific gap. It gives an agent a real X11 desktop in a container, lets it drive the thing computer-use style, and records the framebuffer while it works. Out comes evidence.mp4, shots/*.png, and a manifest.json carrying provenance you can check against the repo. Seven bash scripts, 541 lines, and Docker.

The video is the check, not a description of one. Nothing in the loop lets the agent narrate a step it didn't take. click and type print nothing at all, so the only way it learns what happened is to take another screenshot — the same frame a reviewer ends up looking at. The chapter titles aren't authored either; they're read back out of the log of commands actually sent to the desktop. An agent using this can't tell you about a click it never looked at.

docker container — never on your screen the agent on your host xdotool moves the cursor Xvfb :99 1280x720 framebuffer scrot shots/*.png ffmpeg x11grab 15 fps, real time evidence/ mp4 · shots bind mount records everything screenshot → look → move a real cursor → screenshot; the actions themselves print nothing
The only thing that crosses back out of the container is files.

§The loop

One beat of a real session — bring up the scene, mark the chapter, look, act, look again:

evidence-start.sh                      # → live view: http://localhost:6080/vnc.html
evidence-exec.sh launch-browser http://host.docker.internal:3000
evidence-exec.sh record-start          # → recording to raw/desktop-141207.mp4
evidence-exec.sh chapter "2. Archive a task"
evidence-exec.sh screenshot            # → …/shots/shot-141219.png  ← view this
evidence-exec.sh mousemove 1042 318
evidence-exec.sh click
evidence-exec.sh screenshot archived   # → …/shots/archived.png
evidence-finalize.sh                   # → …/evidence.mp4 + manifest.json

screenshot runs scrot -o /evidence/shots/<name>.png inside the container and then prints the host path, because /evidence is a bind mount of the session directory. The agent reads the PNG off its own filesystem with the tool it already has; no image travels over a protocol. Everything else in that list prints nothing at all. mousemove, click and type are silent, so the only way to learn what happened is the next screenshot. That's the behavior I wanted.1The playbook's phrasing is "never act blind". Coordinates from an old frame are stale the moment the layout moves, and the failure mode isn't an error — it's a click that lands somewhere plausible and wrong, on video.

Session state is one file. evidence/.current-session holds an id like fix-archive-modal-20260730T141152Z: the current git branch, slugged and truncated to 40 characters, plus a UTC stamp. Every script resolves it from --session, then $EVIDENCE_SESSION, then that marker. The container is named pre-<session> and holds nothing of value, so evidence-clean.sh removes it and leaves the files behind.

There's a live noVNC view on :6080 unless you start with --no-live, in which case no ports are published at all. You don't need it. But the first time an agent takes over a desktop and starts moving a cursor with intent, you'll want to be watching!!2It's an open-format Agent Skill, not a plugin — ./install.sh --global drops symlinks into ~/.claude/skills, ~/.agents/skills, ~/.codex/skills and ~/.kimi-code/skills, which covers Claude Code, Codex, Kimi Code, Gemini CLI, Cursor, Amp and OpenCode; all of them follow symlinks. Which is exactly why lib.sh resolves its own physical directory with pwd -P before walking up three levels to find the Dockerfile — from the logical path, cd ../../.. lands in a completely different tree.

§Nothing on your screen

The first question everyone asks is whether this thing is going to start moving their cursor while they're using the laptop. It can't. The reason is structural, not a promise.

your machine container — one docker run, torn down after your display :0 keyboard, mouse, monitor never opened, never grabbed the agent runs evidence-exec.sh reads the PNGs off disk evidence/<session>/ shots/ raw/ logs/ evidence.mp4, manifest.json xdotool moves, clicks, types DISPLAY=:99 only Xvfb :99 1280x720x24 framebuffer in RAM — no monitor exists openbox · chromium · xterm ffmpeg x11grab reads :99, 15 fps writes raw/*.mp4 docker exec -e DISPLAY=:99 bind mount -v dir:/evidence no X connection
Two X displays, no route between them. Only the bind mount crosses.

X11 input goes to a display, and xdotool is given exactly one: every command in evidence-exec.sh is docker exec -e DISPLAY=:99. Display :99 is a framebuffer Xvfb allocates in RAM inside the container, with no monitor attached to it. Your session is :0, on the other side of a container boundary. Nothing in the skill ever opens a connection to it: no -v /tmp/.X11-unix, no --net=host, no $DISPLAY passed inward. ffmpeg reads the same :99, so the recording and the input are pointed at the identical fake screen.

What does cross is one bind mount, -v "$dir:/evidence", and it carries files in one direction: the container writes PNGs and mp4s, the agent reads them off your disk. The optional noVNC view on :6080 is the only published port, and with --no-live even that isn't there.3noVNC is a viewer, not a control channel in any meaningful sense — but it is a VNC server on localhost, so --no-live is the correct default for anything you'd rather not have listening. The rest of the session works identically without it; you just can't watch.

§Recording the framebuffer

record-start execs one detached command and keeps its PID at /evidence/.ffmpeg.pid:

ffmpeg -y -loglevel error -f x11grab -video_size 1280x720 -framerate 15 -i :99.0 \
  -c:v libx264 -preset veryfast -crf 23 -pix_fmt yuv420p 'raw/$file'

:99.0 is the display the entrypoint brings up as Xvfb :99 -screen 0 1280x720x24 — which is why the same resolution appears in two files and must not drift. Recording starts on demand rather than in the entrypoint, so an idle session costs nothing.

Grabbing the framebuffer is the whole difference. A browser-automation screencast records a page: no cursor, because there is no cursor, and nothing outside the viewport exists. x11grab records whatever was composited into the screen — the software cursor Xvfb draws, native menus, an xterm, a GTK dialog, and any window you forgot was open. Real time at 15 fps also means the video's clock is the agent's clock, so a pause while it was thinking is a pause on screen.

That generality is the point, and the repo heads off the objection you're forming:

# The base image is used ONLY for its preinstalled Chromium and system
# libraries — nothing here drives Playwright; input is 100% xdotool.
FROM mcr.microsoft.com/playwright:v1.55.0-noble

launch-browser really does just glob /ms-playwright/chromium-*/chrome-linux/chrome and run the binary with --no-sandbox. Then it spends up to ten seconds polling xdotool search --onlyvisible --class chrom to force the window to 1280x720, because --start-maximized is unreliable under openbox and a browser that opens at the wrong size ruins the take.

§Pacing for a reviewer

The best file in the repo is references/recording-playbook.md, and it exists because of a failure I hadn't anticipated: an agent will produce a recording that is entirely correct and unwatchable. Nothing for twelve seconds, then everything at once, cursor teleporting between coordinates.

So the playbook is mostly about time. Aim for 20–90 seconds and 3–7 chapters, because "a reviewer gives you about a minute". Move the mouse toward a target in two or three steps instead of jumping, since the glide is what makes it read as a person. Linger about two seconds on each result so it can actually be read. A sleep 1 on the host after navigation is a natural pause in the video for free. Number the chapters, because reviewers write "chapter 3" in comments. Don't record spinners or deploy waits. Don't ship a first take with a visible stumble at 0:05 — clean and start over.

type is xdotool type --delay 80, slow on purpose so keystrokes are visible; troubleshooting explains how to speed it up to 15 ms and the playbook tells you not to bother for short strings. And the chapter card is zenity --info --no-wrap --timeout 2: a GTK dialog drawn on the desktop, which ends up in the video because everything on the desktop ends up in the video. No overlay, no compositing pass, no burn-in step.

§The manifest

finalize stops the recorder, concatenates the raw segments if record-start ran more than once, probes the duration, warns if it comes back under two seconds4Which catches my favorite bug. Under colima, $HOME is shared with the VM but /tmp isn't, so running a session from an unshared path bind-mounts an empty directory in total silence and every capture stays trapped inside the container. It looks exactly like a broken recorder., and writes the one file a publisher is meant to read. This isn't an illustration — it's the manifest published on PR #12, trimmed only of whitespace:

{
  "schema": "pr-review-evidence/1",
  "session": "task-priorities-20260730T144730Z",
  "created_at": "2026-07-30T14:56:56Z",
  "capture": { "mode": "desktop", "display": "1280x720", "recorder": "ffmpeg x11grab" },
  "video": { "file": "evidence.mp4", "duration_s": 72, "size_bytes": 172715 },
  "source": {
    "repo": "https://github.com/lucastononro/pr-review-evidence.git",
    "branch": "examples/task-priorities",
    "commit": "fb739417404fbb35755afc9afaccc8dffef8c463",
    "pr_url": "https://github.com/lucastononro/pr-review-evidence/pull/12"
  },
  "agent": { "harness": "kimi-code", "model": "kimi-k2" },
  "chapters": ["1. Add a normal task, then a HIGH one", "2. NEW: High priority filter",
               "3. Complete a task, filter Active"]
}

The two halves aren't equally trustworthy. source is read on the host at finalize time — git remote get-url origin, git branch --show-current, git rev-parse HEAD, gh pr view --json url — so anyone can check that commit against the repo. agent is ${EVIDENCE_HARNESS:-unknown} and ${EVIDENCE_MODEL:-unknown}: self-declared, and unknown unless the harness exported them. Kimi Code did, which is why that run names itself. Nothing verified it. Read that field as a claim, not a measurement.

chapters is the nicest part and the least designed. It isn't authored anywhere: finalize greps actions.jsonl for "cmd":"chapter and reads the titles back out of the log of what was actually sent to the desktop. The chapter list can't disagree with what appeared on screen. Same instinct as the rest of the skill — derive the description from the thing that happened, don't ask the agent to narrate it.

§The endpoint that doesn't exist

Having a video on disk isn't the same as having it in the pull request. This is where the project spent the most time being wrong.

GitHub has no API for attachments. None. The drag-and-drop box under a PR comment posts to an internal endpoint that wants browser session cookies; there is no gh api call for it, and there never has been. So an agent holding an authenticated gh and a 2 MB mp4 has no supported way to put the second thing inside the first. In 2026.

The workaround the skill settles on comes from mrshu's gh-pr-image: release assets are the only fully-CLI upload whose URLs render for anyone with repo access. One prerelease per PR, tagged from the PR number:

tag="pr-${pr:-$(date -u +%Y%m%dT%H%M%SZ)}-evidence"
gh release view "$tag" --repo "$repo" >/dev/null 2>&1 \
  || gh release create "$tag" --repo "$repo" --prerelease \
       --title "Evidence ($tag)" --notes "Uploaded by the pr-review-evidence skill."
gh release upload "$tag" "${files[@]}" --repo "$repo" --clobber

Then it prints PR-ready markdown against https://github.com/$repo/releases/download/$tag/ — a bare link for evidence.mp4, <img src=… width="800"> for each still, list items for each logs/*.log. The agent pastes that into the PR body. Access control is the repository's own permissions, which is the entire appeal: nothing to host, nothing to expire, no password to leak, and --clobber makes a re-record idempotent. The cleanup one-liner goes to stderr as an HTML comment so it lands in the PR body but not in the rendered text.5pr-12-evidence is derived from gh pr view --json number, so it only works from a branch with an open PR; outside one it falls back to a UTC timestamp, which uploads fine and is harder to find later. Deleting the PR's evidence is gh release delete pr-12-evidence --cleanup-tag --yes.

The prerelease behind PR #12: five uploaded assets, plus the two source archives GitHub attaches to any tag
The prerelease behind PR #12: five uploaded assets, plus the two source archives GitHub attaches to any tag

That is the whole storage layer — a prerelease nobody will ever install, holding 280 KB of evidence, deleted when the PR is.

I know release assets are the right answer because I shipped the wrong one first. Version 1.0.0 had a Cloudflare Worker with R2 behind it — then Workers KV, once R2 declined to store a single byte without a credit card on file — serving password-gated links. PR #7 deleted the whole thing. The links never expired. The password traveled in the URL — a password, in a URL. Revoking one session meant rotating the admin token for all of them. Publishing is bring-your-own now, and the default has no moving parts.

§Replaying the examples

The repo contains three small apps under examples/ whose only job is to be something to record: demo-webpage (a greeting card and a counter), launch-checklist (tick items, watch a progress bar fill), and task-tracker (a small Vite SPA). Every feature added to them landed as a real PR with evidence attached by the skill, so the examples are readable rather than described — the merged PRs are the documentation.

PR #12, the one at the top of this post, is the one I'd read in full. Its three chapters were read back out of the action log rather than typed: "1. Add a normal task, then a HIGH one", "2. NEW: High priority filter", "3. Complete a task, filter Active". Three deliberately named stills — high-badge.png, filter-high.png, active-filter.png — and the manifest beside them. The body also names the local session, evidence/task-priorities-20260730T144730Z/, which is the branch slug and the UTC stamp doing their job.

One discrepancy in that screenshot makes the case better than I can: the body says "40s" and the manifest says "duration_s": 72. The manifest was derived with ffprobe from the actual file. The "40s" is me, typing an estimate into a text box. Guess which number I'd trust in a review.

That PR also carries a one-line change to vite.config.ts adding allowedHosts, because the dev server refused requests with a host.docker.internal Host header. It's the container seam biting in the most ordinary way possible. I like that it's in the diff rather than in a troubleshooting page.

The older ones are worth a look for a less flattering reason. #1, #5 and #9 were recorded in the browser mode that #10 then deleted — CDP screencasts with no cursor and teleporting actions. Put #5 next to #12 and the argument for grabbing a framebuffer makes itself. One reads as a slideshow of correct states; the other, as somebody using an app.

And not all evidence is visual. #13 added evidence-log.sh, which runs a command on the host and captures stdout, stderr and the exit code into logs/<name>.log:

evidence-log.sh api-health curl -s http://localhost:3000/health
evidence-log.sh e2e-tests npm test

It works inside a visual session, and standalone with no session and no Docker at all — finalize then writes "video": null with "capture": {"mode": "logs"}, and the uploader ships the log files with markdown links beside the stills. Some changes are proved by an exit code, and pretending otherwise would mean recording a video of a terminal.

§What generalizes, and what doesn't

The transferable idea is duller than the desktop, and it's the manifest, not the video: keep host-checkable provenance apart from self-declared fields instead of averaging them into one blob a reader has to trust wholesale. source comes from git and gh on your machine; agent is whatever the harness said about itself. Any tool emitting machine-generated evidence has that same split to make. Most of them quietly don't.

The limits, plainly: a recording shows one path through a UI, and a well-paced one can conceal as much as it proves. Read uncharitably, the playbook's pacing advice is advice on making a demo look good. A recording tells you the modal opened this once, on this commit, for this agent. That's much more than prose and much less than a test suite.

§What's left, and the invitation

Short list, in the order I'd do it:

  • Auto-fill agent.harness and agent.model so the manifest stops saying unknown unless someone remembered two env vars.
  • One place for the display geometry. 1280x720 is currently written in the entrypoint and again in the ffmpeg line, and nothing stops them drifting.
  • Windows and Linux hosts. In theory it just works — it's Docker plus a bind mount — but it's only tested on macOS.

That's the whole roadmap. Better short than invented.

The repo is github.com/lucastononro/pr-review-evidence — MIT, seven bash scripts, one Dockerfile. Feedback and PRs welcome. Or just lift the scripts and use them for something else, if that's easier — nothing in here is clever enough to deserve being a dependency. If you record something with it, put the evidence in the PR. That's the point.

Tests tell you the logic holds; they say nothing about whether the modal opens behind the header.

]]> notify: the agent shouts down the hallway https://lucastononro.github.io/notify/ https://lucastononro.github.io/notify/ Wed, 27 May 2026 12:00:00 GMT Your machine can already talk; the hard part is stopping the agent from reading file paths aloud. skill The problem with an agent that works for eleven minutes is that you don't watch it for eleven minutes. You leave, and then you either come back too early and hover, or too late and it's been waiting six minutes for a yes/no.

Every product answer involves a push service, an API key and a phone. notify is the unglamorous one: say and afplay, both already on the machine. No key, no network, no telemetry, no vendor waiting for a credit card. On macOS the audio is two subprocess.run calls; everything interesting sits upstream.

Installing it is two lines in Claude Code, and the only prerequisite is uv on your PATH:

/plugin marketplace add lucastononro/notify
/plugin install notify@notify-marketplace

Codex is the same shape — codex plugin marketplace add lucastononro/notify, then codex plugin add notify@notify-marketplace. Check it works by asking for the smallest possible thing: play a sound, or ping me when the tests pass.

§Reads fine, sounds awful

A message that reads fine is usually unlistenable, and nothing tells the model a sentence must survive being spoken. So the rules live in the notify docstring, the one place the model definitely reads:

Speech-friendly rules — the model MUST follow these when composing beats:
- Plain English sentences only. No tables, bullets, markdown, code, file
  paths, URLs, long numbers, version strings, or punctuation-heavy text.
- Each beat should be short and natural — read it out loud mentally first.
  Typical beat: one sentence, ~5–20 words. Length follows content.
- Split into multiple beats when content has distinct parts …

Every prohibition maps to a failure mode. Tables get read column by column and arrive as nonsense. Paths get spelled: src/foo.ts becomes "S R C slash foo dot T S". Percentages get read literally, so 87.4% (+2.1%) is a run of digits you can't hold in your head. Hashes and version strings are worse.

The skill pairs each with what you want to hear. Not "Tests failing in src/auth/login.test.ts at line 47" but "Auth tests are failing on the login endpoint." Not "Coverage: 87.4% (+2.1%)" but "Coverage is up two percent." Not "Blocked. Options: 1) Postgres, 2) Mongo, 3) SQLite" but "Blocked. Need you to choose between Postgres and Mongo."1The rules stop short of banning acronyms: API reads acceptably as "A P I", but SQL comes out as "Sequel" — which the skill flags with "Test if unsure."

That bad column is constructed, not collected — but it's what a model writes by default, because by default it writes for a screen. Note what the rewrites give up: the line number, the exact figure, the third option. That's not compression, it's a change of medium. Detail belongs in the chat.

§Beats, not paragraphs

The other half is pacing. One long sentence, spoken, arrives as a wall — you catch the opening clause and miss the verdict. So the tool takes beats, not a string:

for beat in beats:
    text = (beat.get("text") or "").strip()
    if not text:
        continue
    _speak(voice, text)
    spoken += 1
    pause = beat.get("pause_after", 0.3)
    if pause and pause > 0:
        time.sleep(min(float(pause), 5.0))

Each beat is its own say invocation, so the gap is real silence, not a comma the synthesizer may or may not honor. pause_after defaults to 0.3 seconds and is clamped by min(float(pause), 5.0) — a model that can pass a float will eventually pass thirty, and a tool that holds the session for half a minute is worse than silence. Empty beats are skipped, same instinct.

The docstring caps the count at three: "1–3 beats is normal; more than 3 means rewrite shorter." Past three you aren't notifying, you're presenting. The return is a plain sentence — Played 2 beat(s) with voice 'Ava (Premium)' on mac — so the model sees what it did.

§Fourteen sounds, seven categories

Sound names are public API and platform-agnostic. The indirection that buys that: one dict from name to semantic category, a second from category to whatever the local system calls it:

SOUND_CATEGORY = {
    "hero": "success", "glass": "neutral", "sosumi": "blocked", "basso": "error",
    "funk": "warning", "ping": "fyi", "tink": "fyi", "submarine": "attention",
    …
}
LINUX_CANBERRA_EVENT = {
    "success": "complete", "neutral": "bell", "fyi": "message",
    …
    "blocked": "dialog-warning", "error": "dialog-error",
}
sosumi 1 of 14 names blocked 1 of 7 categories macOS afplay /System/Library/Sounds/Sosumi.aiff Linux canberra event: dialog-warning else dialog-warning.oga via paplay Windows %WINDIR%\Media\Windows Exclamation.wav else MessageBeep MB_ICONEXCLAMATION hop 1 MAC_SOUNDS is keyed by name hop 2 nothing resolves a descriptive string, not an exception Could not play sound 'sosumi' …
One name end to end: two hops off macOS, one on it, and a sentence rather than an exception when every candidate misses.

Fourteen names, seven categories. On macOS the name resolves straight to /System/Library/Sounds/Hero.aiff. Everywhere else the category does the work, so an unfamiliar box degrades instead of failing. Windows walks a list of %WINDIR%\Media candidates, then drops to a winsound.MessageBeep flag for when Microsoft has renamed the WAVs again. Linux asks the freedesktop theme for the event, then hunts .oga files under /usr/share/sounds/freedesktop/stereo.

sosumi is the one I care about — blocked, the agent wants a decision. It sounds genuinely unlike basso, which is error. From the next room that difference is the whole product: one means come back, the other means it can wait.

§Voices and guards

Voice selection is a preference ladder: VOICE_PRIORITY runs from "Ava (Premium)" down through Premium and Enhanced voices to plain "Samantha", which ships on every mac. The server runs say -v ? once, takes the first entry installed, and caches it in _voice_cache.2The check is a regex anchored to a locale code — the name followed by \s+[a-z]{2}_[A-Z]{2} — because say -v ? prints a sample sentence beside each voice, and a substring match hits the sample.

Nothing on that path raises. A missing voice, an absent sound file, a Linux box with no paplay — _play returns False, _speak returns quietly, and the tool turns that into a sentence: Could not play sound 'hero' on linux (no audio backend available). A misspelling gets Unknown sound 'heroo'. Valid: basso, blow, bottle, ….

That's deliberate: the caller is a language model, and a descriptive string is something it can act on where an exception is a wall. A crashed attention tool leaves the agent stuck and silent.

§One command, two harnesses

The MCP server is a published package, not a bundled script. Both plugins register it with a byte-identical command:

uvx --from "git+https://github.com/lucastononro/notify#subdirectory=mcp-server" notify-mcp

uvx builds and caches it from git on first run with its single dependency, mcp[cli]>=1.2.0, resolved against the notify-mcp console script in pyproject.toml. No path appears in either config — no plugin-root substitution variable, no venv, nothing to break on a clone elsewhere. The only difference is the JSON around it: Claude Code wants {"mcpServers": {…}}, Codex a bare server map.

The skills diverge more than I expected. Claude Code's SKILL.md calls the tools mcp__notify__notify and declares allowed-tools: … Bash(afplay:*), Bash(say:*), so it carries a macOS-only shell fallback re-implementing the voice ladder in sed and grep. Codex's calls them notify and play_sound, has no allowed-tools, drops the fallback, and keeps its trigger policy in a sibling agents/openai.yaml — allow_implicit_invocation: true plus a declared MCP dependency.3The descriptions diverge too. Codex's packs the whole trigger-and-don't-trigger policy into description; Claude Code's ends in a keyword list — "Triggers on: notify me, ping me, …".

macOS is tested. Windows and Linux are marked best-effort and untested, and the whole platform layer is one 327-line file — so when it's wrong, it's wrong somewhere obvious.

§Three ways I actually use it

Here it is talking, so the rest of this is less abstract. Turn your sound on!! The whole artifact is audio, and a silent screenshot of it would be a blank rectangle:

A throwaway demo: I asked it to explain something at random, purely so there is a recording of the thing speaking.

That's a deliberately pointless message. The three below are what I actually ask for, verbatim.

Waiting on an interval.

loop on this every 15 minutes: check whether the sweep has finished.
stay silent while it's still running. when it's done, notify me with the
best validation score and whether it beat the baseline.

Polling is the thing computers are for, and my attention is the expensive resource. The instruction that earns its place is stay silent while it's still running. Without it you get a chime every fifteen minutes telling you the thing you already knew, and by the third one you've stopped hearing chimes (ask me how I know).

Telling parallel sessions apart. This is the one that changed how I work.

when you're finished, notify me — and name the repo you were working in and
what you actually changed, so I know which window to come back to. don't
just say "done".

With four cloud sessions running, "done" is a useless notification: it tells me something finished, not which tab to open. What I want is "The billing migration is in. Two tests failed on the retry path" — now I know where to go before I've looked at anything. This is where the speech-friendly rules stop being fussy and become the product, because the beat has to name the project in words. A session id read aloud is noise, and four sessions all saying "task complete" is worse than silence.

Being interrupted on purpose. For a long session pointed at one goal, I ask for two different pings with two different sounds:

work through this on your own — I'll be away. play `hero` and notify me
when it's green. if you hit a decision that's mine to make, stop and
notify me with `sosumi`, and say what the decision is in one sentence.

The second half is by far the more valuable. A finished agent costs me a few minutes of latency. A blocked agent that waits silently costs the rest of the session, and there's no upper bound on how long it will sit there being polite.4Naming the sound in the prompt is optional — the skill picks a sensible one from the situation. I name it anyway for the second case, because blocked and broken need to be distinguishable from the next room, and I would rather not trust a judgment call about my own kitchen acoustics.

None of that needed a feature. It's the same two tools with a different sentence in the request, which is about the ratio I'd expect: the tool is trivial, and the discipline is knowing what to ask to be told.

§What the medium demands

Nothing in a model's training tells it the output will be spoken. Its default target is a screen: a table, a path, a figure to three decimals. So everything in notify that isn't two subprocess.run calls is a translation layer for that mismatch, written where the model definitely reads it, not in prose it might skim. That's the transferable part. Move an agent's output somewhere new — a watch line, a Slack push, an email subject, a sound — and you inherit constraints nobody has written down. The tool has to state them, because the model can't notice them from the inside.

The honest scope is narrow. This solves one person's annoyance about hovering over a terminal. The piece I'd actually miss is the smallest one: sosumi against basso, blocked against error, which only means anything once you've already left the room. Take away the beats and the sound table and I'm back to checking every two minutes.

Side effect I didn't expect: I now leave the room on purpose.

]]> tiny-claude-recycler: a subscription is an API key now https://lucastononro.github.io/tiny-claude-recycler/ https://lucastononro.github.io/tiny-claude-recycler/ Thu, 14 May 2026 12:00:00 GMT Anthropic made a Claude subscription programmable. That turns every seat in a company into quota, which is the uncomfortable part. skill TL;DR — tiny-claude-recycler round-robins a pool of Claude OAuth subscription tokens and falls back to a real API key when they're all rate-limited. pip install tiny-claude-recycler, zero runtime dependencies. It exists because claude setup-token turned a $200 subscription into something a script can spend.

The change that made it interesting is one command. claude setup-token mints a one-year OAuth token, you export it as CLAUDE_CODE_OAUTH_TOKEN, and the Agent SDK bills your Pro or Max plan instead of API credits:

Anthropic's own docs on the long-lived token: one year, subscription-billed, meant for CI
Anthropic's own docs on the long-lived token: one year, subscription-billed, meant for CI

Read that as a pricing document rather than a CI convenience and it says something else. Max 20x is $200 a month, flat. Before this, driving a product off inference meant metered API billing. After it, one seat is a fixed-cost quota you can point a script at — and a company with forty seats has forty of them.

§What it does

Round-robin the pool, retry on failure, fall back to the master key:

from tcr import recycler, Secret

recycler.master_key = Secret("sk-ant-api03-...")
recycler.oauth_keys = [Secret("sk-ant-oat01-..."), Secret("sk-ant-oat01-...")]

@recycler.cycle(retries=3)
def ask(prompt):
    return query(prompt=prompt)

The only subtle part is the unsetting. Claude Code's authentication precedence puts ANTHROPIC_AUTH_TOKEN and ANTHROPIC_API_KEY above CLAUDE_CODE_OAUTH_TOKEN, so a stray API key in your environment silently wins and quietly bills your credit card while you believe you're spending a subscription. Every cycle clears both before it swaps.1The other trap is object lifetime. Construct Anthropic() inside the wrapped function — a long-lived client built before the decorator ran already captured whatever key was set at construction and never sees a swap. Secret redacts itself in repr, so tokens stay out of tracebacks.

§The part I'd rather say out loud

Rotation isn't clever. Rotation is what you write on the afternoon you notice that N seats is N × $200 of quota with no metering attached, and that nothing in the stack checks whose seat it is. The tool doesn't ask, either — it takes a list of strings.

Which is where I'll stop, because there are two very different versions of this. Pooling tokens you own — your Max seat, your CI, your side project — is housekeeping, and that's what I built it for. Pooling your colleagues' seats to run your product is spending someone else's subscription under terms they agreed to and you didn't read, and the fact that it works is not the same as the fact that you may. The mechanism is genuinely interesting. The version of it I keep hearing pitched is a billing exploit with a rotation library in front.

Anthropic shipped a convenience for CI and accidentally shipped a price list; the library is just me being tidy about it.

]]> paper-videos: a paper in, an explainer out https://lucastononro.github.io/paper-videos/ https://lucastononro.github.io/paper-videos/ Sat, 09 May 2026 12:00:00 GMT The explainer nobody has the weeks to animate, assembled beat by beat while you watch. agents Start with the output. The first of these came from an arXiv id, the second from a sentence with no paper behind it. Both came out of paper-videos, which fills in an editor at localhost:5173 while you watch.

“MELT: Letting AI Think Longer Without Bloated Memory” — a paper, explained. Watch on YouTube ↗
“Évariste Galois — Life and Theory” — the same pipeline, no PDF involved. Watch on YouTube ↗

§The part that doesn't get made

Reading the MELT paper is not the hard part. It's a clear paper about a real trick — loop the layers, keep one KV cache, stop paying for extra thinking in memory. Plenty of people could read it over a coffee and tell you accurately what it does.

The second craft is not so widely held. Scripting twelve minutes so the first fifteen seconds earn the rest. Deciding which two contributions survive. Narrating it so it sounds like someone who understands it. Animating an equation so the eye lands on the right term as the voice names it. Pacing the thing so attention holds. That's a different profession, and a slow one: a single 3Blue1Brown video is weeks of animation work, and the people who do it that well are countable. Not a complaint about them — it's why the skill is scarce in a way that reading the paper is not.

So the explanation mostly doesn't get made. The knowledge stays in the PDF, read by the few hundred people who were going to read it anyway. The reader who needed twelve good minutes never gets them.

What this pipeline automates is the production, not the understanding. The brief, the beat-by-beat script, the ElevenLabs narration, the Manim scenes, the Remotion assembly — machine work now. The judgment is not: what to explain, what to cut, whether the analogy is honest, whether the voice is bearable. Taste can't be handed over, so the doctrine stops the producer dead after three narrated beats and makes you listen. The video-qa agent exists because the pipeline isn't trusted to grade its own output. It's alpha, and the boundary cuts both ways: a bad script now yields a well-produced bad video faster than it used to yield a bad one. Cheap production doesn't make the decisions cheap. It stops them being buried under twelve hours of keyframing.

The editor mid-Galois: parent chat left, the player above a filmstrip and the visual-blocks / voice-beats lanes, a time crop selected at 10:51→11:44, QA clean across 208 beats.
The editor mid-Galois: parent chat left, the player above a filmstrip and the visual-blocks / voice-beats lanes, a time crop selected at 10:51→11:44, QA clean across 208 beats.

§The pipeline

arXiv id / PDF or just a topic critic plans the brief storyteller script.md, in beats producer narrates each beat visualizer Manim per beat output.mp4 Remotion live-syncs after every beat — the editor fills in as it goes Claude Code is the orchestrator; each box is a subagent
The orchestrator is Claude Code itself, running inside the repo. Deterministic work lives in `src/tools/` as typed CLIs; the agents only do the parts that need judgment.

That split is most of the design. narrate.ts, render-manim.ts, qa.ts and render-remotion.ts are ordinary commander CLIs — arguments in, JSON on stdout, exit codes that mean something. Anything requiring taste is a subagent: a critic plans the brief, a storyteller writes the beats, a producer narrates, a visualizer animates. Six specialists, no message bus, coordination written down as files — "each subagent has its own context window — the brief / script / manifest in videos/<slug>/ are the persistent contract between them."

Rule 2 keeps that honest: LaTeX comes only from equations.json, never from memory of the paper. In topic mode there is no paper — the Galois video never touched a PDF — so the critic fills that same file from its own research.

§The micro-beat doctrine

The part I'd defend in a fight. A beat is one visual moment and one narration clip: 8–40 words, two to ten seconds, hard cap 300 characters. So a twelve-minute video is 100–160 beats, not thirty segments. As data, a beat is three lines of markdown:

### beat-002
[VISUAL: continue]
"On a thirty-two thousand token generation, Ouro burns about twenty-eight gigabytes of VRAM."

### beat-004
[PAUSE 0.8s]
(silent)

[VISUAL: continue] earns its keep. The parser inherits the previous cue, and the migrator coalesces adjacent identical visuals into one visualBlock with one fade in and one fade out. Re-emit the cue instead and you get two blocks, each fading through the background — a flash to navy and back where nothing changed.

Downstream that becomes two arrays rather than one: voice[], one-to-one with the mp3s, and visualBlocks[], which are spans. MELT has 90 voice beats to 59 blocks. Its opening animation runs 588 frames beneath the first three, plays once, then holds its final frame — which is why scenes must end on a held tableau, never a FadeOut. Held black is indistinguishable from a broken render.

Reason three is the one that actually pays: "Re-rendering one bad beat is cheap; re-rendering a 30-second monologue is not." When beat 87 is wrong, you fix beat 87.

§The padding arithmetic

narrate.ts hands each mp3 to ffmpeg (adelay + apad) with two constants:

const DEFAULT_PAD_LEADING_SEC = 0.25;
const DEFAULT_PAD_TRAILING_SEC = 0.9;

ElevenLabs' character-level alignment is collapsed into words and shifted by padLeading, so captions stay glued to speech that just moved. audioDurationSeconds is the last word's end plus the trailing pad. The segment builder then adds one more slice, secondsToFrames(ts.audioDurationSeconds + 0.2, fps), commented "add 200ms tail so visual settles after voice ends".

Worked through, on beat-001 of the MELT video: the last word ends at 6.015s, audioDurationSeconds is 6.915, durationFrames is ceil(7.115 × 30) = 214, and beat-002's first word starts at exactly 0.250s. Speech-end to speech-start: 1.368s. Segments butt straight up against each other in the manifest — every millisecond of that silence lives inside the audio files.1The repo disagrees with itself here — 0.6s appears in the prose section of CLAUDE.md, in producer.md and in storyteller.md; 0.9s appears in hard rule 4 and in the code. The mp3s on disk say 0.9.

≈1.35s of silence = 0.9 + 0.2 + 0.25 narration mp3 beat-001 · 8–40 words narration mp3 beat-002 · first word 0.25s 0.25s lead 0.9s trail 0.2s segment tail 0.25s lead same pads, every beat narrate.ts pads the mp3 with ffmpeg; every word timestamp shifts +0.25s so captions stay in sync durationFrames = ceil((last word end + 0.9 + 0.2) × 30) — beat-001 lands on 214 frames
Two consecutive beats: the gap is three defaults stacked, not a gap.

Which is why the doctrine preempts the obvious mistake: "Pauses between beats come for free via the audio-padding logic — do NOT also insert [PAUSE 0.2s] beats." [PAUSE Xs] is for deliberately long silences, 0.6s and up.

§Cue-level dispatch

Some beats want photographic motion rather than math. The storyteller signals that inside the cue's own description, with a clip: prefix (veo: is a legacy alias, treated identically):

[MANIM: teaser_markdown_accreting description="clip: cinematic 8 seconds, slow
push-in on a single glowing markdown file titled 'best_skill.md' … 8s, 16:9."]

The visualizer expands that hint into a full cinematographic prompt and walks the chain — ElevenLabs Studio, then Veo, then Manim:

npm run elevenlabs-video -- "<prompt>" -m kling-2.6 --aspect 16:9 \
  --duration 8 --no-audio --out videos/<slug>/manim/beat-NNN.mp4
# exit code 2 = 401/403/404 from the Studio endpoint → private-beta gate, not a failure
npm run veo -- "<same prompt>" --model veo-3.1-generate-preview --aspect 16:9 …

Exit code 2 is reserved for 401/403/404, so "you don't have beta access" is machine-distinguishable from "the render failed". Any other non-zero exit means Manim, always reachable.

The dispatch lives in the cue rather than in an agent's head for a boring reason: the mp4 lands at manim/beat-NNN.mp4 whichever provider produced it, and the composition renders it as a manimClip either way. The winner is recorded in a .elevenlabs.json or .veo.json sidecar carrying the full prompt, so a re-render takes the same path. The script never has to know which key you hold.

§Spot edits

The manifest re-syncs after every narration and every render, and the teaser comes first, so you judge the voice within seconds rather than at minute twelve. A watcher notices each write and remounts the player.2The watcher is capped at depth: 3 and ignores media/, paper-md-assets/ and manim-last-frames/, because watching the whole tree opened 2000+ kqueue FDs on macOS, exhausted the process pool, and made the editor's later spawn('claude') fail with EBADF. The symptom was "chat turn failed: spawn EBADF" and a dead server. So you can select a range on a video that doesn't exist yet:

Dragging horizontally on the filmstrip selects a time crop — `1:28→1:36 · 7.8s` — and an `↗ Spot-edit` pill appears beside it. The lanes below show what falls inside.
Dragging horizontally on the filmstrip selects a time crop — `1:28→1:36 · 7.8s` — and an `↗ Spot-edit` pill appears beside it. The lanes below show what falls inside.

Click it and you get a pin-point conversation about that range rather than about the video — a forked claude --resume session docked on the right, re-narrating and re-rendering while the parent chat carries on:

A thread scoped to `0:51→0:56`, running its own `render-manim` and `sync-manifest` loop while the parent chat reports a separate composition-layer fix.
A thread scoped to `0:51→0:56`, running its own `render-manim` and `sync-manifest` loop while the parent chat reports a separate composition-layer fix.

The thread gets the crop and a finding aid, not a task list — "Stay inside the time crop. If a fix genuinely requires editing material outside it, stop and explain rather than silently widening scope." It closes itself by ending a turn with one line beginning SUMMARY:, which the server posts into the parent chat. Spawn several and each becomes a tab!!

§Gates

The first gate is afplay on beats 001 to 003: the producer stops and makes you listen before spending anything else. Fail fast, in the voice you'll hear for the next twelve minutes. The remedies are script-side — bad pronunciation is a phrasing problem, and pacing gets fixed with commas and em-dashes, not voice settings.

Then npm run qa -- <slug>, which exits 2 on anything error-severity: audio overlaps, silent gaps over 1.5s, missing mp3s or mp4s, unknown equation ids, bboxes off the page, adjacent same-content blocks — the flicker regression above, caught mechanically. My favorite check (having a favorite QA check is its own diagnosis) re-runs the LaTeX splitter over every equation and errors on any fragment with unbalanced \begin/\end. A regex splitter tears the \\ row separators inside a pmatrix, and KaTeX renders the wreckage as red error text mid-frame while the narration carries on, unbothered. Underneath sits the least glamorous requirement in the repo: scenes must use MathTex, never Text("vₜ"). So install.sh puts TinyTeX in user space — without it, as the installer says, "Manim equations will fall back to broken Unicode."

The gallery keeps what didn't survive alongside what did:

The home gallery: one card per folder in `videos/`, with duration and beat / block counts. Some carry a gold `✓ rendered` pill; the rest are drafts at `0 beats · 0 blocks`, several still titled `(unknown — will be filled by paper-extractor)`.
The home gallery: one card per folder in `videos/`, with duration and beat / block counts. Some carry a gold `✓ rendered` pill; the rest are drafts at `0 beats · 0 blocks`, several still titled `(unknown — will be filled by paper-extractor)`.

§Running it yourself

Two paid accounts and one script. You need a Claude Code login (Pro/Max or API credits — it's the orchestrator and the chat backend) and an ElevenLabs key for narration; the free tier covers short demos. Everything else is handled:

git clone https://github.com/lucastononro/paper-videos && cd paper-videos
./install.sh             # idempotent — re-run any time
claude /login            # once; the claude CLI itself comes from docs.claude.com/claude-code
$EDITOR .env             # paste ELEVENLABS_API_KEY
./run.sh                 # editor at localhost:5173

install.sh checks Node 20+, installs uv and ffmpeg if missing, runs npm install, uv sync (Manim + Marker + PyMuPDF) and npx remotion browser ensure, and puts TinyTeX in user space — the step you'll be tempted to skip with --quick, right up until your first equation renders as yellow boxes. Then open the gallery, click + New video, give it an arXiv id or just a topic, and watch the timeline fill in. The only secret in the whole setup is the ElevenLabs key in a gitignored .env; Claude Code brings its own OAuth session and arXiv needs nothing.

§The unit is the design

The beat is not a formatting convention. It's the decision that sets the cost of being wrong, and everything else here is downstream of it. A pipeline whose smallest replaceable piece is a thirty-second monologue has to be right early, so the effort migrates into planning. Make the smallest piece four seconds of audio and one animation, and being wrong forty times is an ordinary afternoon. The partial manifest can truncate at the first unfinished beat only because beats don't depend on each other. A thread can be confined to a crop because beat 87 is something you can name and re-run.

That generalizes past video. The useful question for a generative pipeline is not how good its output is on a good day. It's how much you must discard to change one thing you dislike. Small units aren't free: they bought me [VISUAL: continue], block coalescing and three stacked padding constants, all of which exist to hide seams a monologue would never have had.

The machine will make the video in an afternoon now. Deciding it was worth making is still the whole job.

]]> pr-brief: a reviewer for PRs nobody wants to read https://lucastononro.github.io/pr-brief/ https://lucastononro.github.io/pr-brief/ Sat, 25 Apr 2026 12:00:00 GMT It groups a huge PR into features, picks the reading order, and narrates each diff from the side. skill Three honest reasons pr-brief exists. I had to review enormous PRs and couldn't ask everyone to stack them. The needles in that haystack aren't evenly distributed, so I wanted to know the right order to look. And reading code isn't the hard part — connecting a migration in one folder to a cache read in another is. That's the part you skip at file 140.

“PR Brief skill” — the whole loop. Watch on YouTube ↗

§Grouping before reading

Ask Claude to "review this pr" and one opus agent gets the file list, the diff and the PR body, and returns a features.json. A feature is a kebab-case change theme of one to three words — ownership-guards, asgi-middleware. Grouped by theme, not directory. A file carrying two themes joins them with a +: sse-stability + inbox-ux.

The ordering heuristic:

- Foundation first: migrations, schemas, core modules, shared utilities, types
- Domain/services next: business logic, repositories
- Entry points: controllers, API endpoints, middleware
- UI: components, stores, pages
- Last: docs, tests, build/ci
- Within each layer: higher blast_radius first

blast_radius (high for schema/auth/migrations, low for docs/tests/lockfiles) is only the tiebreak inside a layer. Sorting the whole PR by blast radius is the obvious move, and it's wrong: it puts the scary auth guard ahead of the migration that defines the column being guarded. Dependency order first, fear second. The agent also emits five to twelve edges, pairs like ["migrations","ai-enabled-flag"] meaning "understanding A helps read B".1SKILL.md mandates opus for every subagent: it "makes fewer mistakes on diff analysis and feature grouping, which is load-bearing". Past 200 files it falls back to top-level directories.

§Two kinds of pill

Explain pills are the purple ✨ callouts in the 400px track right of each diff. Each must do three of four jobs: what the block does, why this approach, where it connects (a specific call site by path:symbol), and how it fits the feature. Job three hurts to write:

{
  "title": "Pure ASGI to unblock SSE",
  "body": "Switched from Starlette's `BaseHTTPMiddleware` to raw ASGI because the former buffers `StreamingResponse` bodies through an asyncio queue.\n\n**Why now:** `inbox_service.stream_conversations` (next file) emits SSE — the buffer would have collapsed all events into a single chunk, which is the bug `sse-stability` fixes."
}

It names the approach that failed, the file you'll open next, and the feature it belongs to. None of that comes from reading one file. That's why step 6a exists — nothing but "walk the sequence, find the seam where feature A's output becomes feature B's input" before any pill is written.

§Severity that means something

Critique pills share that track in red, with an S1–S5 badge. As written:

  • S1 (nit) — style, naming, minor readability.
  • S2 (suggestion) — a cleaner approach exists, not a bug.
  • S3 (issue) — real concern: subtle bug, missing edge case, technical debt that will bite.
  • S4 (problem) — a bug, security/perf concern, broken contract. Should block until addressed.
  • S5 (blocker) — must-fix. Production-impacting bug, data-loss risk, security hole.

The definitions are the easy half. The distribution guidance is what stops the scale collapsing: a typical 50-file PR "might have 0 S5s, 1–3 S4s, 5–10 S3s, more S1–S2s", and a genuinely fine file gets zero critiques, not a forced S1. If everything is S5, nothing is. Vague critiques are banned — name the cleanup or skip the pill.

{
  "title": "Race in cache invalidation",
  "body": "**`session_management.py`** clears the cache *before* `db.commit()`. A concurrent reader between the clear and the commit will repopulate stale data from the old transaction snapshot, defeating the invalidation.\n\nMove the `cache.delete()` call after the commit, or use a write-through cache.",
  "severity": 4,
  "start_line": 142,
  "end_line": 147,
  "side": "RIGHT",
  "suggested_change": "        db.commit()\n        cache.delete(f\"ai:{phone_group_key}\")"
}

The good part is the mechanism — "repopulate stale data from the old transaction snapshot" — not the word "race". suggested_change is optional, and it's what earns the click. "Use as comment →" pre-fills the inline editor with the critique and wraps the replacement in a GitHub ```suggestion block, so the author gets the one-click accept UI.2Saves POST immediately, throttled server-side to one write per 1.5s (GitHub asks for ≥1s). On a secondary rate limit the server returns 429 and the UI flips itself into Batch mode, posting everything as one review.

§Positions before the browser

The build step flattens each file's diff into line objects carrying their GitHub position:

one file's diff, from the first @@ three coordinates, one physical line old_line new_line position @@ -1,5 +1,10 @@ hunk — — 1 foo context 1 1 2 +bar add — 2 3 -baz del 2 — 4 qux context 3 3 5 +quux add — 4 6 counts the @@ row too resets each file the pill author emits start_line 2 · end_line 4 · side RIGHT post-change numbers only RIGHT matches new_line LEFT matches old_line resolved at build time unmatched anchors dropped
The holes are the point: an add row has no old_line, a del row has no new_line, and only position counts the @@ row.
{ "position": 1, "type": "hunk", "content": "@@ -1,5 +1,10 @@" }
{ "position": 2, "type": "context", "content": " foo", "old_line": 1, "new_line": 1 }
{ "position": 3, "type": "add", "content": "+bar", "new_line": 2 }
{ "position": 4, "type": "del", "content": "-baz", "old_line": 2 }

position is not a line number. It's a 1-indexed offset into that file's unified diff, counting from the first @@ header. Hunk headers, context, adds and deletes all increment it. It resets per file, and \ No newline at end of file markers are skipped. So one physical line has three coordinates — old_line, new_line, position — and the pill author only ever sees the second. (I know.) Pills are written as start_line/end_line/side, and the build step resolves them — for RIGHT, the context-or-add line whose new_line matches; LEFT uses old_line — dropping any anchor missing from the diff with a warning.

Resolving that in the browser means shipping a diff parser and hiding an off-by-one in the one layer with no tests — and the failure isn't cosmetic: GitHub rejects the comment.

Then the twist: positions never reach GitHub for the comments you write. /api/post-comment sends line and side (plus start_line/start_side for a range), the newer shape that supports multi-line ranges. position is the app's internal coordinate system: data-position per row, data-start-position per pill, end_position as the local key for a pending comment. The only one crossing the wire is /api/post-briefs, pinning each file's brief at position: 1. One consequence: files are stacked, so rows restart at position 1 in each. A global querySelector opens the editor in the first match in document order — a file far up the page, invisible to whoever clicked. Lookups are scoped to .diff-wrap.3SKILL.md cites a build_data.py that isn't in the repo; step 4 has Claude write the stdlib parser inline each run.

§Chat that already knows

It starts as a context menu. Select any code inside a diff, right-click, and the only item is the one you want:

Selecting code in a diff and right-clicking surfaces a "Chat about `path:lines`" item — the whole affordance
Selecting code in a diff and right-clicking surfaces a "Chat about `path:lines`" item — the whole affordance
Click it and the drawer opens on the right, pre-loaded with the file, the line range and the selected code, streaming an answer
Click it and the drawer opens on the right, pre-loaded with the file, the line range and the selected code, streaming an answer

A still of a streaming panel is a poor likeness of one, so here's the demo from the repo:

Select, right-click, ask, read. The panel already knows what PR this is.

Select code, right-click, and a 440px drawer opens. Every panel — across files, across reloads — resumes the same Claude Code session that wrote the brief, via claude --output-format=stream-json --verbose --dangerously-skip-permissions --resume <id> -p <text>, streamed to the browser as newline-delimited JSON. So it starts with the whole PR in context. No re-seeding, no vector store.

Getting the id is fiddly. Just before launch the skill takes the most recently modified *.jsonl under ~/.claude/projects/<cwd-with-slashes-turned-into-dashes>/ and writes {session_id, cwd} to ~/.claude/pr-review/pr-<num>/session.json. The cwd matters as much as the id: --resume looks the id up scoped to the directory it runs in, so resuming from the wrong one gives "No conversation found" instead of a useful error.

The snippet travels only on a panel's first message; after that firstSent flips and follow-ups go verbatim. Every panel is one conversation, so panel three can say "compare this with the previous file" and mean it. One turn at a time: the server takes a non-blocking lock and answers 409 if you overlap. Stop SIGTERMs the subprocess, and -SIGTERM is deliberately not counted as a failure. If no id was captured, /api/chat-context reports enabled: false, the context menu is never wired, and the feature doesn't exist instead of half-working.

524 lines of SKILL.md, one 2241-line HTML file, a 511-line stdlib server, no node_modules.

§Ordering without the model

The two halves of this come apart, and only one needs a model. Being told that nine files are one idea called sse-stability, that the migration comes before the guard reading the column it adds, that understanding A helps with B — that's a sorting problem over a file list, and it removes most of the work before a single pill is written. I think you could delete the explain and critique tracks and it would still be worth running. The reverse is not true: pills hung on an arbitrary file order are annotations on a haystack.

The severity scale generalizes further. S1–S5 definitions are the free part; any tool can write them. What stops the scale being decoration is the distribution guidance — a fine file gets zero critiques, a 50-file PR shouldn't yield a dozen S4s. Without a stated expectation of how rare the top of the scale is, nothing stops a model spending it, and by the second file I stop reading the badge. That holds for any generated output carrying a number it assigned itself.

The pills are the nice part. The ordering is the useful part.

]]> ccli-prompt: cmd+k for your terminal https://lucastononro.github.io/ccli-prompt/ https://lucastononro.github.io/ccli-prompt/ Mon, 20 Apr 2026 12:00:00 GMT Press Cmd+K, describe the command you've forgotten, and read what comes back before running it. skill I don't forget commands. I forget flags. find with a size predicate, tar with the right four letters, whatever incantation evicts whatever is squatting on port 9090. Cursor fixed this inside the editor with Cmd+K, and I wanted the same key to do the same thing one pane over.

ccli-prompt binds a zsh widget to ^[k. Press Cmd+K, type "find files bigger than 100mb" into a minibuffer, and a command appears on your command line!! Nothing runs. It sits in $BUFFER until you read it and press Enter yourself. (Esc then K works too — the widget quietly raises KEYTIMEOUT to 100 so the chord doesn't expire in zsh's default 0.4 s.)

Asking for a command, getting a command, running nothing.
Cmd+K or Esc then K zle widget reads your ask unix socket 0600, your user daemon warm HTTPS /v1/messages haiku 4.5 $BUFFER the command, unrun you press Enter the step that runs the daemon outlives the keystroke — one warm connection, thirty idle minutes errors come back as # … and print above the prompt everything above only types — pressing Enter is the only thing that runs it
Cmd+K in, a command in $BUFFER out — and nothing runs until the last box.

§The prompt is the product

pipx install ccli-prompt, then ccli-prompt install. The widget is copied to ~/.ccli-prompt/, one source line is appended to your .zshrc, and a two-step wizard runs — auth, then a model list pulled live from /v1/models with Haiku 4.5 sorted to the top. That leaves the part that actually decides whether any of this is useful, ~/.ccli-prompt/prompt.md:

- Output ONLY the command. No prose, no explanation, no markdown, no code fences, no comments.
- Match the user's shell and OS (macOS `zsh` unless told otherwise — prefer BSD-flavored flags…)
- If the user's request cannot be fulfilled with a shell command, output a single line beginning with `# `…
- Never output destructive commands (`rm -rf /`, `dd`, disk/partition formatting, force-push to main…)
…
- Most requests ("kill port 9090", "tar this folder", "find files bigger than X") are obvious — answer immediately…
- If the request depends on specifics you don't know … use an ALL-CAPS placeholder like `<FILENAME>`, `<PORT>`…

Every rule there is a scar. No code fences, because a model that helpfully wraps its answer in backticks has just put three backticks on your command line.1The widget doesn't trust it either. A sed pass deletes fence lines from whatever comes back, then trims surrounding whitespace, before anything reaches $BUFFER. BSD-flavored flags, because half the internet's find invocations are GNU and fail differently on a Mac. Do not deliberate, because at a prompt, thinking is latency you can feel. And the placeholder rule is the one that keeps it usable: the model isn't allowed to ask a clarifying question, so it commits to a shape and leaves you a hole to fill.

§Three requests

These are illustrative, not recorded, but each is what those rules demand. "Kill whatever is on port 9090" is the case the prompt names as obvious, so it arrives as one line with nothing around it: lsof -ti tcp:9090 | xargs kill -9. Ask the same thing without naming a port and you get lsof -ti tcp:<PORT> | xargs kill -9 — a placeholder, not a question. Ask "which test file covers the auth flow" and you get # need more context: the test files in this repo, because the daemon has no tools and can't read your disk.

The context it does get matters more than I expected. The widget sends your working directory, ${SHELL:t}, uname -sr, and — the interesting one — whatever was already on your command line, as a draft field. Type tar -c, press Cmd+K, ask to compress a folder, and the last rule in the prompt tells the model to finish your line instead of starting its own.

§A daemon, not a subprocess

Shelling out to claude -p per keystroke means paying for a CLI boot every time you can't remember a flag. So the widget talks to a stdlib-only Python daemon over a Unix socket, spawning it on first use:

local req=$'cwd\t'"$PWD"$'\nshell\t'"${SHELL:t}"$'\nos\t'"$(uname -sr)"…
output=$(printf '%s' "$req" | nc -U "$CCLI_SOCKET" 2>/dev/null)

Tab-separated lines and nc -U. No HTTP, no JSON, no framing to parse. The socket is /tmp/ccli-$USER.sock, chmoded to 0600 immediately after start_unix_server returns. That matters: anything that can write to that socket can spend your tokens and see your $PWD, so on a shared box, mode 0644 would be an open proxy with your subscription attached. The daemon keeps one HTTPSConnection to api.anthropic.com open with a 30 s timeout, resets and retries once if it finds the connection dead, and exits after CCLI_IDLE_TIMEOUT — 1800 seconds of nobody asking it anything. The README's ~500 ms warm figure (yes, I measured) is TLS you already paid for plus Haiku's first tokens. The cold path adds a nohup spawn and a poll loop that waits forty 50 ms ticks before giving up.2The system prompt is sent with cache_control: {"type": "ephemeral"}, which on Haiku 4.5 achieves nothing — it's under 500 tokens, far below that model's 4096-token minimum cacheable prefix. Harmless, aspirational.

§Auth without an API key

My favorite part, because it asks you for nothing. With no $ANTHROPIC_API_KEY set, the daemon runs security find-generic-password -s "Claude Code-credentials" -w, lifts claudeAiOauth.accessToken out of the JSON Claude Code already stored there, and picks its headers off the token's prefix:

if is_oauth_token(self._token):
    headers["Authorization"] = f"Bearer {self._token}"
    headers["anthropic-beta"] = "oauth-2025-04-20"
else:
    headers["x-api-key"] = self._token

sk-ant-oat… rides Authorization with the oauth beta header; sk-ant-api03… rides x-api-key. Identical code path otherwise. A 401 reloads the token once and retries, on the assumption that it rotated rather than that you've been thrown out.3Linux never touches the Keychain — it reads the same claudeAiOauth.accessToken from ~/.claude/.credentials.json, which is also the macOS fallback.

§Hash-space

The detail I'd keep if I had to throw the rest away. Daemon failures come back as strings beginning # , and the widget tests for that prefix before it goes anywhere near your command line:

if [[ "$output" == \#\ * ]]; then
  print -P "%F{red}$output%f"
  BUFFER="$saved_buffer"
  CURSOR=${#BUFFER}
  zle reset-prompt
  return 1
fi

Three things happen there. The message prints above the prompt instead of landing in the buffer — the widget has already called zle -I, so it can write outside its own line. The half-typed draft you had before pressing Cmd+K is restored verbatim, so a failed guess costs you nothing you'd typed. And # is also a shell comment, so the worst case — a convention I got wrong somewhere — is a line that does nothing when executed: # api error 401: … is not a command you can accidentally run. The system prompt reuses the same prefix for I can't do that in a shell, so model refusals and HTTP failures leave by the same door.

§Nothing runs, on purpose

The design commitment worth stealing is the one that sounds like a limitation. The daemon has no tools, can't read the disk, and hands its answer to $BUFFER — a human presses Enter. That one decision pays for most of what I never had to build: no sandbox, no confirmation dialog, no audit of what the model may touch, no rollback. The # prefix is only viable because of it. A comment is a safe failure mode when the worst case is a line you read and delete, not an action already taken.

It also settles where the human sits. Reviewing generated shell is a chore when it means opening something to review it. Here the review is the keypress you were going to make anyway, so it costs nothing and can't be skipped. The price shows up in the "which test file covers the auth flow" case — no tools, no answer. Before adding autonomy to anything like this, I'd now ask where the workflow already puts a person. That seat is cheaper than one I'd have to build.

Zero runtime dependencies, which felt right for something whose entire job is to type.

]]> trainable: the data was never the bottleneck https://lucastononro.github.io/trainable/ https://lucastononro.github.io/trainable/ Sun, 05 Apr 2026 12:00:00 GMT Every company has the data; almost none have the two specialist weeks that turn it into a model. agents Trainable is a studio where agents do the data science. Start with the demo — it's faster than anything I can write here.

Trainable - Agents for data science and Machine Learning — the current build, one session, end to end. Watch on YouTube ↗

§Having the data is not enough

Every company I've worked with already has the data. That was the last decade's project and it worked: the warehouse exists, the events are instrumented, the nightly job runs, and storage costs less per terabyte than the meeting about it.

What almost none of them have is the person who turns that table into a model somebody will actually deploy. That person knows which distribution to be suspicious of, reads a 0.99 AUC as a leaked column rather than a result, and insists on beating the dull benchmark before anyone is allowed to try something interesting. They're scarce, expensive, and slow to hire — check how long your last data-scientist req stayed open. So the table sits there. The bottleneck was never storage. It hasn't been compute for years. It's the calendar of about four people.

Most of what fills that calendar isn't the judgment — it's the two weeks afterwards. Profiling every column, arguing with the missingness pattern, choosing encodings, splitting the data once and meaning it, fitting the boring baseline, thirty Optuna trials on top, writing up the leaderboard and the feature importances. Those two weeks are what Trainable automates, and I want to be exact about where it stops. It doesn't frame the problem, decide whether the model should exist, or notice that the label you were handed is a proxy for something you'd rather not predict. A human still picks the target, says what a false negative costs, and reads the review. The roster includes a reviewer agent whose entire brief is to disagree with the others — leakage, temporal leakage, features derived from the target. It's there because judgment is the part that needs checking, not the part that got replaced.

The input doesn't have to be a table, either. sample-data/ ships seven datasets: six are a single CSV with a guessable header — Titanic, telco churn, wine quality, California housing, heart failure, employee attrition — and the seventh is 1,765 photographs of cars with a COCO annotation file beside them. Nothing in the upload path inspects a file: bytes go to S3 and to the Modal Volume under whatever relative path the browser sent, the content type falls back to application/octet-stream, and the agent opens them with Python. The honest limit is the paved road rather than the door: the prep explorer and the tabular training skill both want parquet splits and a named target, so a detector gets the sandbox, a GPU and a live grid of prediction overlays, but no pre-written methodology.

One of three images in the smoke-test split, 472×303 in the dataset, carrying a single COCO box at 249, 186, 101, 49 — nothing a SELECT could return
One of three images in the smoke-test split, 472×303 in the dataset, carrying a single COCO box at 249, 186, 101, 49 — nothing a SELECT could return
practitioner brings the data browser Next.js studio Trainable nine agents: EDA, prep, train, review Claude API Agent SDK · MCP Modal sandboxes + volume S3 / MinIO boto3 · browsable copy uses SSE Redrawn from the repo's own C4 context diagram; the bottom row is all somebody else's cloud.
Redrawn from the repo's own C4 context diagram: the whole system is one box, and the person stands outside it.

§Every call, a sandbox

pip install trainable-ai && trainable init, and a wizard handles Docker and keys. What took the time isn't the agent roster. It's what happens after an agent decides to run some Python. The execute-code skill hands a string to services/sandbox.py:run_code, which builds a fresh Modal Sandbox per call:

sb = await modal.Sandbox.create.aio(
    "python", "-u", "-c", full_code,
    image=_get_image(),
    volumes={"/data": get_volume()},
    gpu=gpu,
    timeout=effective_timeout,
    workdir=f"/data/sessions/{session_id}",
    app=await _get_app(),
)

Every argument there is a decision. -u because buffered stdout defeats the streaming below. The image is debian_slim(python_version="3.11") with pandas, scikit-learn, xgboost, lightgbm, duckdb, optuna, shap, statsmodels, CPU-wheel torch and tensorflow-cpu preinstalled — memoized in a module global, so the definition is assembled once per backend process and Modal serves the built layers to every sandbox after that. gpu and timeout come from a profile on the project row, so moving a project onto an A10G is a settings modal and the next call picks it up.1Compute choices are coerced to a known set — cpu, T4, L4, A10G, A100-40GB, A100-80GB, H100 — and anything unrecognized falls back to cpu, so that a misspelled value never silently lights up an expensive A100. When Modal kills a sandbox at the 600-second mark the handler doesn't fail the run: it returns a tool result with is_error=True and prose explaining the options — split the work, shrink the data, or retry with heavy=true.

§State between calls

The sandbox is gone the moment it exits. What survives is the Modal Volume at /data, and workdir=/data/sessions/{session_id} is the line I'd keep if I could only keep one. Every execution and every notebook cell runs with the session workspace as its cwd, so pd.read_parquet("data/train.parquet") lands on the volume without anyone reasoning about absolute paths.

Modal needs that workdir to exist before Python starts, so an empty src/__init__.py is written there before every spawn, and the preamble prepended to every execution puts that src/ on the path:

def _bootstrap_session_repo() -> None:
    session_src = _VOL_ROOT / "sessions" / _SID / "src"
    session_src.mkdir(parents=True, exist_ok=True)
    …
    sys.path.insert(0, src)

So the workspace is a small repo. One agent promotes a helper out of a notebook into src/loaders.py, and a different agent, later, does from loaders import load_raw instead of rewriting it. State passes between agents as code, not as a summary of code. Notebooks get the one thing a per-call sandbox can't offer — a kernel that remembers — from a long-lived sandbox per session driven over its own stdin, so variables set in data-overview are visible in baseline-model.

The volume is reloaded before each spawn so a new sandbox sees the last one's writes, and afterwards a recursive diff emits one event per new path. Nothing in the UI knows where a figure is supposed to live: a post-run pass classifies every file by extension into an artifacts table — .png a chart, .md a report, .pkl a model — and the frontend renders from that index.2The folder layout — report.md, figures/, data/ — is convention, not enforcement. The prep explorer resolves the parquet splits through three tiers (prep metadata, then the artifact table, then a recursive scan) precisely so nothing hardcodes a path. Agents rearrange their workspace; indexes don't mind.

S3 (MinIO in dev) is the second tier and pointedly not the authority. The volume is primary because it's what sandboxes mount. S3 takes a mirror of the finished workspace, and gets read back only to surface files that were uploaded but never made it into a sandbox, reported per file as in_sandbox: false.

§Stdout as protocol

browser Next.js studio FastAPI broadcaster + routers Modal Sandbox 600s per call Postgres metrics, artifacts Modal Volume /data/sessions/{id} S3 / MinIO browsable copy SSE execute_code stdout: metrics, charts, new files rows, never files mounted at /data s3_sync mirrors the finished workspace Solid = a request. Dashed = a stream. The sandbox mounts the volume; S3 is the copy you can browse.
Where a line of agent-written Python goes, and where its output ends up.

There is no agent in the loop of the live dashboard. Inside the sandbox, trainable.log() prints a line:

def log(step, metrics, run=None):
    payload = {"step": int(step),
               "metrics": {k: float(v) for k, v in dict(metrics).items()}}
    if _MODE == "sandbox":
        _emit(payload)      # print(json.dumps(...), flush=True)
        return

run_code consumes the sandbox's stdout chunk by chunk, buffers it into lines, and sends each through parse_stdout_line. Recognized envelopes — scalar metrics, a chart_config, or a rich event such as an image, table or confusion matrix — get persisted and published; everything else is just output the model will read back. The whitelist of rich types is deliberately small. A new one needs a frontend renderer first; otherwise a typo in agent code quietly fills the table with rows nothing can draw.

Publishing means a per-session asyncio.Queue in an in-process broadcaster, drained by exactly one SSE route. The routers never learn the event types; they call broadcaster.publish, and the generator behind the endpoint is thirteen lines long (I counted).

This is the part that changes how the thing feels, and the part I'd defend to somebody who cares about none of the above. A chart that appears while the fit is still running is a chart you check at minute two, when the wrong column is still cheap to fix!! A report twenty minutes later is something you audit; a metric arriving now is something you supervise.

§What you can actually do

A project owns datasets; an experiment is an agent-declared bundle of processed dataset, model and metrics inside a session. fork-experiment copies the parent's input datasets under a new name and hypothesis and leaves the parent alone — the "same prep, different model" loop, agent-only, no button. Comparing runs is one route: up to eight session ids in, and metric series, feature-column overlap and per-session cost out. The prep explorer runs DuckDB over the parquet splits in-process and locks the connection down after loading rather than before: SET enable_external_access = false, SELECT-only, read_csv and glob blocked, limit forced to 1000.

Promotion is where the session stops mattering. promote_session_model copies artifact bytes to a versioned path under the project, freezes the metric summary, and infers the framework from the extension. The module docstring is the design: sessions die; promoted models don't. Deploying is a second, separate step, because a pickle isn't a service — an agent writes a Modal app.py next to the artifact, where @app.cls loads the model at container start and a fastapi_endpoint exposes predict behind a typed Pydantic contract. validate-serving-app runs before the Deploy button becomes clickable, and the check that earns its keep is the dullest one: that the artifact path resolves on the volume. A wrong path makes @modal.enter raise FileNotFoundError, Modal then retries the container forever, and that presents as a hung endpoint rather than an error.

And you can leave. The session download streams a zip of the workspace with a requirements.txt, a README and a local shim for the trainable module appended, so scripts that from trainable import log still run on a laptop.

§Where this started

The demo at the top is an update. This is the first version, recorded before most of the machinery above existed:

Trainable - Introducing a different path to Data Science — the original cut, and the argument unchanged. Watch on YouTube ↗

I keep it up because the framing in that title is the one I still hold; everything that changed since is underneath it. A different path to the same deployed model: the labor is automatable, the judgment is not, and the interesting engineering is the part nobody demos — a machine per call, a directory that outlives it, and a stream that lets you catch a mistake at minute two instead of in the write-up.

§State as code, not summaries

The question underneath all of this: how do you let a language model run arbitrary code for twenty minutes without the run becoming unauditable? My answer turned out to be a shape of storage, not a permission model: destroy the machine after every call, keep the directory. Everything a run produced is a file on the volume with a path, so a recursive diff and a classification by extension are enough to build the index the UI reads. Nothing had to be declared up front. No agent had to remember to register its own output.

The transferable idea is the smaller one. When an agent promotes a helper out of a notebook into src/loaders.py and a later agent imports it, the second one isn't working from a paraphrase of the first one's reasoning — it's running the same code, and it fails loudly if that code was wrong. Summaries drop exactly the details that break. Delete the volume and what remains is a transcript of a data science project: fluent, ordered, and reproducible by nobody.

The agents are the demo; the two weeks are the point.

]]> ralph-installer: make a loop anywhere https://lucastononro.github.io/ralph-installer/ https://lucastononro.github.io/ralph-installer/ Mon, 19 Jan 2026 12:00:00 GMT One line of bash runs the loop; the state files around it are the actual work. skill The Ralph loop is a coding agent restarted over and over with the same prompt. Nothing carries over in context: the instructions are a file, the plan is a file, the notes the agent leaves for its successor are a file, and every iteration is a fresh context window that reads all three off disk. Geoffrey Huntley, who named it after the Simpsons character, writes it as one line:

while :; do cat PROMPT.md | claude-code ; done

"Ralph is a technique. In its purest form, Ralph is a Bash loop." That's the whole idea, and it spread: awesome-ralph catalogs forty-plus implementations, and in December 2025 Anthropic shipped a ralph-wiggum plugin inside Claude Code that does the same thing with a Stop hook instead of a shell loop.1Check this before reaching for it. Dex Horthy, who wrote a history of the technique, says the plugin "dies in cryptic ways" and "misses the key point of ralph" — a Stop hook keeps you in one session, so you lose the fresh context window per iteration the bash loop gives you for free.

ralph-installer is my version. npx ralph-installer@latest install drops five files into ralph/ and two skills into .claude/skills/. MIT, published to npm the same day the repo started.

prompt.md never edited fresh agent one story typecheck, test then commit prd.json + progress.txt COMPLETE? or max iters exit 0 the prompt goes in again byte for byte — only the repo changed no memory between passes
The same prompt every pass; the repository is the only thing that changes.

§What the loop reads

ralph/ralph-claude.sh, trimmed to the part that matters:

for i in $(seq 1 $MAX_ITERATIONS); do
  PROMPT_CONTENT=$(cat "$SCRIPT_DIR/prompt.md")
  OUTPUT=$(claude --print --dangerously-skip-permissions "$PROMPT_CONTENT" 2>&1 | tee /dev/stderr) || true

  if echo "$OUTPUT" | grep -q "<promise>COMPLETE</promise>"; then
    echo "Ralph completed all tasks!"
    exit 0
  fi
  sleep 2
done
exit 1

prompt.md is re-read from disk on every pass and handed over untouched. So the only difference between iteration three and iteration four is the repository — a commit, a passes: true flipped in prd.json, and a few lines appended to progress.txt.

The stopping condition is a string the agent has to say. prompt.md ends by telling it to check whether all stories have passes: true and, if so, reply <promise>COMPLETE</promise>. Exit 0 when that appears, exit 1 when the loop burns through MAX_ITERATIONS without it. Different exit codes for "done" and "gave up" matter the moment something other than you reads them.

The rest of prompt.md is ten numbered steps: read the PRD and the progress log, check out the branch named in prd.json, take the highest-priority story where passes: false, implement that one, run the project's checks, commit as feat: [Story ID] - [Story Title], flip the flag, append notes. Frontend stories aren't complete until verified in a browser.

§Why a wrapper exists

The loop is five lines. Everything that decides whether it works is in the files it reads, and each one is a place to be slightly wrong.

The plan has to be cut to the size of a context window. skills/ralph.md is blunt: "Each story must be completable in ONE Ralph iteration (one context window)... If a story is too big, the LLM runs out of context before finishing and produces broken code." The rule of thumb is that if you can't describe the change in two or three sentences, split it. Criteria have to be checkable — the skill lists "Works correctly" and "Good UX" as bad ones, and appends "Typecheck passes" to every story.

The progress log has to have a shape, or its notes are useless to the next pass. Each iteration appends a Learnings for future iterations block, then promotes anything general into a ## Codebase Patterns section pinned to the top. That section is the closest thing the design has to memory.

And the installer has to not destroy any of it on a second run. The line I'd point at:

// User data files that should never be overwritten if they exist
const PRESERVE_FILES = ['prd.json', 'progress.txt', 'prompt.md'];

Those three are the loop's state. Upgrading to a newer runner script would otherwise wipe your plan and your accumulated notes, so --force overwrites the scripts and still refuses to touch them — there's a test named exactly that, out of eighteen. Anything else that already exists is a conflict, and conflicts exit 2 with the list rather than guessing.

The rest is small and dull, which is the right amount of interesting for an installer. ensureFrontmatter fills a skill's missing name from the kebab-cased filename and description from its first H1 before writing .claude/skills/<name>/SKILL.md. The runner archives itself: when branchName no longer matches .last-branch, the old plan and log move into archive/YYYY-MM-DD-<name>/ and a clean log starts. And view serves /api/prd and /api/progress off a stdlib Python server on 8089, to a canvas that redraws every two seconds — you watch stories go green instead of tailing a log.

§Usage as a stopping condition

The criticism of Ralph with numbers attached is cost. The Register puts Huntley's figure at "about US $10 of compute and/or SaaS resources each hour", which he notes is "far closer to wages paid to fast food workers" than to developer salaries. Anthropic's plugin README cites a contract "completed for $297 in API costs" and tells you to always set --max-iterations.

scheduled-ralph.sh measures a different meter, the one you're on when you run claude rather than the API. It lifts the Claude Code OAuth token out of the macOS Keychain (or ~/.config/claude-code/auth.json on Linux) and asks:

curl -s -f "https://api.anthropic.com/api/oauth/usage" \
  -H "Authorization: Bearer $token" \
  -H "anthropic-beta: oauth-2025-04-20"

Then it reads .five_hour.utilization, .seven_day.utilization and .five_hour.resets_at. --max-usage 70 stops the loop at 70% of the five-hour block. --wait parks it instead, re-checking every 300 seconds until the block rolls over. --wait-next-session sleeps through the current block before starting — set it before bed and the loop runs on tomorrow's quota!!

If usage is already past the ceiling and you didn't pass --wait, it exits 2 instead of starting a loop it can't finish.2When it can't get a token, get_usage_percent returns -1, the script logs a warning, and the loop runs anyway — bounded only by iterations. It fails open, not closed. That's the wrong default for a script whose whole job is a budget.

§When it's the wrong tool

The people who like this technique hold the sharpest limits on it. Huntley: "There's no way in heck would I use Ralph in an existing code base," and "Engineers are still needed." His framing is greenfield bootstrapping, "with the expectation you'll get 90% done." how-to-ralph-wiggum states the failure plainly: "Ralph can go in circles, ignore instructions, or take wrong directions—this is expected and part of the tuning process." Horthy adds the two that hurt: "If the specs are bad, the results will be meh," and "If you are iterating/exploring, you probably don't want ralph in the first place."

The critics land in the same place from the other side. On the thread for that history, the top comment is "Just look at the code quality produced by these loops... It's complete garbage, and since it runs in a loop, the amount of garbage multiplies over time." Horthy's own reply is the sober version: "I don't think anyone serious would recommend it for serious production systems." Further down sits the shape that does work — somebody Ralphing personal Mac menu bar apps, who reports that "the apps work and scratch the itch that motivated them" and admits they can't judge the Swift.

The reports agree on one thing: the loop needs a machine-checkable definition of done. Migrations, lint sweeps, dependency bumps, test cleanups. Tessl's writeup describes a Jest-to-Vitest migration where done meant tests pass, vitest.config exists, jest.config is gone and no Jest imports remain, bounded by "explicit limits on iteration count, token usage, and cost." Nothing on that list requires taste. The same piece quotes Huntley on the part everyone skips: "you really want to babysit this thing."

Measured against that, passes: true plus "Typecheck passes" is a weak gate, since the agent sets its own flag. The PRD is doing a spec's job, and a vague criterion gets marked done by an agent that meant well. The honest limit: this makes a good loop cheap to start, and it does not make a bad PRD safe.

None of the difficulty is in the five lines.

]]> localiza&co: what we actually ship with LLMs https://lucastononro.github.io/localiza-videocast/ https://lucastononro.github.io/localiza-videocast/ Wed, 16 Apr 2025 12:00:00 GMT A Portuguese-language videocast on how Localiza uses advanced analytics and LLMs day to day. talks I joined Juliana Aarão Rocha, Samuel Oliveira and Thamires Souto on the Localiza&Co videocast, for an episode called Revolução Digital: Como a AI está transformando nosso mundo. It is in Portuguese, and the title is grander than the conversation, which I mean as a compliment.

Most of it was concrete: how we use advanced analytics and LLMs inside Localiza, across a wide spread of situations and products. Four people who work on this from different angles, comparing notes on what the technology actually does in a real company — not in a keynote.

No grand thesis came out of it. I think that's the correct outcome. The interesting part of this work is almost always the specific problem, not the pattern you can draw over the top of it.

Open in Spotify ↗

The useful part of these conversations is hearing colleagues describe the same tools from a different seat.

]]> consulting cast: I was the guest https://lucastononro.github.io/consulting-cast/ https://lucastononro.github.io/consulting-cast/ Fri, 22 Dec 2023 12:00:00 GMT A guest spot on UFRJ's Consulting Cast, talking about consulting, AI and LLMs. talks I was the guest on Consulting Cast, a podcast run out of UFRJ. We talked about consulting, AI and LLMs, and what they are doing to the industry.

It's in Portuguese.

Open in Spotify ↗
]]> llm-food-delivery: the agent loop was a switch statement https://lucastononro.github.io/llm-food-delivery/ https://lucastononro.github.io/llm-food-delivery/ Sun, 17 Dec 2023 12:00:00 GMT Ordering dinner by voice in 2023: eleven hand-written function schemas and a loop that lived in the browser. agents In December 2023 I wanted a chatbot that could drive a web interface rather than talk about one. llm-food-delivery is a fake delivery app — FastAPI, Vue, SQLite, Pinecone — where you order dinner by voice and never have to touch the deliberately cluttered UI. whisper-1 in, tts-1 out (voice shimmer), gpt-4-32k in the middle. The README calls it an "Interface-omniscient" agent, which tells you roughly what 2023 felt like.

Ordering dinner without touching the interface. December 2023.

§Eleven schemas, by hand

Every action a user could take was mapped to a function. Each function was a YAML block I typed out myself:

- name: get_restaurant_pages
  description: This function finds a restaurant that has similar food or name to …
  parameters:
    type: object
    properties:
      type_of_restaurant:
        type: string
        description: The type of food or restaurant that the user requested.
          E.g. Italian, Pizza, Sushi, Indian, Vegan, etc...
      quantity:
        type: integer
        description: … if the user request is broad you should set 2,3,4 or even 5
  required: [name_of_restaurant, type_of_restaurant, food_requested, …]
  allow: true

Eleven of those, each ending in allow: true (a flag I wrote eleven times and never once flipped). prompts/functions/__init__.py reads the YAML, keeps the allowed entries, pops the allow key and writes signatures.json — at import time, as a side effect of importing the package. That's the sort of thing you do when the schema list is also your build system.

What strikes me now is how much of the control flow is written in English, inside YAML descriptions, addressed to a model free to ignore it. There was no way to express ordering, so the descriptions do it: open_restaurant_page says "This function can only be called after the function get_restaurant_pages", and close_restaurant_page says "you can check the recent actions with get_user_actions to verify it, before calling the function". Sequencing, preconditions, confirmation prompts — all prose, all politely unenforced. Today the schema falls out of a type hint and the ordering is a graph you can look at. Then, it was a paragraph and a hope.1close_restaurant_page's description ends "goes back to the main page (menu with all the restaurants) x". The stray x is still sitting there in config.yaml, and I have never worked out what it was going to be.

§The actual call

The whole model layer is one function, and it predates everything you'd expect:

model_args = {
    "model": CFG_OPENAI.get("chat_model", "gpt-4"),
    "temperature": CFG_OPENAI.get("temperature", 0),
    "max_tokens": CFG_OPENAI.get("max_tokens", 512),
    …
}

# Incrementing in cases of function calling
if len(functions) > 0:
    model_args["functions"] = functions
    model_args["function_call"] = "auto"

functions, not tools. function_call: "auto", not tool_choice. The config pins temperature: 0.0 and max_tokens: 256, and the system prompt closes with "You do not answer with more than 70 words". The ceiling was a budget as much as a guard.

Three absences did most of the shaping. There are no tool-call ids, so a result goes back as a bare {role, name, content} triple, and PromptHandler.get_messages has to re-shape every message so that only function ones carry a name. There are no parallel calls, so two items into the cart meant two round trips — exactly why add_food_to_cart's description pleads "You can call this function multiple times in a row". And there was no loop. The API returned one function_call and considered the matter closed. Every service function is wrapped in @retry(wait=wait_random(min=1, max=5), stop=stop_after_attempt(5)), because that was the other thing 2023 gave you a lot of.

§The loop in the browser

you, talking MediaRecorder whisper-1 /chat/transcribe gpt-4-32k functions=[…] switch, 11 cases handleFunctionCall a Vue method @action: breadcrumb role: "function" back into history ask it again top row: FastAPI — bottom row: one Vue file The tools are interface actions, not computations, so the loop closes in the browser. Breadcrumbs only come back when the model calls get_user_actions.
The unusual bit: the cycle closes through a Vue component, not through the server.

Since nobody supplied a loop, the loop went where the tools were — the frontend. handleFunctionCall is a switch over eleven names:

switch (functionCallSignal.name) {
  case "get_restaurant_pages":
    functionCallResponseContent = this.handleGetRestaurant(functionCallResponse);
    break;
  case "open_restaurant_page":
    functionCallResponseContent = this.handleOpenRestaurant(functionCallResponse);
    break;
  …
}
resolve({ role: "function", content: functionCallResponseContent,
          name: functionCallSignal.name });

and generateAnswer calls itself with the result appended:

this.handleFunctionCall(functionCallSignal)
  .then((functionCallOutput) => {
    this.messages.push(functionCallOutput)
    this.generateAnswer(
      chatHistory=this.getChatHistory(),
      // function_call=false,
      function_call=true,
    )
  })

handleOpenRestaurant parses the uuid out of the response and then calls this.$refs.restaurantsContainer.selectRestaurant(restaurantId). A tool call is a $refs poke. This is why the backend has dummy_function(). Six of the eleven server handlers (close_restaurant_page, get_user_actions, open_shopping_cart, close_shopping_cart, place_order, activate_handsfree) route to a coroutine that returns "", and add_food_to_cart echoes its own arguments straight back. The round trip exists to keep the shape uniform. Nothing happens on the server because there is nothing there to happen: "open the cart" has no headless meaning.

That's the part I'd defend. If your tools are interface actions, the agent loop belongs on the same side of the wire as the interface, and the backend degrades into a schema server with a retry decorator.2chatHistory=this.getChatHistory() looks like a Python keyword argument. In JavaScript it assigns to the enclosing scope's variable and passes the value along, which happens to do the right thing. Nothing counts the hops, either — the recursion's only brake is the model deciding to answer.

§Memory as breadcrumbs

The model can't see the DOM, so the interface narrates itself. Every component emits upward into one method with a one-line body — this.actions.push("@action:" + msg + " at " + this.getCurrentTime()). The call sites are written for a reader rather than a parser: registerAction("opened the restaurant page " + this.selectedRestaurant.name + " with restaurant_uuid=" + this.selectedRestaurant.uuid), or "incremented the amount of food Margherita Pizza to 2 items (in the restaurant page, not in the shopping cart)". Two vocabularies coexist: @action: for anything that happened in the interface, @agent-action: for the string handed back to the model after it caused something. The system prompt introduces the convention directly — "Every time the user or you do something ... you will receive a message starting with @action:" — and get_user_actions hands over the last ten on demand. Ask it "what did I just do?" and it goes and looks.3The implementation is JSON.stringify(this.actions.splice(-10)), and splice removes. Reading the memory deleted it. In a demo, nobody noticed.

Timestamps come from hourTime + ':' + minuteTime + ':' + secondTime, unpadded, so the log reads at 14:2:9. It never mattered.

§Synthetic everything

Restaurants and menus are a literal Python list in populate_fake_data.py — Italian Bistro, Sushi House, Taco Fiesta, twenty-five of them with eight items each. The README credits the list to GPT-4. The photographs are DALL·E 3, one call per row, with a prompt of magnificent brevity: "Photorealistic zoomed picture of food: " + food.name, skipped if the file already exists. Search is llama-index over a Pinecone index called auto-food-order: one document per restaurant, its text the name, the description and every dish concatenated. Retrieval is ExactMatchFilter(key="search_type", value="restaurant") with top_k=quantity — the model's own guess, from that schema above, at how many restaurants it ought to look at.

§The loop follows the tools

I've said where the loop went. The general form is the part I'd keep: placement is settled by where the tools are, not by where the model call is convenient to make. A tool call here was a $refs poke, which is why six of the eleven server handlers returned "" and the backend stayed in the cycle only for the shape of it. Anything the user could otherwise have clicked pulls the loop toward the client the same way.

The other thing is a note about my own labor. Nearly everything I typed out by hand has since become somebody else's code. The eleven YAML blocks fall out of a type hint. The sequencing I wrote as prose in a description field is a graph. The recursion in generateAnswer ships in the SDK. None of that was foresight — the platform arrived and the scaffolding stopped being mine. Which leaves the bit worth carrying forward: I only hand-write what nothing supplies yet. So whichever piece of this year's scaffolding I'm proudest of having built myself is probably next.

The config on disk has since drifted to gpt-4-0125-preview, which is the only part of this that has aged as expected.

]]>