On November 2, 2026, Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab of MIT CSAIL published "Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights". It is a blog post with released logs, not a peer-reviewed paper. The idea is that a general-purpose LLM reads or writes a network's weights directly, through reasoning, "not through an optimizer, and in particular not with gradient descent".
Reading weights
In the first experiment, the authors trained four identically shaped policies (17,253 weights each) with PPO on their own grid versions of Breakout, Pong, Pac-Man and Freeway. Given only the raw weights as text and no list of games, Astra named the right game for all four. In a harder test, three 7.16-million-weight character-level transformers were trained on Python code, Japanese in romaji and English. The authors removed the input and output layers and did not allow Astra to run any part of the network. It still named the right language for all three in both attempts and identified every copying (induction) head, matching heads measured by running the models, using a weight-multiplication method the authors note they never described to it.
Editing a real model
The headline result is on Qwen3.5-0.8B, whose model card warns that it is prone to thinking loops. Before the edit, 85 of the model's first 96 practice answers ran to the 32,768-token limit. Astra "traced the looping to one direction written by the MLPs of layers 15 and 19, and subtracted that direction from 64 output columns in each (131,072 weights)", which is 0.016 percent of the model. With a budget of 50 forward passes to test its edits, MATH-500 rose from 7.9 to 48.1 percent, GSM8K from 13.8 to 67.1 percent and IFEval from 21.0 to 62.3 percent, and the share of MATH-500 answers hitting the token limit fell from 93.0 to 33.1 percent. With 10 or 25 forward passes the gains were small (MATH-500 12.3 and 14.1 percent).
The authors then ran the control that makes the result readable. Gemma 3 270M rarely loops (8.5 percent of answers hit its limit). There, Astra "could not improve it": no edit moved MATH-500 by more than about a point (6.8 percent base, 7.7 at best). Their own conclusion: "Editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked."
What remains uncertain. These are early results from one model provider's frontier model on small networks and one sub-billion-parameter language model, reported by the authors. The full logs are released in a GitHub repository created October 29, and someone the authors credit at Hugging Face rebuilt the edited Qwen model and re-ran its evaluation, but we have not reproduced anything. The edit resembles known direction-ablation techniques, and the post does not compare it with a person applying those techniques. The authors say they are building a benchmark, so the method has no standard evaluation yet.
Analysis: A gain from 7.9 to 48.1 percent sounds like new capability, but most of it came from answers that had stopped running out of tokens. When the model stopped looping, it could finish answers it already knew how to reach. Any evaluation of a small model should report how often answers are truncated, next to accuracy. At Belgavi's Book Store, Tanishka could add that column to the help-desk model's evaluation sheet. If a new model version suddenly scores much better on the CASE ticket set, check first whether it simply stopped running out of tokens.
Sources and evidence
MIT CSAIL researchers report (blog post, November 2) that GPT-6 Astra identified what small networks were trained on from raw weights alone, and reduced Qwen3.5-0.8B's thinking loops by editing 131,072 weights (0.016 percent), raising MATH-500 from 7.9 to 48.1 percent as token-limit hits fell from 93 to 33 percent. The same approach could not improve Gemma 3 270M, which rarely loops. In the authors' words, editing removed an obstacle but did not add an ability.