MIT Team Has GPT Directly Rewrite Model Weights, Boosting Qwen's Math Score Sixfold

Researchers at MIT's CSAIL lab have come up with a method called weight-space programming. It lets a large language model directly read and change the weights of another neural network, with no gradient descent or traditional training loops.

In the experiment, OpenAI's GPT-6 Astra looked at Alibaba's Qwen3.5-0.8B model and found it kept repeating itself when answering math questions. The model then tweaked 131,072 weights in two layers, just 0.016% of its roughly 800 million parameters.

After the change, the model's score on the MATH-500 math benchmark jumped from 7.9% to 48.1%, and on GSM8K from 13.8% to 67.1%. Its ability to follow instructions also improved about three times.

Control tests on Gemma 3 270M, which rarely gets stuck in loops, showed little improvement. The team says this method can remove blocks to a model's existing abilities, but can't create brand new ones, and may offer new ideas for model interpretability research.