Autoresearch

Autoresearch

Let an AI agent modify and rerun your LLM training script, keeping only experiments that beat the baseline.

Visit website

What is Autoresearch?

Tuning a small language model usually means staring at the training script, guessing what to change, and then waiting to see whether the loss actually moves. Autoresearch turns that loop into a job you can hand off: you describe the direction in a program.md file, and an AI agent edits the training script on its own, runs the experiment, and keeps only the version that scores better.

The project grew out of Karpathy's nanochat work and is packaged as a standalone repository. The script the agent rewrites is train.py, a compact roughly 630-line file holding a full GPT model, a Muon plus AdamW optimizer setup, and the training loop. Each run is capped at five minutes and judged by one number, validation bits per byte (val_bpb). Beat the baseline and the change stays; fall short and it is thrown away.

That rhythm matters when you have a hunch but no time to test twenty variants. At about twelve experiments an hour, a night of letting it run can cover roughly a hundred tries while you sleep, and you come back to a version that survived every check rather than the one you happened to stop on. It is a practical first look for anyone curious how far an agent can carry a research loop on this scale; FindAIGo is a reasonable place to weigh it against other open model tooling before committing a GPU weekend.

To run it you need an NVIDIA GPU (the project was tested on an H100), PyTorch and a handful of dependencies. It is released under the MIT license, and community branches for macOS and Apple's MLX have appeared. What the agent does is narrow and honest: it proposes edits to the code, not new ideas about the model. The research direction still comes from the person writing the Markdown.

加载中 / Loading

Discussion

Loading comments…