Karpathy Open-Sources Autoresearch: AI Automatically Runs LLM Training Experiments, Up to 100 Times Overnight

Andrej Karpathy, co-founder of OpenAI and founder of AI education company Eureka Labs, recently open-sourced the autoresearch project. This project packages the AI Agent auto-tuning workflow from his previous LLM training project nanochat into an independent repository, making it easy for developers to try over the weekend.

The core design is "humans write Markdown, AI writes code": users define research directions via program.md, and the AI Agent autonomously modifies the 630-line train.py (including the full GPT model, Muon+AdamW optimizer, and training loop). Each experiment runs for 5 minutes, using validation bits per byte (val_bpb) as the only metric—if it outperforms the baseline, it is retained; otherwise, discarded. This allows about 12 experiments per hour, up to 100 overnight.

The project only requires an NVIDIA GPU (tested on H100), relies on PyTorch and a few packages, and is open-sourced under the MIT license. Community branches for macOS and MLX have already emerged.