OpenAI Improves Framework, Boosting GPT-5.6 Sol's ARC-AGI-3 Score by 188%

OpenAI recently announced improvements to their testing framework, boosting the GPT-5.6 Sol model's score on the ARC-AGI-3 benchmark by 188%. ARC-AGI-3, created by the ARC Prize team, tests AI's ability to learn and adapt in unfamiliar environments. Previously, GPT-5.6 Sol scored 7.8%, while GPT-5.5 only got 0.4%.

OpenAI found two issues with the official framework: it discarded private reasoning after each action, and used a rolling context window that lost early actions. They introduced two fixes: inference preservation, which keeps the model's reasoning across interactions via the Responses API, and context compression, which optimizes information retention in long tasks.

With these changes, GPT-5.6 Sol's score jumped from 13.3% to 38.3% while using six times fewer output tokens, a 188.42% improvement. This shows how framework design can greatly impact AI performance, offering new insights for AI evaluation.