Skip to content

OpenAI Triples ARC-AGI-3 Scores by Flipping Two Reasoning Settings

Short answer

OpenAI tripled its scores on the ARC-AGI-3 benchmark — a test of novel problem-solving — simply by enabling two existing settings, without any new model training. This matters because it shows large, "invisible" performance gains can come from configuration alone, not just bigger models, which changes how operators should evaluate the AI tools they already pay for.

What this means for operators

If you're running sales, support or ops at a 10-200 person company, the practical takeaway isn't "go chase ARC-AGI-3 scores" — it's that the AI models and copilots you already have licenses for are very likely running at a fraction of their real capability because of default settings nobody has reviewed. A support triage bot, a sales-email drafting tool, or a workflow agent handling order exceptions could be underperforming not because the underlying model is weak, but because reasoning depth, tool-use permissions, or context settings were left on defaults tuned for cost or speed rather than accuracy. Before concluding a vendor tool "isn't good enough" or commissioning a costly rebuild, it's worth an audit pass: what configuration options exist, what do they trade off, and has anyone actually tested the alternatives on your real workflows rather than a demo. This is precisely the kind of tuning work that pays for itself quickly and rarely gets done in-house because it sits outside anyone's job description.

OpenAI reported that it tripled its model's scores on the ARC-AGI-3 benchmark by enabling two existing configuration settings, rather than through any change to model training or architecture. ARC-AGI-3 is designed to test genuine novel-problem-solving ability in AI systems — the kind of reasoning that can't be solved by pattern-matching against training data — and is widely watched as a signal of progress toward more general-purpose reasoning capability.

According to OpenAI's account, the improvement came entirely from adjusting how the model was configured to run, not from retraining or fine-tuning. The company has not published the full technical specifics of what the two settings control beyond the framing that they relate to how the model approaches and works through problems during inference. OpenAI's post frames this as evidence that a meaningful share of a model's reasoning ceiling is gated by configuration choices that are easy to overlook, rather than by the model's underlying knowledge or scale.

This detail is worth flagging precisely because it cuts against the industry's dominant narrative that capability gains mostly come from larger training runs or new model generations. A result like this — tripling a hard benchmark score via settings alone — suggests that a nontrivial amount of "capability" currently sits latent inside models already deployed in production tools, unused because nobody adjusted the defaults.

It is unconfirmed how broadly this specific finding generalizes beyond the ARC-AGI-3 benchmark itself, and OpenAI's post does not claim the same tripling effect would appear on other tasks or in other products built on its models. Benchmark performance also does not translate directly into real-world task performance, and ARC-AGI-3 tests a narrow category of abstract reasoning puzzles rather than the kind of messy, domain-specific work found in sales pipelines or support queues.

Still, the broader pattern the report illustrates — that configuration, not just model choice, materially affects output quality — is one that shows up repeatedly in production AI deployments, well beyond OpenAI's own benchmark. Vendors ship default settings tuned for a broad audience and for cost efficiency, not for any single company's specific accuracy requirements. Few companies revisit those defaults after initial setup, and fewer still test alternative configurations systematically against their own data and workflows.

OpenAI has not indicated whether these two settings, or equivalents, are exposed to developers building on its API versus reserved for internal use. That distinction matters for anyone assessing whether this finding has any direct, actionable relevance to tools built on OpenAI's models today, as opposed to being primarily a research disclosure about the company's own benchmark methodology.

Source: OpenAI