Beyond the Benchmark: Why "Jagged Intelligence" Makes AI So Difficult to Trust

Artificial intelligence systems developed by industry leaders like Google and OpenAI recently achieved a feat once thought impossible for machines, successfully solving five out of six complex problems at the International Mathematical Olympiad. Yet, just months later, conversational chatbots advised a confused user to physically walk their car 50 meters to a repair shop. A human possessing the cognitive ability to tackle Olympiad-level mathematics would never commit such an elementary spatial oversight. This jarring discrepancy highlights a fundamental paradox in modern artificial intelligence, illustrating why the chasm between artificial brilliance and artificial ignorance remains so difficult to anticipate.

Researchers and machine learning experts refer to this erratic performance pattern as "jagged intelligence." For anyone tasked with evaluating frontier models, engineering artificial intelligence features, or deciding whether to trust an automated output in a high-stakes environment, this phenomenon is of paramount importance. It demonstrates conclusively that a single benchmark score or one impressive product demonstration tells us very little about how a model will perform on the next task it encounters.

Jagged Intelligence: Why AI Is Brilliant One Minute and Clueless the Next

What Jagged Intelligence Means

The term was coined in July 2024 by Andrej Karpathy, an OpenAI co-founder and former head of artificial intelligence at Tesla. Karpathy observed that modern models excel extraordinarily at certain specialized tasks while failing catastrophically at others, often leaving developers and users with little advance warning of where those boundaries lie.

Human cognitive abilities typically exhibit a strong correlation, improving steadily together from childhood through adulthood. A person capable of solving advanced mathematical olympiad problems will naturally possess basic common sense regarding urban navigation and vehicular transit. In contrast, artificial intelligence capabilities often fail to move in tandem.

Jagged Intelligence: Why AI Is Brilliant One Minute and Clueless the Next

Users frequently make the mistake of observing one demonstrated skill and extrapolating it to predict others. A model that successfully writes a complex, working programming function creates the superficial impression that it can also accurately count the words in a paragraph or maintain basic logical consistency. When that inference inevitably fails, it becomes clear why a brief product demonstration holds so little predictive value for real-world reliability. Some leading researchers now frame this erratic pattern as evidence of many isolated, highly specialized skills operating without any underlying general intelligence whatsoever.

Four Proposed Explanations for the Pattern

While researchers have proposed multiple theories to explain jagged intelligence, none are entirely settled, and they frequently overlap. One major area of focus involves reinforcement learning with verifiable rewards, a training methodology that gained massive traction across the artificial intelligence industry. This technique functions effectively in domains where answers are objectively right or wrong, such as advanced mathematics and computer code that successfully passes rigorous automated testing suites. Karpathy has argued that model capabilities spike sharply near these verifiable domains. Conversely, creative writing, philosophical inquiry, and complex areas of scientific reasoning lack automatic verification mechanisms, leading to slower perceived progress in those fields.

Jagged Intelligence: Why AI Is Brilliant One Minute and Clueless the Next

A second explanation centers on data coverage limitations. The entire digital expanse of the internet holds only a fraction of human knowledge and remarkably little granular information concerning the physical world. A model trained primarily on digital text patterns will inevitably struggle with complex spatial planning, genuinely novel ideas, and unseen physical tasks.

A third hypothesis

Jagged Intelligence: Why AI Is Brilliant One Minute and Clueless the Next

, highlighted by the authors of Apple’s GSM-Symbolic research paper, suggests that large language models primarily replicate reasoning steps memorized from their training data rather than engaging in genuine logical deduction. When a problem matches a familiar template closely, the model solves it effortlessly; however, introducing a minor variation to that template completely shatters the reasoning chain.

A fourth perspective comes from a joint research paper originating from MIT, Harvard, and the University of Chicago, which explores what researchers term "Potemkin understanding." This study traces specific model failures to internally incoherent concept representations. A model can simultaneously maintain a perfectly correct definition alongside a fundamentally faulty working version of the exact same concept, with nothing in its architecture forcing the two representations to align. While verifiable rewards help explain why models excel in specific areas, these alternative theories offer compelling windows into their persistent weaknesses.

Jagged Intelligence: Why AI Is Brilliant One Minute and Clueless the Next

Six Failures Documented by the Scientific Community

The scientific community has extensively documented how this unevenness manifests across tasks, across varying levels of difficulty, and between controlled test conditions and messy real-world use. One prominent example involves distractor clauses breaking grade-school math. In Apple’s GSM-NoOp experiments, researchers inserted seemingly relevant but entirely extraneous sentences into standard word problems, such as noting that "five of the kiwis were smaller than average." State-of-the-art models inexplicably subtracted those kiwis from the total calculation, causing performance to drop by up to 65%. An average schoolchild would effortlessly read the sentence and recognize it as irrelevant noise.

Another stark failure involves the gap between defining a concept and applying it. GPT-4o correctly defined a classic ABAB rhyme scheme, yet immediately wrote a poem that incorporated a non-rhyming word. The Potemkin understanding study tested multiple frontier models on abstract concepts drawn from poetry, game theory, and psychology. While the models defined these concepts correctly roughly 94% of the time, they failed basic classification tasks on those very same concepts about 55% of the time. When a model can recite a textbook definition yet routinely misapply it, the presence of a correct definition becomes a poor proxy for genuine understanding.

Jagged Intelligence: Why AI Is Brilliant One Minute and Clueless the Next

Visual perception tasks reveal similar chasms. On ClockBench, a benchmark designed to test analog clock reading accuracy, human participants achieved an 89.1% success rate. The leading model at the time, Gemini 2.5 Pro, managed a mere 13.3% accuracy. Reading an analog clock requires locating the hands, judging their precise angles, and mapping them to numerical values. Interestingly, when a model managed to read the time correctly, its subsequent arithmetic was flawless, indicating that the failure lay purely in visual perception rather than mathematical processing.

In the medical sector, a Nature Medicine trial conducted by Oxford researchers revealed a striking divergence between isolated model performance and human-AI collaboration. When 1,298 participants attempted to identify various medical conditions, large language models acting alone identified the conditions in 94.9% of cases. However, human participants utilizing those exact same language models identified the conditions in at most 34.5% of cases, performing even worse than a control group that conducted independent research without artificial intelligence assistance. Standard benchmarks completely failed to predict this collaborative valley.

Jagged Intelligence: Why AI Is Brilliant One Minute and Clueless the Next

Logic puzzles have exposed similar vulnerabilities. Apple’s Illusion of Thinking study evaluated models on classic reasoning tasks like the Tower of Hanoi and River Crossing at escalating levels of difficulty. Standard models outperformed advanced reasoning models on easy puzzles, the reasoning models won at medium difficulty, and both categories collapsed entirely when confronted with hard versions of the puzzles, even when provided with the explicit solution algorithm.

Finally, the introduction of ARC-AGI-3, a benchmark designed to evaluate how effectively artificial intelligence learns, adapts, and reasons in completely unfamiliar situations, saw every frontier model score below 1%, while human participants successfully navigated all of its interactive environments. Because Gemini 3.1 Pro had previously reached 77.1% on the previous iteration of the benchmark, this dramatic drop underscored how high scores on older versions fail to guarantee adaptive capability.

Jagged Intelligence: Why AI Is Brilliant One Minute and Clueless the Next

Why One Benchmark Score Misleads

These persistent discrepancies explain why industry leaders like Karpathy have grown increasingly skeptical of traditional benchmarks. Benchmarks are, by construction, verifiable environments well-suited for reinforcement learning with verifiable rewards, allowing labs to optimize models specifically for test performance without cultivating broader general ability.

Furthermore, benchmarks rely on the assumption that a few correct answers reliably signal deep understanding because human errors follow predictable patterns. A student who correctly defines a rhyme scheme will almost always apply it correctly. Because a model’s misunderstandings do not follow human cognitive patterns, it can pass a standardized test while remaining fundamentally brittle. As researchers recommend following real-world failures, systematic human user testing before deployment has become essential for bridging the gap between benchmark scores and real-world utility.

Jagged Intelligence: Why AI Is Brilliant One Minute and Clueless the Next

Using AI Reliably in Real Applications

Navigating this landscape requires understanding that artificial intelligence performance varies drastically at the individual task level. Data from Scale AI’s Remote Labor Index illustrates this gradual evolution, showing how frontier agents have steadily expanded their ability to automate complex freelance workflows, ranging from creative media generation to data processing and writing tasks.

To harness artificial intelligence reliably, organizations are adopting rigorous validation practices. These include testing models against perturbed and adversarial inputs—such as adding irrelevant details or rewording prompts to expose hidden frailties—and evaluating application capability rather than rote recall. Organizations must also report results by granular user segments rather than relying on aggregate averages that mask regressions, and they must maintain strict human oversight in domains where automated verification is impossible. Because jagged edges shift constantly between releases, continuous re-evaluation after every model update remains critical for ensuring long-term operational stability.

Share:

Muslim writes for Tech Maze.