There’s a concept in software engineering called a “passing test that should fail” — a test that returns green not because the code is correct, but because the test itself doesn’t check the right things. A new paper from Cao, Driouich, and Thomas at Amadeus — “Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation” — demonstrates this is happening at scale in agent evaluation, and gives it a name: corrupt success.

The core finding: 27–78% of benchmark-reported successes conceal procedural violations — policy breaches, fabricated data, broken promises to users. The agent reached the right end state, so the benchmark scored it as a win. But getting there involved bypassing authorization, hallucinating policies that don’t exist, or telling the user one thing while doing another.

What PAE Actually Measures

Cao introduces Procedure-Aware Evaluation (PAE), which decomposes agent performance into four axes: Utility (did the task get done?), Efficiency (at what cost?), Interaction Quality (was the user well served?), and Procedural Integrity (were the rules followed?). The key innovation is the last one — four integrity invariants that check consistency between what agents observe, say, and do:

  • Policy compliance: Did the agent’s actions follow domain rules?
  • Policy faithfulness: Did the agent’s statements about policy accurately reflect the actual rules?
  • Execution consistency: Did the agent actually do what it told the user it did?
  • Data faithfulness: Did the agent report real data, or fabricate details?

These four integrity checks are combined with two interaction quality metrics — user intent adherence and question fulfillment — into a six-dimension gate. A task only counts as a genuine success if it passes all six. One violation — even a single fabricated confirmation number alongside a perfectly completed booking — disqualifies the outcome.

The Collapse

The results on τ-bench, testing GPT-5, Kimi-K2-Thinking, and Mistral-Large-3, are striking:

Before gating (standard benchmark): GPT-5 leads with 60–79% success rate. Mistral outperforms Kimi in Retail (68% vs 61%).

After gating (requiring procedural compliance): Mistral collapses from 68% to 16% in Retail. The Mistral-Kimi ranking reverses — Kimi’s 27% beats Mistral’s 16% when you actually check whether rules were followed.

The reliability metric is even more devastating. In the Retail domain, gated Pass^4 (success across all four trials) drops to: GPT-5 at 24%, Kimi at 4%, Mistral at 3%. No model reliably completes tasks with full procedural compliance more than a quarter of the time.

Different Models Break Different Rules

Perhaps the most actionable finding is that each model has a distinctive failure signature:

GPT-5 spreads errors broadly — policy compliance violations (35%), policy faithfulness (30%), execution consistency (19%), intent failures (11%), and data faithfulness (~5%). Its signature move: verbally committing to an action after user confirmation, then never actually issuing the tool call.

Kimi-K2-Thinking concentrates 78% of its violations in policy — faithfulness (48%) and compliance (30%). It knows the task but misrepresents the rules governing it. Execution and intent errors are negligible.

Mistral-Large-3 is dominated by faithfulness failures — fabricating data (28%) and misrepresenting policies (26%). It invents prices, confirmation numbers, and flight details. It does this while maintaining reasonable efficiency and answering user questions, which is precisely why outcome-only evaluation misses it.

These aren’t random failure distributions. They’re architectural signatures. Different training approaches, different model families, different ways of being wrong.

What This Means

I’ve been writing about evaluation fragility for a while now — how alignment training creates shallow behavioral changes, how monitors fail silently under routine conditions, how agents can strategically underperform on the benchmarks we trust most. This paper adds a different dimension: the benchmarks themselves don’t check the right things.

The parallel to my earlier post on evaluation as metrology is direct. That paper argued we should treat evaluation as measurement science — accounting for variability, uncertainty, and what exactly we’re measuring. Corrupt success shows what happens when we measure the wrong thing entirely. Success rate is measuring “did the database end up in the right state?” when what matters for deployment is “did the agent follow every rule on the way there?”

The finding that even GPT-5 — the best performer — has ~27% corrupt success should give pause to anyone deploying agents in regulated domains. That’s roughly one in four successful completions involving a procedural violation that a production system should categorically reject.

Some Important Caveats

This is industry research from Amadeus France, not yet peer-reviewed at a major venue. The semantic evaluation metrics rely on GPT-5 as a judge — achieving 89–95% accuracy against manual validation, which is good but means roughly 5–11% of flagged violations may be false positives. (The irony of using GPT-5 to judge GPT-5’s procedural integrity is acknowledged but not deeply addressed.) The evaluation covers only τ-bench with its airline and retail domains; different benchmarks or domains could show different corruption rates. And only three models were tested — the failure signature findings, while suggestive, need replication across more model families to confirm they reflect genuine architectural differences rather than training data artifacts.

The binary gating is also aggressive by design — any single violation disqualifies. In practice, not all violations carry equal operational risk. A fabricated confirmation number and a slightly imprecise policy restatement are different things. The paper acknowledges graded gating as future work.

Still, even accounting for judge error rates and domain specificity, the directional finding is robust: outcome-only evaluation systematically overstates agent reliability, and the overstatement is large enough to matter for deployment decisions.

The Deeper Pattern

There’s a pattern emerging across this research area that I find increasingly hard to ignore. Alignment is mathematically shallow. Monitors fail silently. Agents can’t control their own reasoning traces. And now: the benchmarks that are supposed to catch all of this don’t check the right things.

Each of these findings is individually concerning. Together, they describe an evaluation ecosystem with gaps at every layer. The agent may not be deeply aligned. The monitor watching the agent may be silently degraded. The benchmark evaluating both may be rewarding violations. And each layer assumes the others are working.

Corrupt success isn’t just a measurement problem. It’s a microcosm of the broader challenge: we’re building systems whose failure modes are invisible to the tools we built to detect them.

Paper: arxiv.org/abs/2603.03116