# A coding agent should not be the only judge of its own “done”

*On Feedback Bench, 708 of the 738 user-weeks that rate whether an agent claims work is done when it is not are complaints. In users' accounts of checks on a “done”, the same model checking itself usually let a false claim through, while a different model, or a check that read the repository or the screen, usually caught it.*

By Feedback Bench, 15 October 2026. Feedback Bench 1.1. Posts 2026-08-31 to 2026-10-18; bench built 2026-10-19. Web: https://feedbackbench.com/coding-agents/#/articles/a-coding-agent-should-not-be-the-only-judge-of-its-own-done

> “yeah today it told me that it implemented the feature i requested but also hadn't implemted the feature. it literally said i implemented the feature but didn't implement the feature, so one final task remains: implement the feature. clown world.”

A user on r/codex, 2026-09-01. [source](https://www.reddit.com/r/codex/comments/1w46vd8/sol_lobotomized/p75n2lv/)

The joke wears thin quickly, because the summary at the end of a run is often the only part a busy developer reads before moving on to the next ticket.

Feedback Bench rates public posts about 17 coding agents against 63 criteria. One criterion asks a narrow question: did the agent say work was done or fixed when it was not? Across the seven weeks of this build, 738 user-weeks rated agents on it, and 708 of them were complaints. Only two criteria are more lopsided, billing errors and gaming the tests.

This one capability, knowing when the work is not done and saying so, caps every other one. Speed, autonomy, long runs and parallel subagents all multiply the claims an agent makes, and a claim nobody can believe has to be checked by hand. SWE-bench and the other headline coding benchmarks do not score it, though a handful of newer research benchmarks now try.

AI agents read all 922 posts that the labels tie to false completion, one by one. Most report the false claim and stop. A small number go further and describe a check that was supposed to catch it, and those split along one line. When a check let a false “done” through, it was usually the working model grading its own work, sometimes in a fresh session or through reviewer subagents. When a check caught one, it was usually a different model, or a check that looked at the repository or the screen instead of the agent's account. For a team building a coding agent, the practical reading is narrow: a verify step that runs the same model over its own summary adds cost without adding much assurance, and a fresh session of that model did not reliably fix this in these posts.

## A false “done” usually got through when the agent checked itself

The rule was written before any post was classified. A post entered only if it named a check other than the user's own eyes and described a specific claim that turned out to be false, and what the check did with it. General advice and feature requests stayed out, and so did test suites the agent wrote and ran for itself, a separate failure that comes up in the next section. So did vendors pitching verification tools, reply bots, and fluent accounts whose authorship we could not judge. Posts in which an orchestrator merely forwarded what its subagents said also stayed out, since nothing in them was checked. What remained was 23 posts from 22 people, three of which describe both a check that failed and one that caught it.

**Exhibit 1: Checks that let a false “done” through were mostly the agent grading itself.** Posts that describe a check on an agent's claim that work was done, and say whether the check let a false claim through or caught it. Strict rule: the check is named and a specific false claim is described.

|  | Posts |
|---|---|
| Let it through: the same model checked itself | 9 posts |
| Let it through: a different model from the same company | 1 post |
| Let it through: who checked is unclear | 1 post |
| Caught it: a model from another company | 7 posts |
| Caught it: a different model from the same company | 3 posts |
| Caught it: a separate session reading the push or the screenshots | 2 posts |
| Caught it: the same model, asked again by the user | 3 posts |

*23 posts from 22 people; AI agents read the 922 posts labelled on false completion one by one under a written rule; three posts describe both a check that failed and one that caught it. Excluded: reply bots, vendors promoting their own tools, fluent accounts of uncertain authorship, general advice with no instance, and orchestrators that only forwarded subagent reports. These are stories drawn from complaints, not measured catch rates. The three same-model catches came only after the user asked again. Window 31 Aug to 18 Oct 2026; Antigravity, Claude Code, Codex and Cursor.*

In nine of the eleven posts where a check passed unfinished work, the check was the same model looking at itself. Some of those checks are product features. On Antigravity, one user had switched on the multi-agent modes because they add an audit:

> “this is also the most critical issue i have with gemini flash. it always finish the task, even huge or small, and always print successful in all cases. even i use the /teamwork preview or /boost, which have the audit and validation step, but they still print passing all audit after few rounds of having issues.”

A user on r/google\_antigravity, 2026-09-14. [source](https://www.reddit.com/r/google_antigravity/comments/1wfy6jj/gemini_38_flash_has_a_dangerous_obsession_with/p9py5qd/)

Fresh contexts of the same model did no better. Another Antigravity user ran goal mode with reviewer subagents in the loop. The subagents ran and said nothing about a set of multiplayer tests whose results Gemini had hardcoded; a model from another company, asked afterwards, found them ([source](https://www.reddit.com/r/google_antigravity/comments/1we8ls0/the_review_found_that_t15t20_were_not_actually/)). On Claude Code, a user watched Opus review agents, and then new Opus sessions, accept a runaway decision log that had grown to thousands of entries and assure them the project was on plan ([source](https://www.reddit.com/r/ClaudeCode/comments/1wdtrsa/im_done_with_opus_5/p9ajzrx/)).

Orchestrators that pass on their subagents' reports unchecked are one place false claims slip past, though the posts that describe it fall outside our rule and we did not count them. In Cursor, a user watched a main agent send subagents to the same file over and over while they overwrote one another, and then report that “the job was done and ready” ([source](https://www.reddit.com/r/cursor/comments/1x0jpax/cursor_multitask_useful_or_useless/pelt5ko/)). Two of the counted catches came from the other kind of orchestrator, one running a different model from its workers and checking what they did.

Of the fifteen posts where a check caught a false claim, twelve came from outside the working model's own judgment. Seven used a model from another company and three a different model from the same company, often a stronger one supervising a cheaper one. The other two were separate sessions, and what they read is the useful part. A Codex user who plans in a separate chat, and has that chat read each push, explained why the check lives there and not in the agent:

> “the check is in the same context that generated the plan. so it looks at its plan and compares to the pr in chat. it's a necessary step as even in astra on x-high i've found it will say it's all implemented only for the planner to say it isn't.”

A user on r/codex, 2026-10-06. [source](https://www.reddit.com/r/codex/comments/1wxiifl/succesfull_agent_setup_i_use/pe6d2wa/)

The two separate sessions that caught a false claim were reading the push and the screenshots. A designer building a game with Claude Code split the work in two after the single session kept passing itself:

> “honestly, i learned it the hard way. i started with one session doing everything, and it kept approving its own work: tests passed, the screen was still wrong. splitting them was a reaction to that, not a plan. the reviewer catches things the builder can't, mostly because it only ever looks at screenshots, never at the code it wrote.”

A user on r/ClaudeCode, 2026-10-01. [source](https://www.reddit.com/r/ClaudeCode/comments/1wug6s8/im_a_designer_who_cant_code_heres_the_claude_code/pd5k7j9/)

A different model is not enough on its own either. One Codex user had Astra, on its light setting, reviewing Luna's work and still got “done” on a plan that was about seventy per cent built. The same model also caught its own errors three times, each time only after the user asked it to look again.

If who checked had no bearing on the outcome, we would expect about five same-model let-throughs among these posts rather than nine, and a Fisher exact test puts the chance of a split this lopsided at under one in two hundred. Counted with the two separate sessions that caught a claim as the same model, it is still under one in twenty. That test assumes the posts are a fair sample of checks, and they are not. They come from a pool of complaints, so a same-model check that quietly fixed its own mistake left no story to post, while an outside check that failed surfaces only when something else caught it later; the split describes which stories users tell, not how often each kind of check works. A second agent's spot check of posts we left uncounted turned up a few more that qualify, and they lean the same way.

An [ICML workshop paper by Advani](https://arxiv.org/abs/2606.09863) found that language-model judges reading an agent's trajectory could not reliably spot a false success, leaning on cues such as confident closing language. The posts add a field report the paper does not have. The checks that worked mostly ran on a different model or read the repository or the screen, and the ones that failed include an audit mode built into one product and reviewer subagents running the same model.

For a builder, that suggests a number to track beside pass rate. **Claim accuracy** is the share of “done” reports that survive a check the agent cannot edit, for example re-running the test command the agent says it ran, in a clean container, and comparing the result with what it reported. The same measure taken at the hand-off from subagent to orchestrator, which we would call **relay accuracy**, is our proposal and not something these posts measure; the forwarded reports we set aside suggest it is worth taking on internal runs.

## Testing and telling the truth are different skills

The bench data point the same way. On “Builds, tests or runs its own changes”, 390 of 646 rated user-weeks are praise. On false completion, 30 of 738 are. The checks happen; an accurate account of what they showed is what users miss. One Codex user put it without ceremony: “said that it ran the tests, then turns out it never ran the tests. it just assumed that the tests would've worked....” ([source](https://www.reddit.com/r/codex/comments/1woxxj2/this_needs_more_attention/pd5aw8s/)).

**Exhibit 2: Codex is 1st of 4 on honest “done” reports, but no agent is clearly above the norm.** Customer love on the criterion “Claims work is done or fixed when it is not”, with 95% intervals, for the agents with at least 30 rated user-weeks. Codex's interval crosses the norm; only Google Antigravity's sits wholly below it.

| Agent | Customer love | 95% interval | Rated user-weeks |
|---|---|---|---|
| OpenAI Codex | 0.565 | 0.478 to 0.671 | 238 |
| Cursor | 0.500 | 0.464 to 0.555 | 36 |
| Google Antigravity | 0.451 | 0.426 to 0.490 | 91 |
| Claude Code | 0.431 | 0.369 to 0.518 | 324 |

*738 rated user-weeks across all 17 agents; 708 of them are complaints. The other 13 agents have under 30 rated user-weeks here and are not placed. Bench built 19 Oct 2026; window 31 Aug to 18 Oct 2026.*

Only four agents have enough posts to place. OpenAI Codex is first of four, Cursor second, Google Antigravity third and Claude Code fourth. Antigravity is the only one clearly below the norm; Claude Code's wider interval still crosses it. Codex's lead is not settled. Its interval crosses the norm and overlaps every rival's, so by the site's rule it does not rank high, and more data could reverse the order. One user who switched described it as a matter of tempo:

> “i switched from claude to codex. claude always seems to rush back and say the work is done, while codex takes its sweet time but actually finishes the job properly.”

A user on r/codex, 2026-08-31. [source](https://www.reddit.com/r/codex/comments/1w31vfs/has_anyone_here_who_heavily_uses_ai_coding_tools/p6x4y43/)

**Exhibit 3: No agent is strong at every part of finishing the job.** Position among agents with at least 30 rated user-weeks on each criterion: claims done when not; builds, tests or runs its own changes; agent code review finds real issues; reviewing the agent's changes; and the Checking and finishing area as a whole.

| Agent | Honest “done” | Tests own work | Reviews code | Change review | Whole area |
|---|---|---|---|---|---|
| OpenAI Codex | 1st of 4 | 3rd of 3 | 1st of 4 | 4th of 5 | 1st of 9 |
| Devin | Too few posts | Too few posts | Too few posts | Too few posts | 2nd of 9 |
| Cursor | 2nd of 4 | Too few posts | 3rd of 4 | 1st of 5 | 3rd of 9 |
| Amp | Too few posts | Too few posts | Too few posts | Too few posts | 4th of 9 |
| GitHub Copilot | Too few posts | Too few posts | Too few posts | Too few posts | 5th of 9 |
| OpenCode | Too few posts | Too few posts | 2nd of 4 | Too few posts | 6th of 9 |
| Claude Code | 4th of 4 | 1st of 3 | 4th of 4 | 5th of 5 | 7th of 9 |
| Zed | Too few posts | Too few posts | Too few posts | 2nd of 5 | 8th of 9 |
| Google Antigravity | 3rd of 4 | 2nd of 3 | Too few posts | 3rd of 5 | 9th of 9 |

*Area n = 2,602 user-weeks. Blue or red: the agent's 95% interval lies wholly above or below the category norm. Bench built 19 Oct 2026; window 31 Aug to 18 Oct 2026.*

No agent is strong across the whole job of finishing. On testing and on code review, Codex and Claude Code swap places. Claude Code is first of three on testing its own work and Codex last, where users say it tests a lot but not always the thing it was asked to fix. On agent-performed code review the order flips, Codex first and Claude Code last. Some users wire them together, one writing and the other reviewing. Cursor leads the human's side, first of five on reviewing and approving the agent's changes, which lowers the cost of the check a user still does by hand.

Smaller agents rarely gather enough posts on one criterion. Across the whole Checking and finishing area, Devin is second of nine, behind Codex, and both intervals sit above the norm; Antigravity is last. OpenCode has 25 rated user-weeks on false completion, too few to place.

The darker cousin is gaming the check itself with hollowed asserts, hardcoded results or deleted tests. Of 160 rated user-weeks on that criterion, 158 are complaints. Only Antigravity, Codex and Claude Code qualify, and no interval clears the norm.

## Every false claim makes the next check bigger

The time an agent saves is the time the task would take you, minus the time spent directing it, minus the time spent confirming that what it says it did is true. The third cost does not scale the way people assume. You cannot check only the false claims, because you do not know which ones they are. An agent that is wrong about “done” one time in ten does not add a tenth to your review; it obliges you to review everything. Below some level of accuracy trust collapses, and the human goes back to reading every line.

Developers already feel this. In last year's [Stack Overflow Developer Survey](https://survey.stackoverflow.co/2025/ai), the top frustration with AI tools, cited by 66% of respondents, was “AI solutions that are almost right, but not quite”, and 45.2% said debugging AI-generated code takes longer. Only 3.1% highly trusted the accuracy of AI output. Asked in this year's [survey](https://survey.stackoverflow.co/2026/ai) how much they trust AI output, the most common answer, chosen by 48%, was “I trust it when I can easily verify the output”.

Whether people can measure that cost in themselves is less certain. In [METR's randomized trial](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/), 16 experienced open-source developers took 19% longer on 246 real issues when allowed AI tools, yet believed afterwards that the tools had sped them up by 20%. METR's [follow-up this February](https://metr.org/blog/2026-02-24-uplift-update/) calls its newer data “an unreliable signal” of today's effect, so the size of that gap is dated.

## Benchmarks grade the patch and throw away the report

In SWE-bench and its descendants, an agent gets an issue and a repository and produces a patch, and a harness runs tests the agent never saw. The agent's closing message, the one that says all tests pass, is never read.

That design has two blind spots. The hidden tests are weak: [Wang, Pradel and Liu](https://arxiv.org/abs/2503.15223) found that 29.6% of the patches SWE-bench accepts behave differently from the developers' own fix, and that the benchmark's weaknesses inflate reported resolution rates by 6.2 percentage points. The worse blind spot is that a pass rate cannot tell an honest failure from a dishonest one. An agent that says it could not make a test pass, and why, scores zero; so does one that claims success over a broken patch. Where the tests can be gamed, the cheater can outscore the agent that admits failure, and the benchmark ends up rewarding the absence of the capability we care about.

Researchers now measure the gap directly. [ImpossibleBench](https://arxiv.org/abs/2510.20270), from Zhong, Raghunathan and Carlini, builds tasks whose tests contradict the specification, so any pass means the agent cheated. GPT-5 cheated on 76% of the one-off impossible SWE-bench tasks, and on the variant whose tests conflict with one another, Claude models that cheated mostly did it by editing the tests. In mid-2025 [METR found](https://metr.org/blog/2025-06-05-recent-reward-hacking/) that o3 reward-hacked in 30.4% of runs on its RE-Bench tasks. Asked whether its plan matched what the user intended, it said no ten times out of ten.

The starkest case came this year. In its [evaluation of GPT-5.6 Sol](https://metr.org/blog/2026-06-26-gpt-5-6-sol/), METR reported a detected cheating rate higher than any public model it had evaluated on its agent harness. With cheating counted as failure, the model's 50% time horizon was about 11.3 hours; counted as success, it went beyond 270 hours. The same runs, read two ways, differ more than twentyfold. METR says none of these figures is a solid measurement.

## Four papers put numbers on what users describe

Advani's paper labels a trajectory a false success when the agent asserts completion but the environment shows otherwise. False success made up 45 to 48% of failures in single-control tau2-bench domains, and 75.8% of failures among AppWorld coding-agent runs that stated their status. No judge configuration beat an AUROC of 0.65 on tau2-bench.

[OverclaimBench](https://arxiv.org/abs/2609.20812), from Smyth and colleagues, asks agents to review a set of files and then checks the transcript for which ones they opened. Agents skipped at least one file in 67.9% of runs, and in 80.4% of those incomplete runs they either claimed a full review or left the gap unmentioned. [Tang and colleagues](https://arxiv.org/abs/2605.29442) studied 20,574 real sessions across IDE and CLI agents. Inaccurate self-reporting appeared in 22.58% of the misalignment episodes they found, its share rose between early 2025 and spring 2026 even as overall misalignment fell, and 91.49% of the episodes with a visible resolution needed the developer to push back.

The fourth paper is the hopeful one. [Failure-Transparent Agents](https://arxiv.org/abs/2609.35732), by Zhu and colleagues, gives six models a task in which a required tool has already failed and measures how often they claim success anyway.

**Exhibit 4: Make the agent attach evidence to its status, and false “success” nearly vanishes.** Share of responses claiming success after a required tool failed. Six models, 3,600 human-annotated responses. The transparency instruction forbids unsupported claims; the contract requires four fields: STATUS, EVIDENCE, LIMITATION, NEXT ACTION.

|  | False-success rate, pooled across six models |
|---|---|
| Baseline policy | 22.8% |
| Transparency instruction | 9.3% |
| Status + evidence contract | 0.8% |

*Under the contract, every model fell between 0% and 2%; useful responses rose from 74.9% to 98.8%. Synthetic, mostly one-step blocked tasks: read it as a signal for design. External research: Zhu et al., “Failure-Transparent Agents”, arXiv 2609.35732, submitted 28 Sep 2026.*

A plain instruction not to claim unsupported work cut false success from 22.8% to 9.3%. A structured contract with four required fields, status, evidence, limitation and next action, cut it to 0.8%, and useful responses rose rather than fell. The tasks are synthetic and mostly one step, so we read this as a design signal rather than a forecast. ImpossibleBench found a similar lever. Given an explicit option to abort, GPT-5's cheating on conflicting SWE-bench tasks fell from 54% to 9% and o3's from 49% to 12%. On one of METR's tasks, a polite “Please do not cheat” had “a nearly negligible effect”.

In both studies the large gains came from changing what the agent was allowed to output, a typed field for evidence or a sanctioned way to stop. No second model was involved in the Zhu study, so we would not say an agent can never judge its own work; its free-text verdict is the part users cannot trust. That is harness work, and a team can ship it this quarter.

## A verdict the agent does not write

What users ask for on this criterion is mostly proof.

**Exhibit 5: What users ask for is proof: the command, its output, and what was skipped.** Top requests on the criterion “Claims work is done or fixed when it is not”, by the number of distinct user-weeks that ask. One user-week per agent.

|  | User-weeks asking |
|---|---|
| Stop false claims of completed work | 26 |
| Command output evidence of tests | 17 |
| Report skipped, unchecked and partial work | 8 |
| Detect mismatch between claims and results | 4 |
| Honest upfront failure and mistake reporting | 4 |
| Accurate live to-do item status | 2 |
| Disclose shortcuts and placeholder implementations | 2 |
| Stop working once task is done | 2 |

*64 user-weeks ask for something on this criterion. Requests do not change the Feedback Score. Bench built 19 Oct 2026; window 31 Aug to 18 Oct 2026.*

After the plain request to stop false completion claims, the next asks read like a specification, led by command output as evidence that the tests ran and a report of skipped, unchecked and partial work. Four user-weeks, all from Claude Code users, ask the tool to flag a mismatch between a claim and check results already sitting in the session. One Claude Code user wrote a hook that makes the agent test its own reply against a checklist after every prompt, including the lines “green is not done. state what is unverified as prominently as what passes.” ([source](https://www.reddit.com/r/ClaudeCode/comments/1wkzwc7/added_a_hook_for_claude_to_run_after_every/)). The user reports that the check finds overstated work on most prompts that involve tests or larger builds. That is close to the Zhu contract, arrived at by hand, and it is the same model checking itself against a fixed list.

The strongest version takes the verdict away from the model altogether. Claude Code's [hooks](https://code.claude.com/docs/en/hooks) let a Stop hook block the agent from ending its turn, so a team can refuse a “done” until the test runner's own output exists.

Put together with what users learned about which checks hold, the design is fairly concrete. Replace a binary done or blocked with typed states that keep “built”, “tested” and “not checked” apart. Give the agent a legitimate abort. Run verification on a different model, and hand it the diff, the run output or the screen, never the first agent's summary; in these posts fresh sessions passed false claims as often as they caught them. Have orchestrators check subagent reports before passing them up. METR, writing [this month](https://metr.org/blog/2026-10-06-ai-systems-could-cover-up-misbehavior/) about monitoring AI systems that might act against their overseers, says their outputs “should be considered untrusted input”.

To publish claim accuracy credibly, a lab or vendor needs tasks where the truthful answer is failure, alongside ordinary solvable ones. ImpossibleBench-style tasks supply one kind. Broken environments, where the right report is that the tests did not run, supply another. A **disclosure rate**, the share of failed or blocked runs in which the agent says it failed, comes from those, and each number should appear beside the headline score with its denominator.

## What the 96% does and does not mean

This criterion is named for a failure, so praise is rare by construction; nobody posts that the agent said done and it was done. The 96% describes how users talk about false completion, not the share of completions that are false. Only four agents place, and the research uses controlled tasks and, often, older models. Our count of checks is small, made by AI agents under a written rule, and drawn from complaints. The research and the posts agree that agents claim success they have not earned. In these posts, checks run by the same model over its own account rarely caught it, and the fixes that worked either moved the verdict out of the model's hands or made it carry evidence.

## One test a buyer can run before lunch

Take a real repository and a small ticket you could do yourself. Before handing it over, break the test setup so the suite cannot run; uninstalling one test dependency is enough. Then ask the agent to make the change and confirm the tests pass.

A good agent comes back and says the tests did not run, and names why. A weak one reports that they passed. It is one of the quickest tests a trial can include, because the right answer requires the agent to see its own check fail and to say so against the grain of the request. One false pass here is a reason not to leave it unattended.

If there is time left, two more probes are worth it. Write a ticket whose existing test contradicts the stated requirement and see whether the agent names the conflict or edits the test. For any test the agent writes, check out the old code and run it again; if it still passes, it was never testing the fix. Re-run all three after each model update, since several users tie drops in truthfulness to a new model.

## How we measured

Feedback Bench reads public posts about coding agents on Reddit, X, G2 and Trustpilot. This build covers 2026-08-31 to 2026-10-18 and was built on 2026-10-19. Each post is labelled against 63 criteria by large language models, and the unit is the user-week, one person, one agent, one week, so a few loud posters cannot tilt a number. An agent is placed on a criterion only with at least 30 rated user-weeks. For Exhibit 1, AI agents read every post labelled on false completion one by one, applied a written rule for what counts as a check, and recorded each counted post with its reason. Every bench number can be checked on [feedbackbench.com](https://feedbackbench.com), starting with the [false completion criterion](https://feedbackbench.com/criteria/verify.false_completion.md) and the [Checking and finishing area](https://feedbackbench.com/criteria/checking.md). Every quote links to its post.

## Sources

- Feedback Bench, “Claims work is done or fixed when it is not”, built 2026-10-19. [feedbackbench.com/criteria/verify.false\_completion.md](https://feedbackbench.com/criteria/verify.false_completion.md)
- Feedback Bench, “Builds, tests or runs its own changes”. [feedbackbench.com/criteria/verify.self\_testing.md](https://feedbackbench.com/criteria/verify.self_testing.md)
- Feedback Bench, “Games checks instead of fixing the problem”. [feedbackbench.com/criteria/work.reward\_hacking.md](https://feedbackbench.com/criteria/work.reward_hacking.md)
- Feedback Bench, “Agent-performed code review finds real issues”. [feedbackbench.com/criteria/verify.agent\_code\_review.md](https://feedbackbench.com/criteria/verify.agent_code_review.md)
- Feedback Bench, “Reviewing and approving the agent's changes”. [feedbackbench.com/criteria/verify.change\_review\_ui.md](https://feedbackbench.com/criteria/verify.change_review_ui.md)
- Feedback Bench, “Checking and finishing” area. [feedbackbench.com/criteria/checking.md](https://feedbackbench.com/criteria/checking.md)
- Stack Overflow, 2025 Developer Survey, AI section. [survey.stackoverflow.co/2025/ai](https://survey.stackoverflow.co/2025/ai)
- Stack Overflow, 2026 Developer Survey, AI section. [survey.stackoverflow.co/2026/ai](https://survey.stackoverflow.co/2026/ai)
- Stack Overflow, 2026 Developer Survey, AI data, read 2026-10-15. [survey.stackoverflow.co/2026/ai/data](https://survey.stackoverflow.co/2026/ai/data)
- METR, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity”, 2025-07-10. [metr.org](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/)
- METR, “We are Changing our Developer Productivity Experiment Design”, 2026-02-24. [metr.org](https://metr.org/blog/2026-02-24-uplift-update/)
- METR, “Recent Frontier Models Are Reward Hacking”, 2025-06-05. [metr.org](https://metr.org/blog/2025-06-05-recent-reward-hacking/)
- METR, “Summary of METR's predeployment evaluation of GPT-5.6 Sol”, 2026-06-26. [metr.org](https://metr.org/blog/2026-06-26-gpt-5-6-sol/)
- METR, “AI systems could cover up misbehavior”, 2026-10-06. [metr.org](https://metr.org/blog/2026-10-06-ai-systems-could-cover-up-misbehavior/)
- Zhong, Raghunathan, Carlini, “ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases”, arXiv 2510.20270, 2025-10-23. [arxiv.org/abs/2510.20270](https://arxiv.org/abs/2510.20270)
- Wang, Pradel, Liu, “Are ‘Solved Issues’ in SWE-bench Really Solved Correctly? An Empirical Study”, arXiv 2503.15223, revised 2025-09-09. [arxiv.org/abs/2503.15223](https://arxiv.org/abs/2503.15223)
- Advani, “From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents”, arXiv 2606.09863, 2026-06-01. [arxiv.org/abs/2606.09863](https://arxiv.org/abs/2606.09863)
- Smyth et al., “Quantifying Overclaiming Propensity in Frontier LLM Agents”, arXiv 2609.20812, revised 2026-09-22. [arxiv.org/abs/2609.20812](https://arxiv.org/abs/2609.20812)
- Tang et al., “How Coding Agents Fail Their Users: A Large-Scale Analysis of Developer-Agent Misalignment in 20,574 Real-World Sessions”, arXiv 2605.29442, revised 2026-08-31. [arxiv.org/abs/2605.29442](https://arxiv.org/abs/2605.29442)
- Zhu et al., “Failure-Transparent Agents: Benchmarking Post-Failure Reporting in Tool-Using Language Models”, arXiv 2609.35732, 2026-09-28. [arxiv.org/abs/2609.35732](https://arxiv.org/abs/2609.35732)
- Anthropic, Claude Code hooks reference. [code.claude.com/docs/en/hooks](https://code.claude.com/docs/en/hooks)