# Feedback Bench 1.1: Coding Agents, Ranked by their users’ feedback Built 2026-10-19. Window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/. Built by Enterpret (https://www.enterpret.com), which runs the same feedback analysis on a company's own customer feedback. Feedback Bench 1.1 ranks 17 coding agents from what their users say in public: Reddit, X, G2 and Trustpilot. Every post is labelled against 63 criteria in 9 areas. In Feedback Bench 1.1, popularity and customer love count equally: the Feedback Score is 50 + 10 × z(½ z(Popularity) + ½ z(Customer love)), where each z-score is the distance from the average agent in standard deviations. 50 is an agent that is average on both. Popularity is how many people post about the agent (log share of users, from the least-discussed agent at 0 to the most-discussed at 1); Customer love is how those posts rate it against the category (0.5 = category norm). Every week since 2026-08-31 counts, and recent weeks count more: a week's weight halves every 4 weeks. Full method: [https://feedbackbench.com/coding-agents/method.md](https://feedbackbench.com/coding-agents/method.md). Also ranked: [personal agents](https://feedbackbench.com/index.md). ## Why this benchmark Lab benchmarks test an agent on a fixed set of tasks. Feedback Bench ranks 17 coding agents on what their users say about them in public. - **What it measures.** Large language models label each post against 63 criteria in 9 areas. Each agent gets a score on every criterion. The posts come from Reddit, X, G2 and Trustpilot. - **How it counts.** The unit is the user-week: one person, about one agent, in one week. Twenty posts by one person in one week count once. The Feedback Score gives equal weight to two parts: how many people post about the agent, and how they rate it against the category norm. Agents with few posts are pulled toward that norm. - **Sources and data.** Every number links to the posts behind it, quoted word for word, with a link to the original. The method and the data are public as Markdown, JSON and CSV. - **Coding agents.** Coding agents are the first category. The ranking updates every week. Personal agents are the second. ## Ranking Rank is the position by Feedback Score; the rank range column gives the ranks each agent reaches in 95% of bootstrap resamples. Criteria ranking high / low counts, of the 63 criteria, those where the agent clearly sits in the top / bottom half. Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. | Rank | Agent | Maker | Feedback Score | 95% interval | Rank range | Popularity | Customer love | Customer love 95% interval | Criteria ranking high / low | |---|---|---|---|---|---|---|---|---|---| | 1 | [Claude Code](https://feedbackbench.com/coding-agents/agents/claude-code.md) | Anthropic | 68.5 | 67.4–69.8 | 1–1 | 1.000 | 0.573 | 0.564–0.582 | 30 / 9 | | 2 | [Devin](https://feedbackbench.com/coding-agents/agents/devin.md) | Cognition | 64.2 | 62.4–65.8 | 2–4 | 0.638 | 0.611 | 0.595–0.626 | 12 / 2 | | 3 | [OpenCode](https://feedbackbench.com/coding-agents/agents/opencode.md) | Anomaly (open source) | 63.2 | 61.7–64.5 | 2–4 | 0.844 | 0.562 | 0.550–0.571 | 20 / 8 | | 4 | [Pi](https://feedbackbench.com/coding-agents/agents/pi.md) | Earendil Works (open source) | 62.4 | 60.7–63.8 | 2–4 | 0.641 | 0.596 | 0.582–0.610 | 16 / 1 | | 5 | [Amp](https://feedbackbench.com/coding-agents/agents/amp.md) | Amp | 56.0 | 54.2–57.8 | 5–6 | 0.482 | 0.578 | 0.563–0.593 | 11 / 1 | | 6 | [Factory](https://feedbackbench.com/coding-agents/agents/factory.md) | Factory | 55.1 | 52.8–57.2 | 5–7 | 0.475 | 0.572 | 0.553–0.591 | 9 / 0 | | 7 | [Cursor](https://feedbackbench.com/coding-agents/agents/cursor.md) | Anysphere | 52.9 | 51.4–54.4 | 6–7 | 0.802 | 0.487 | 0.474–0.500 | 6 / 20 | | 8 | [Zed](https://feedbackbench.com/coding-agents/agents/zed.md) | Zed Industries | 48.7 | 47.4–50.1 | 8–10 | 0.541 | 0.507 | 0.496–0.519 | 2 / 4 | | 9 | [GitHub Copilot](https://feedbackbench.com/coding-agents/agents/copilot.md) | GitHub | 48.1 | 46.5–49.9 | 8–11 | 0.557 | 0.499 | 0.485–0.514 | 2 / 7 | | 10 | [OpenAI Codex](https://feedbackbench.com/coding-agents/agents/codex.md) | OpenAI | 47.7 | 46.6–48.8 | 8–11 | 0.980 | 0.409 | 0.400–0.417 | 10 / 27 | | 11 | [Cline](https://feedbackbench.com/coding-agents/agents/cline.md) | Cline (open source) | 46.7 | 45.2–48.3 | 9–11 | 0.497 | 0.500 | 0.487–0.514 | 1 / 6 | | 12 | [Conductor](https://feedbackbench.com/coding-agents/agents/conductor.md) | Melty Labs | 43.3 | 42.4–44.3 | 12–13 | 0.305 | 0.512 | 0.504–0.521 | 1 / 0 | | 13 | [Kiro](https://feedbackbench.com/coding-agents/agents/kiro.md) | AWS | 41.6 | 40.4–43.1 | 12–15 | 0.404 | 0.478 | 0.467–0.490 | 0 / 4 | | 14 | [Google Antigravity](https://feedbackbench.com/coding-agents/agents/antigravity.md) | Google | 41.3 | 40.0–42.7 | 13–15 | 0.807 | 0.394 | 0.382–0.405 | 2 / 30 | | 15 | [Warp](https://feedbackbench.com/coding-agents/agents/warp.md) | Warp | 40.6 | 39.9–41.3 | 13–15 | 0.246 | 0.502 | 0.497–0.508 | 0 / 0 | | 16 | [Grok Build](https://feedbackbench.com/coding-agents/agents/grok-build.md) | xAI | 35.6 | 35.1–36.2 | 16–16 | 0.030 | 0.507 | 0.502–0.512 | 0 / 0 | | 17 | [Augment Code](https://feedbackbench.com/coding-agents/agents/augment.md) | Augment | 34.1 | 33.6–34.6 | 17–17 | 0.000 | 0.501 | 0.497–0.504 | 0 / 0 | ## Top quadrant Rule: popularity of at least 0.5 and customer love of at least 0.5. At the edge: the customer love 95% interval still includes 0.5. | Agent | Rank | Score | Popularity | Customer love | Customer love 95% interval | At the edge | |---|---|---|---|---|---|---| | Claude Code | 1 | 68.5 | 1.00 | 0.573 | 0.564–0.582 | no | | Devin | 2 | 64.2 | 0.64 | 0.611 | 0.595–0.626 | no | | OpenCode | 3 | 63.2 | 0.84 | 0.562 | 0.550–0.571 | no | | Pi | 4 | 62.4 | 0.64 | 0.596 | 0.582–0.610 | no | | Zed | 8 | 48.7 | 0.54 | 0.507 | 0.496–0.519 | yes | ## Week by week Weekly rank and Feedback Score; each week is scored with the same method on that week's posts alone. | Agent | Week of 2026-08-31 | Week of 2026-09-07 | Week of 2026-09-14 | Week of 2026-09-21 | Week of 2026-09-28 | Week of 2026-10-05 | Week of 2026-10-12 | |---|---|---|---|---|---|---|---| | Claude Code | rank 15 (13–15), score 39.9 | rank 8 (6–11), score 48.8 | rank 4 (4–7), score 55.6 | rank 1 (1–1), score 79.3 | rank 1 (1–1), score 78.2 | rank 1 (1–1), score 77.8 | rank 1 (1–2), score 70.2 | | Devin | rank 5 (3–7), score 55.7 | rank 2 (2–2), score 67.7 | rank 2 (1–3), score 67.4 | rank 2 (2–3), score 61.8 | rank 4 (3–5), score 56.3 | rank 4 (3–5), score 56.8 | rank 3 (3–5), score 58.9 | | OpenCode | rank 1 (1–1), score 76.3 | rank 1 (1–1), score 73.4 | rank 1 (1–2), score 69.4 | rank 3 (2–4), score 59.3 | rank 2 (2–3), score 62.1 | rank 2 (2–3), score 61.5 | rank 2 (1–2), score 68.5 | | Pi | rank 3 (2–5), score 59.1 | rank 3 (3–5), score 59.5 | rank 3 (2–3), score 63.6 | rank 4 (3–5), score 56.3 | rank 3 (2–4), score 59.8 | rank 3 (2–4), score 58.4 | rank 6 (4–7), score 54.7 | | Amp | rank 7 (5–10), score 51.6 | rank 7 (6–10), score 49.3 | rank 5 (4–9), score 53.4 | rank 6 (5–8), score 51.0 | rank 7 (6–11), score 49.7 | rank 5 (4–6), score 55.1 | rank 5 (3–6), score 55.9 | | Factory | rank 10 (8–11), score 48.1 | rank 12 (10–14), score 45.0 | rank 7 (5–10), score 51.6 | rank 10 (7–11), score 48.2 | rank 6 (5–8), score 51.5 | rank 7 (6–8), score 52.3 | rank 4 (3–6), score 56.8 | | Cursor | rank 4 (2–7), score 57.6 | rank 4 (3–5), score 59.2 | rank 6 (4–10), score 52.9 | rank 5 (4–7), score 52.7 | rank 5 (4–6), score 55.0 | rank 6 (4–7), score 53.9 | rank 7 (5–9), score 51.8 | | Zed | rank 9 (7–11), score 49.5 | rank 10 (8–12), score 47.0 | rank 8 (5–11), score 51.3 | rank 8 (6–10), score 49.3 | rank 9 (7–11), score 48.7 | rank 9 (8–9), score 48.8 | rank 8 (7–9), score 50.4 | | GitHub Copilot | rank 8 (5–11), score 51.3 | rank 6 (6–10), score 49.8 | rank 11 (6–12), score 49.4 | rank 7 (5–10), score 50.3 | rank 8 (6–11), score 49.1 | rank 8 (7–9), score 50.1 | rank 10 (9–12), score 47.7 | | OpenAI Codex | rank 2 (2–5), score 59.4 | rank 9 (6–12), score 47.1 | rank 12 (11–13), score 44.9 | rank 13 (12–15), score 43.4 | rank 14 (13–15), score 42.0 | rank 12 (10–13), score 45.2 | rank 9 (7–11), score 49.6 | | Cline | rank 11 (8–11), score 47.7 | rank 11 (8–14), score 45.9 | rank 10 (6–11), score 49.7 | rank 9 (6–11), score 48.8 | rank 10 (7–11), score 48.5 | rank 10 (10–12), score 45.8 | rank 11 (10–12), score 46.9 | | Conductor | rank 12 (12–14), score 43.4 | rank 13 (10–14), score 44.8 | rank 13 (12–13), score 42.8 | rank 14 (13–14), score 43.1 | rank 12 (11–13), score 44.6 | rank 13 (13–13), score 43.2 | rank 13 (13–13), score 43.2 | | Kiro | rank 14 (12–15), score 43.2 | rank 14 (11–15), score 44.2 | rank 14 (14–15), score 39.9 | rank 11 (10–12), score 46.5 | rank 13 (11–14), score 44.0 | rank 11 (10–12), score 45.5 | rank 12 (10–12), score 46.4 | | Google Antigravity | rank 6 (3–9), score 54.6 | rank 5 (3–5), score 56.0 | rank 9 (5–12), score 49.9 | rank 12 (10–14), score 45.5 | rank 11 (8–13), score 46.5 | rank 15 (13–16), score 40.4 | rank 15 (15–16), score 37.1 | | Warp | rank 13 (12–15), score 43.2 | rank 15 (14–15), score 42.2 | rank 15 (14–15), score 39.3 | rank 15 (14–15), score 41.8 | rank 15 (14–15), score 41.3 | rank 14 (14–15), score 41.8 | rank 14 (14–14), score 42.0 | | Grok Build | rank 16 (16–16), score 37.0 | rank 16 (16–16), score 36.8 | rank 16 (16–16), score 35.0 | rank 16 (16–16), score 37.5 | rank 16 (16–16), score 37.9 | rank 17 (17–17), score 34.4 | rank 17 (17–17), score 34.6 | | Augment Code | rank 17 (17–17), score 32.5 | rank 17 (17–17), score 33.5 | rank 17 (17–17), score 33.9 | rank 17 (17–17), score 35.2 | rank 17 (17–17), score 34.8 | rank 16 (15–16), score 38.9 | rank 16 (15–16), score 35.1 | ## Criteria 63 criteria, most discussed first, with the agent of highest customer love among agents with at least 30 rated user-weeks. | Criterion | Area | Rated user-weeks | Highest customer love | |---|---|---|---| | [Can do your kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | Quality of work | 15665 | Claude Code 0.698 | | [Plan worth the price](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | Pricing and limits | 13894 | OpenCode 0.664 | | [Usage quota lasts](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | Pricing and limits | 13190 | Pi 0.639 | | [Quality doesn't slip over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | Models | 6451 | Claude Code 0.693 | | [Fast responses](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | Reliability | 5313 | Claude Code 0.684 | | [Fair quota resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | Pricing and limits | 5196 | Claude Code 0.552 | | [Subagents work well](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | Quality of work | 4128 | Amp 0.556 | | [Clear display and settings](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | Interface | 3749 | Claude Code 0.587 | | [No cutoffs mid-task](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | Pricing and limits | 3494 | Claude Code 0.559 | | [Runs without crashing](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | Reliability | 3491 | Claude Code 0.613 | | [Plan terms don't change under you](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | Pricing and limits | 3348 | OpenCode 0.618 | | [Offers the models you want](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | Models | 3013 | OpenCode 0.645 | | [Few outages and errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | Reliability | 2472 | Amp 0.514 | | [Clear usage meter](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | Pricing and limits | 2227 | Claude Code 0.641 | | [MCP and plugins work](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | Setup | 1989 | Pi 0.583 | | [Follows instructions](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | Instructions and context | 1906 | OpenAI Codex 0.517 | | [Helpful support](https://feedbackbench.com/coding-agents/criteria/account.support.md) | Support and account | 1774 | Amp 0.612 | | [Auto mode picks the right model](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | Models | 1657 | Claude Code 0.574 | | [Stays in scope](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | Quality of work | 1539 | OpenCode 0.538 | | [Easy install and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | Setup | 1485 | Amp 0.540 | | [Good mobile and remote control](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | Interface | 1420 | Amp 0.559 | | [Compaction keeps what matters](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | Instructions and context | 1407 | Pi 0.543 | | [Doesn't get stuck](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | Quality of work | 1403 | OpenCode 0.566 | | [Runs long unattended](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | Quality of work | 1357 | Claude Code 0.557 | | [Good free tier](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | Pricing and limits | 1328 | Devin 0.548 | | [Effort setting works](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | Models | 1314 | Claude Code 0.580 | | [Follows project rules](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | Instructions and context | 1314 | Claude Code 0.535 | | [Updates don't break things](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | Reliability | 1222 | OpenCode 0.635 | | [Sensible permission prompts](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | Quality of work | 1216 | Claude Code 0.574 | | [Remembers across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | Instructions and context | 1185 | Claude Code 0.545 | | [Good frontend UI](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | Quality of work | 1109 | Claude Code 0.562 | | [Concise, clear replies](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | Quality of work | 1075 | OpenAI Codex 0.574 | | [No false safety refusals](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | Quality of work | 1074 | OpenAI Codex 0.636 | | [Good IDE integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | Setup | 1061 | Claude Code 0.587 | | [Clear pricing](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | Pricing and limits | 1055 | Claude Code 0.544 | | [Stays sharp in long sessions](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | Instructions and context | 1024 | OpenAI Codex 0.525 | | [Cloud sessions work](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | Interface | 1023 | Amp 0.590 | | [No unfair bans](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | Support and account | 1019 | OpenAI Codex 0.552 | | [Reliable session history](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | Interface | 1016 | Pi 0.559 | | [Code review finds real bugs](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | Checking the work | 952 | OpenAI Codex 0.585 | | [Charges are correct](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | Support and account | 936 | Cursor 0.555 | | [Own keys and local models work](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | Setup | 933 | Pi 0.544 | | [Your subscription works in other tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | Pricing and limits | 861 | OpenAI Codex 0.597 | | [Good browser and computer use](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | Quality of work | 856 | OpenAI Codex 0.593 | | [No risky actions unasked](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | Quality of work | 846 | Claude Code 0.563 | | [Prompt cache works](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | Pricing and limits | 827 | Pi 0.541 | | [Doesn't break working code](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | Quality of work | 767 | Claude Code 0.526 | | [Honest about 'done'](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | Checking the work | 738 | OpenAI Codex 0.565 | | [Finds the right files](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | Instructions and context | 661 | Cursor 0.545 | | [Tests its own work](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | Checking the work | 646 | Claude Code 0.563 | | [Easy to learn](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | Setup | 582 | Pi 0.537 | | [No surprise overage charges](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | Pricing and limits | 574 | OpenAI Codex 0.556 | | [Finishes the task](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | Quality of work | 553 | Claude Code 0.630 | | [Finds and fixes bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | Quality of work | 515 | OpenCode 0.513 | | [Useful plan mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | Quality of work | 512 | Claude Code 0.552 | | [Protects your data](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | Support and account | 486 | Claude Code 0.502 | | [Easy to review its changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | Checking the work | 439 | Cursor 0.567 | | [Honest pushback](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | Quality of work | 385 | Claude Code 0.526 | | [Handles git well](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | Quality of work | 376 | Claude Code 0.525 | | [Asks when unsure](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | Instructions and context | 322 | Claude Code 0.500 | | [Reads images and files](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | Instructions and context | 280 | Claude Code 0.552 | | [Easy to interrupt and steer mid-run](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | Interface | 257 | Cursor 0.534 | | [Doesn't game the checks](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | Quality of work | 160 | Google Antigravity 0.511 | ## Pages and data - Agents: [Claude Code](https://feedbackbench.com/coding-agents/agents/claude-code.md), [Devin](https://feedbackbench.com/coding-agents/agents/devin.md), [OpenCode](https://feedbackbench.com/coding-agents/agents/opencode.md), [Pi](https://feedbackbench.com/coding-agents/agents/pi.md), [Amp](https://feedbackbench.com/coding-agents/agents/amp.md), [Factory](https://feedbackbench.com/coding-agents/agents/factory.md), [Cursor](https://feedbackbench.com/coding-agents/agents/cursor.md), [Zed](https://feedbackbench.com/coding-agents/agents/zed.md), [GitHub Copilot](https://feedbackbench.com/coding-agents/agents/copilot.md), [OpenAI Codex](https://feedbackbench.com/coding-agents/agents/codex.md), [Cline](https://feedbackbench.com/coding-agents/agents/cline.md), [Conductor](https://feedbackbench.com/coding-agents/agents/conductor.md), [Kiro](https://feedbackbench.com/coding-agents/agents/kiro.md), [Google Antigravity](https://feedbackbench.com/coding-agents/agents/antigravity.md), [Warp](https://feedbackbench.com/coding-agents/agents/warp.md), [Grok Build](https://feedbackbench.com/coding-agents/agents/grok-build.md), [Augment Code](https://feedbackbench.com/coding-agents/agents/augment.md) - Criteria: one page per area and criterion under https://feedbackbench.com/coding-agents/criteria/ (list in [https://feedbackbench.com/coding-agents/llms.txt](https://feedbackbench.com/coding-agents/llms.txt)) - Articles, analysis separate from the data: [https://feedbackbench.com/coding-agents/articles.md](https://feedbackbench.com/coding-agents/articles.md) - All data as JSON: [https://feedbackbench.com/coding-agents/data/coding.json](https://feedbackbench.com/coding-agents/data/coding.json) - Ranking as CSV: [https://feedbackbench.com/coding-agents/data/rankings.csv](https://feedbackbench.com/coding-agents/data/rankings.csv) - Every agent × criterion as CSV: [https://feedbackbench.com/coding-agents/data/criteria.csv](https://feedbackbench.com/coding-agents/data/criteria.csv) - Top requests per agent and criterion as CSV: [https://feedbackbench.com/coding-agents/data/requests.csv](https://feedbackbench.com/coding-agents/data/requests.csv) # How Feedback Bench 1.1 works Window: every full Monday-to-Sunday week since 31 August 2026. It grows by one week each week. **Disclosure.** Feedback Bench is built and maintained by Enterpret. Enterpret sells feedback analytics to software companies. No vendor pays to be listed, removed or re-scored. ## 1. What this ranks Feedback Bench ranks AI agents by what their users say about them in public. Lab benchmarks test a model on fixed tasks. They do not show what it is like to pay for an agent, hit its limits and trust its work for a month. Users write about that in public every day. Each category, such as coding agents or personal agents, is ranked on its own: its agents are compared only with each other, on criteria written for that category. Every category uses the method on this page. Where a step differs by category, this page describes the category you are reading. Version 1.1 asks two questions about every agent and weighs them equally: - **Popularity**: how many people talk about the agent, on a log scale from the least-discussed agent (0) to the most-discussed (1). - **Customer love**: how well its users, speaking from their own use, rate it compared with the other agents, on the same criteria. The norm of its peers is 0.5. The **Feedback Score** puts both on a common scale (section 7). 50 is an agent that is average on both; one standard deviation is 10 points. Four rules shape the method: 1. Only stances from the author's own use count as ratings. 2. Each agent is compared with the other agents, never with its own users. 3. Popularity is on a log scale, and small samples are pulled toward the norm, so an agent needs volume to rank high. 4. One account counts once per agent, platform and week. A **post** is a Reddit post or comment, an X post, or a G2 or Trustpilot review. A **channel** is the platform it comes from. *Figure 1: From post to rank. Drawn on the [Method page](https://feedbackbench.com/coding-agents/#/method).* ## 2. Collecting and attributing posts Each category lists the channels where its users post, and collects every agent through the same kinds of routes: the agent's own spaces, posts that name it, and reviews of the product itself, never of a parent brand. Every coding agent is collected by the same four routes: 1. **Its official subreddits**, all posts and comments. 2. **Reddit posts and comments whose own text names the agent**, from any subreddit we collect and from one Reddit search per agent. 3. **X posts that mention the agent's official product handles**, not staff members' personal handles. OpenAI Codex has no product handle, so it uses an X search for "OpenAI Codex", "Codex CLI" and "Codex app". 4. **G2 and Trustpilot reviews** of the product, not of a parent brand. Reddit and X carry almost all of the data. Some agents lack a route: Grok Build has no subreddit or product handle, Amp has no subreddit, and Pi, Zed and Conductor have no review pages. Devin includes r/windsurf and the Windsurf handle, because Windsurf is now Devin. Google Antigravity includes r/GeminiCLI and @geminicli, because Antigravity CLI replaced Gemini CLI. **Attribution.** A post in an agent's own subreddit counts for that agent. Any other post counts only when the labeller (section 3) confirms that it is about the agent. A name match is not enough: "cursor" can be a mouse cursor and "amp" an amplifier. A post that names two agents counts for both. ## 3. Labelling The **labeller**, a large language model, labels every post with its category's **codebook**, v1.0: a written list of criteria, each with a definition and a boundary with its nearest neighbours. One model labels each week. From the week of 12 October 2026 a newer model labels the new weeks; earlier weeks keep their labels, and a post that arrives late is labelled by its own week's model, so every agent in a week is labelled the same way. For each post and agent it records: - **About**: whether the post refers to this agent. - **Judges**: whether the post takes a stance on the agent. A question or a news link can be about an agent without judging it. - **Criteria and polarity**: which criteria the stance names, and for each one praise, complaint or mixed. This is aspect-based sentiment, applied to product mechanisms (Hu and Liu 2004). A second pass by a large language model reads every judging post and marks whether its stance comes from the author's own use of the agent. A user who reports what the agent did, what it cost or how it fits their work speaks first-hand. A reaction to an announcement, a benchmark or a price change, a plan to try the agent, a relayed report and a promotion do not. Only first-hand stances count as ratings. Review sites that accept reviews only from verified customers carry fewer promotional reviews (Mayzlin, Dover and Chevalier 2014). Large language models now label text at scale for social science, with a written codebook as the instrument (Ziems et al. 2024). A **criterion** names one product mechanism that one team could own and one post can show, such as the usage quota. On the site each has a short label that names the outcome users want, such as "Quota lasts". The criteria sit in **areas** ordered along the user's path: from paying and limits to account and support. Each category's criteria came from its own posts: a large language model listed every concrete thing a stratified sample of judging posts praised or complained about, with no categories given, and grouped the items into criteria, as in inductive codebook development (MacQueen et al. 1998). Two large language models then assigned every item to one criterion independently, and criteria they could not tell apart were merged or redrawn. The technical reference lists every criterion of the category, with its definition and boundary. ## 4. The unit: the user-week One frustrated user can write twenty comments in one thread, so posts are not independent, and the unit of analysis must be the person, not the post (Altman and Bland 1997). The unit of measurement is the **user-week**: one user, on one platform, talking about one agent, in one Monday-to-Sunday week. For each criterion and area, a user-week adds up that user's labels that week: +1 for praise, −1 for complaint, 0 for mixed. A positive sum makes a **positive** user-week, a negative sum a **negative** one, and zero is not rated. Positive and negative user-weeks are **rated user-weeks**. Customer love counts them, never raw posts. A **user** is a Reddit or X account, or a named G2 reviewer. A Trustpilot review counts as its own user, because the collected reviews carry no reviewer name. **Recency weight.** Every week since 31 August counts, but recent weeks count more, because opinion of a product drifts as it changes (Koren 2010). G2 also weights older reviews less (G2 2026). Each user-week carries a weight: ``` weight = 0.5 ^ (age / 4) age = weeks before the latest full week ``` The latest full week weighs 1, a week 4 weeks earlier 0.5, and a week 8 weeks earlier 0.25: the weight halves every 4 weeks. Customer love, the peer norms and the area weights add up these weights. Popularity counts each user with the weight of their most recent week. The counts the site shows are not weighted. ## 5. Popularity Popularity counts the accounts that post in or about an agent's spaces: every user with at least one post that counts for the agent. A complaint counts the same as praise. An agent's **share of voice** is its distinct users, each weighted by their most recent week, divided by the same sum over all agents. Popularity puts that share on a log scale, from the least-discussed agent to the most-discussed: ``` Popularity(a) = (log S(a) − log S_min) / (log S_max − log S_min) ``` The log scale gives every tenfold increase in people the same step. The formula is the income index of the Human Development Index (Klugman, Rodríguez and Choi 2011). Popularity measures how many accounts post about an agent on the collected channels, not how many people use it. ## 6. Customer love Customer love measures how well an agent's users rate it compared with the other agents. Complaints outnumber praise across the category, so each agent is compared with its peers, not judged on its raw share of praise. It is computed for each area, then combined. **Step 1: the peer norm.** For each agent and each area or criterion, the **peer norm** is the positive share of weighted rated user-weeks of all the other agents, across all channels, smoothed by one praise and one complaint: (positive + 1) / (positive + negative + 2), Laplace's rule of succession (Gelman et al. 2013). An agent's own users never set its own bar. This matters most for the largest agents, whose users would otherwise make up much of the norm they are compared with. **Step 2: shrinkage.** A share from a handful of users is not reliable. Five praises out of seven is not evidence of excellence. So before we compare, we add 200 imaginary user-weeks that rate the agent exactly at its peer norm: ``` p(a, c) = (positive + 200 × peer norm) / (positive + negative + 200) ``` Take a norm of 40% positive. An agent with 5 positive and 2 negative user-weeks has a raw share of 71%. With the 200 imaginary user-weeks, 80 positive and 120 negative, its share is (5 + 80) / (7 + 200) = 41%: close to the norm, because 7 user-weeks say little. An agent with 3,500 positive and 1,500 negative moves only from 70% to 69%, because 200 imaginary user-weeks barely move 5,000 real ones. At 200 real user-weeks, an agent's own evidence and the norm count equally. This is a beta prior worth 200 user-weeks, and shrinkage of this kind improves estimates for small samples (Efron and Morris 1975). *Figure 2: How shrinkage pulls small samples to the norm. Drawn on the [Method page](https://feedbackbench.com/coding-agents/#/method).* **Step 3: the comparison.** The agent's shrunk share is compared with its peer norm on the log-odds scale: ``` lift(a, c) = logit p(a, c) − logit peer norm(a, c) love(a, c) = sigmoid(lift) ``` On the log-odds scale a move from 5% to 10% counts for more than a move from 45% to 50%. That matters, because area norms differ widely. A love of 0.5 means exactly the peer norm; above 0.5, users rate the agent above its peers. The site shows the same quantity for each criterion, for reading only. **Step 4: one number.** Customer love is the weighted mean of the area lifts, mapped back to 0 to 1: ``` Customer love(a) = sigmoid( Σ_area w_area × lift(a, area) ) w_area = the area's share of all weighted area ratings, all agents ``` The weights are the same for every agent, so no agent gains because its own users discuss its strengths. This is direct standardisation (Fleiss, Levin and Paik 2003). The weights follow how often users rate an area. The current weights and norms are in the technical reference at the end of this page. ## 7. The Feedback Score Each measure is first standardised across the agents in the category. A **z-score** is the number of standard deviations an agent sits above or below the average agent: ``` z_pop(a) = z-score of log share of voice z_love(a) = z-score of logit customer love Feedback Score(a) = 50 + 10 × z( ½ × z_pop(a) + ½ × z_love(a) ) ``` Popularity is standardised on the log scale (section 5), and customer love on the log-odds scale, where its lifts are measured (section 6). The average of the two z-scores is standardised once more, so the score has mean 50 and standard deviation 10 across the agents. Standardising first matters: a measure with a wider spread would otherwise dominate the composite at any nominal weight (Paruolo, Saisana and Saltelli 2013). Equal weights on standardised parts are the most common choice for a composite indicator when there is no basis for other weights (OECD and JRC 2008; Decancq and Lugo 2013), and unit weights on standardised measures are known to be robust (Dawes 1979). G2's software rankings also combine two components, a satisfaction score and a market presence score (G2 2026). Scores are relative to the category: 50 is its average agent, and each build standardises again. Every measured agent is ranked, with no minimum volume, once a full Monday-to-Sunday week after its launch is in the window. Until then a new agent is not shown, and its posts count toward no score, norm or weight, so its launch week moves no other agent. Posts from before an agent's launch do not count. A small agent needs volume to rank high: log popularity keeps it low, and shrinkage keeps its customer love near the norm until it has many ratings. *Figure 3: Popularity × customer love. Drawn on the [Method page](https://feedbackbench.com/coding-agents/#/method).* ## 8. Worked example: the top two Figure 4 works through the two agents at the top of the current build. For each, it shows every area's weight times lift, the sum that gives customer love, both z-scores and the Feedback Score. It also shows the customer love at which the two would swap places, with every other agent held fixed. *Figure 4: The top two, area by area. Drawn on the [Method page](https://feedbackbench.com/coding-agents/#/method).* ## 9. How firmly each position holds Every customer love value and every score carries a 95% **interval**, and every agent has a **rank range**. They show how much each result rests on a few voices. An agent whose customer love comes from many people holds its position firmly. **Bootstrap.** We resample each agent's users with replacement, 1,000 times, with seed 20260927 (Efron and Tibshirani 1993). A user's weeks move together, as in a cluster bootstrap (Field and Welsh 2007). Each resample recomputes norms, area weights, customer love, the standardisation and the score. Popularity counts every collected account, so it is held fixed. An agent's rank range runs from the 2.5th to the 97.5th percentile of its rank across the resamples, as Goldstein and Spiegelhalter (1996) recommend for league tables. The board shows plain positions; the rank ranges are in the standings table at the end of this page and in the data files. **Readings.** A criterion or area reads as the agent's **position**: its place among the agents with at least 30 rated user-weeks there, ordered by customer love, such as "3rd of 12". Its plausible places run from 1 plus the number of agents whose interval lies wholly above its own, to the field size minus the number whose interval lies wholly below. Requiring intervals not to overlap is a conservative test (Schenker and Gentleman 2001). It **ranks high** when both its position and the middle of that span are in the top half, and **ranks low** when both are in the bottom half. With fewer than 4 agents in the field it ranks neither. Under 30 rated user-weeks, unweighted, it reads **too few posts**. No correction for multiple comparisons is applied. ## 10. Sensitivity We rerun the full ranking under each variant below, each changing one choice: a one-at-a-time version of the sensitivity analysis Saisana, Saltelli and Tarantola (2005) propose for composite indicators. - **Combination**: min-max scaling, mean rank, a standardised geometric mean. - **Parameters**: shrinkage strength 50, 100 or 400; equal area weights; area weights without paying and limits; a recency half-life of 2 or 8 weeks, or no recency weight. - **Rules**: popularity from posts marked as about the agent only; a norm that includes the scored agent's own users. - **Sources**: Devin and Google Antigravity without their earlier sources. - **Bootstrap**: two bootstraps that also resample popularity. - **Weights**: popularity alone, customer love alone, and the version 1.0 formula. Figure 5 shows each agent's rank under every variant, and Kendall's tau for each variant against the published order (Kendall 1938), where 1 means identical. *Figure 5: Rank under each variant. Drawn on the [Method page](https://feedbackbench.com/coding-agents/#/method).* ## 11. Other outputs **Weekly scores.** Each week is also scored alone with the same method, without recency weights, on that week's users, norms, weights and standardisation. An agent with no posts in a week has no score that week. Figure 6 marks the model releases in the window. *Figure 6: Week by week, with model releases. Drawn on the [Method page](https://feedbackbench.com/coding-agents/#/method).* **Requests.** A separate pass by a large language model reads every counted post and returns each explicit ask to add or change something, in a neutral wording of at most 12 words, much as feature requests are mined from app reviews (Maalej and Nabil 2015). The wordings are grouped into themes within each criterion; a second pass then merges themes that ask for the same change under different criteria, so one ask counts once. A theme shows once at least 2 user-weeks ask for it. Requests do not enter the score. **Failure modes.** For each criterion with at least 30 complaining user-weeks, a large language model reads 150 complaint posts drawn at random and names 4 to 8 failure modes: recurring situations in which an agent fails its users there. A second model then puts every complaint post on that criterion in one mode, or in Other. Modes stay fixed once named; each week's new posts are sorted into them. A mode counts distinct user-weeks, per agent. Failure modes do not enter the score. **Quotes.** Pages quote whole public posts that clearly praise, or clearly complain about, the agent on one criterion. A second large language model confirms each quote before it shows. Links and personal details are removed and marked, such as [link]. **Briefs.** Each agent, area and criterion page carries a brief written by a large language model and labelled as interpretation. On an agent page, code first computes the findings: each area's and criterion's share of the agent's ratings against other agents, its position, the head-to-head counts, the weekly series and the levers. A published rule picks the reasons on each side, and their figures. For each reason, code gathers first-hand posts on that criterion; the model writes from these findings and posts only, a second pass edits the draft, and code rejects any text that cites a post outside its reason's set, quotes a passage that is not in the post, or uses a number that is not in the findings. Area and criterion pages follow the same pattern. Code computes the standing, every agent's position, lift and counts, the failure modes and the requests. The position rule picks the agents that rank high and low. The model quotes only praise from the agents that rank high and complaints from those that rank low, each confirmed by the quote check, and the same checks reject numbers outside the findings. ## 12. Limitations 1. **Labels.** Every label comes from a large language model applying one codebook. The model changed once, from the week of 12 October 2026. The newer model marks fewer posts as judging an agent, so weekly counts before and after that week differ in part because of the labeller. 2. **Who posts.** People who post are not a random sample of users (Ruths and Pfeffer 2014). Strong opinions and heavy users are over-represented (Hu, Pavlou and Zhang 2017), and early adopters tend to rate a new tool more warmly than later users (Li and Hitt 2008). 3. **Collection.** A Reddit search returns only its newest 1,000 or so results, and X feeds may be capped per day (Morstatter et al. 2013). Both bias popularity down for the busiest agents. 4. **Coordinated accounts.** Each account counts once per week, but many accounts acting together would each count (Luca and Zervas 2016). On Reddit, AutoModerator, deleted accounts and names ending in "bot" are dropped. Accounts that post far more than others about one agent, repeat the same text, or write in a product team's voice are reviewed by a large language model, and only those it marks as users count; vendor staff, automated accounts and promotion are left out. Report a suspected campaign with the challenge form at the end of this page. ## 13. Data and citation Every view on the site has a machine-readable twin: [llms.txt](llms.txt) for agents and scripts; [llms-full.txt](llms-full.txt) with the ranking, method and every agent page; [method.md](method.md) with every rule quoted from the build file; the full build as JSON, linked from llms.txt; and [rankings.csv](data/rankings.csv), [criteria.csv](data/criteria.csv) and [requests.csv](data/requests.csv). In the data files, popularity is `reach`, customer love is `regard` and user-weeks are `authorWeeks`. Scores change with each build, so cite the version and the build date shown in the technical reference: > Feedback Bench 1.1, Coding agents, built YYYY-MM-DD, https://feedbackbench.com/coding-agents/. Built by [Enterpret](https://www.enterpret.com). ## References - Altman, D. G. and Bland, J. M. (1997). Statistics notes: Units of analysis. *BMJ* 314(7098), 1874. - Dawes, R. M. (1979). The robust beauty of improper linear models in decision making. *American Psychologist* 34(7), 571–582. - Decancq, K. and Lugo, M. A. (2013). Weights in multidimensional indices of wellbeing: An overview. *Econometric Reviews* 32(1), 7–34. - Efron, B. and Morris, C. (1975). Data analysis using Stein's estimator and its generalizations. *Journal of the American Statistical Association* 70(350), 311–319. - Efron, B. and Tibshirani, R. J. (1993). *An Introduction to the Bootstrap*. Chapman and Hall. - Field, C. A. and Welsh, A. H. (2007). Bootstrapping clustered data. *Journal of the Royal Statistical Society, Series B* 69(3), 369–390. - Fleiss, J. L., Levin, B. and Paik, M. C. (2003). *Statistical Methods for Rates and Proportions*, 3rd ed. Wiley. - G2 (2026). Research scoring methodologies. - Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A. and Rubin, D. B. (2013). *Bayesian Data Analysis*, 3rd ed. CRC Press. - Goldstein, H. and Spiegelhalter, D. J. (1996). League tables and their limitations: Statistical issues in comparisons of institutional performance. *Journal of the Royal Statistical Society, Series A* 159(3), 385–443. - Hu, M. and Liu, B. (2004). Mining and summarizing customer reviews. *Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining*, 168–177. - Hu, N., Pavlou, P. A. and Zhang, J. (2017). On self-selection biases in online product reviews. *MIS Quarterly* 41(2), 449–471. - Kendall, M. G. (1938). A new measure of rank correlation. *Biometrika* 30(1–2), 81–93. - Klugman, J., Rodríguez, F. and Choi, H.-J. (2011). The HDI 2010: New controversies, old critiques. *Journal of Economic Inequality* 9(2), 249–288. - Koren, Y. (2010). Collaborative filtering with temporal dynamics. *Communications of the ACM* 53(4), 89–97. - Li, X. and Hitt, L. M. (2008). Self-selection and information role of online product reviews. *Information Systems Research* 19(4), 456–474. - Luca, M. and Zervas, G. (2016). Fake it till you make it: Reputation, competition, and Yelp review fraud. *Management Science* 62(12), 3412–3427. - Maalej, W. and Nabil, H. (2015). Bug report, feature request, or simply praise? On automatically classifying app reviews. *2015 IEEE 23rd International Requirements Engineering Conference*, 116–125. - MacQueen, K. M., McLellan, E., Kay, K. and Milstein, B. (1998). Codebook development for team-based qualitative analysis. *CAM Journal* 10(2), 31–36. - Mayzlin, D., Dover, Y. and Chevalier, J. (2014). Promotional reviews: An empirical investigation of online review manipulation. *American Economic Review* 104(8), 2421–2455. - Morstatter, F., Pfeffer, J., Liu, H. and Carley, K. M. (2013). Is the sample good enough? Comparing data from Twitter's Streaming API with Twitter's Firehose. *Proceedings of the International AAAI Conference on Web and Social Media* 7(1), 400–408. - OECD and JRC (2008). *Handbook on Constructing Composite Indicators: Methodology and User Guide*. OECD Publishing. - Paruolo, P., Saisana, M. and Saltelli, A. (2013). Ratings and rankings: Voodoo or science? *Journal of the Royal Statistical Society, Series A* 176(3), 609–634. - Ruths, D. and Pfeffer, J. (2014). Social media for large studies of behavior. *Science* 346(6213), 1063–1064. - Saisana, M., Saltelli, A. and Tarantola, S. (2005). Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators. *Journal of the Royal Statistical Society, Series A* 168(2), 307–323. - Schenker, N. and Gentleman, J. F. (2001). On judging the significance of differences by examining the overlap between confidence intervals. *The American Statistician* 55(3), 182–186. - Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z. and Yang, D. (2024). Can large language models transform computational social science? *Computational Linguistics* 50(1), 237–291. ## Earlier version Feedback Bench 1.0, the version published until 16 October 2026, used the same posts, labels and bootstrap. It combined the two measures as 100 × √(popularity × customer love), with popularity = log(1 + share / 0.5%) / log(1 + leader's share / 0.5%). Its customer love counted every stance and compared each agent with channel norms built from all agents, its own users included. --- ## Appendix A: formulas For agent `a`, criterion or area `c`: ``` User-week value v(u, w, c) = Σ over u's first-hand judging posts in week w of (+1 praise, −1 complaint, 0 mixed) on c positive if v > 0, negative if v < 0, not rated if v = 0 (a criterion's value also adds to its area's value) Recency weight r(w) = 0.5 ^ (age(w) / h) h = 4 weeks; age(w) = weeks from w to the latest full week pos and neg below sum r(w) over positive and negative user-weeks; n = pos + neg Share of voice S(a) = Σ_u r(latest week of u for a) / Σ_b Σ_u r(latest week of u for b) u = a user who counts for the agent Popularity R(a) = (log S(a) − log S_min) / (log S_max − log S_min) Peer norm π(a, c) = (pos(c) − pos(a, c) + 1) / (n(c) − n(a, c) + 2) all other agents, all channels Shrunk share p(a, c) = (pos(a, c) + m · π(a, c)) / (pos(a, c) + neg(a, c) + m) Lift L(a, c) = logit p(a, c) − logit π(a, c) Love on c sigmoid(L(a, c)) Area weight w(c) = n(c) / Σ_c' n(c') n = weighted rated user-weeks; over the areas, all agents Customer love G(a) = sigmoid( Σ_area w(area) · L(a, area) ) z-score z(x)(a) = (x(a) − mean of x over agents) / standard deviation of x over agents (divided by the number of agents) Feedback Score F(a) = 50 + 10 · z( ½ · z(log S)(a) + ½ · z(logit G)(a) ) Bootstrap for b in 1..1000: resample each agent's users with replacement (all their weeks together), recompute π, w, L, G, the z-scores and F with S fixed; rank the agents by F Intervals 2.5th and 97.5th percentiles of G, love on c and F across resamples; rank range: the order statistics at the 2.5th and 97.5th percentiles of rank Positions agents ordered by F; rank ranges are reported in the standings table and the data files ``` ## Appendix B: parameters | Parameter | Value | Reason | |---|---|---| | Window | From 2026-08-31 to the latest full Monday-to-Sunday week | Measurement began on 2026-08-31. Every full week since then counts. | | Unit | User-week, per platform | One account counts once per week. | | Excluded accounts | Reddit AutoModerator, deleted accounts, names ending in "bot"; reviewed accounts marked as vendor staff, automated or promotion; posts that share a referral, invitation or redeem code | Removes automated, unattributable and promotional posts, and posts by the vendor's own staff. | | Author review | 40 or more first-hand ratings of one agent, the same text 3 or more times, or 2 or more posts in a product team's voice | Heavy and repeated posters are checked before they count. | | Recency half-life | 4 weeks | Recent weeks decide most of the rank, while every week counts. | | New-agent embargo | 1 full week after launch | A launch week draws early enthusiasts and launch offers; a new agent is ranked only once a full week has passed. | | m, shrinkage strength | 200 weighted user-weeks | An agent's own evidence carries half the weight at 200 weighted rated user-weeks. | | Smoothing of norms | One praise and one complaint added | No norm of exactly 0% or 100%. | | Area weights | Each area's share of all weighted area ratings | The same for every agent. | | Combination | Equal weights on the z-scores of log share of voice and logit customer love | Section 7. | | Score scale | Mean 50, standard deviation 10 | 50 is an agent that is average on both measures. | | Bootstrap | 1,000 resamples of users, seed 20260927 | The fixed seed makes intervals reproduce exactly. | | Minimum for a reading | 30 rated user-weeks, unweighted | Below it the site shows "too few posts". | ## Technical reference (built 2026-10-19) Each rule below is quoted verbatim from the build file. Names: popularity is `reach` and customer love is `regard` in the data files (coding.json, rankings.csv, criteria.csv). ### Sources 2026-08-31 to the Sunday before 2026-10-19. Four channels, collected the same way for every agent: (1) the agent's official subreddits, posts and comments; (2) Reddit posts and comments whose own text names the agent, from any subreddit we collect and from a Reddit search per agent; (3) posts on X that mention the agent's official handles (for OpenAI Codex, which has no product handle, an X search for "OpenAI Codex", "Codex CLI" and "Codex app"); (4) G2 and Trustpilot reviews of the product. Parent-brand review pages and staff handles are excluded. Each post counts once per agent. ### Attribution A post in an agent's own subreddit counts for that agent. Any other post counts for an agent only when the classifier confirms it is about that agent; a post can count for several agents. ### Labels Every post is labelled by a large language model with the criteria codebook v1.0 (63 criteria in 9 areas): whether it is about the agent, whether it judges it, which criteria its stance names, and praise, complaint or mixed for each. On a pilot of 308 posts, a second large language model labelling independently agreed at kappa 0.75 on whether a post judges the agent, 0.73 overlap on its criteria, 0.78 on its areas, and 98% on polarity. Human annotators have not yet validated v1.0. 3 post(s) the classifier declined to label are left out. ### Unit A user-week: one user on one platform, about one agent, in one week. For each criterion it is positive if that user's praise outweighs their complaints that week, negative if the reverse. On Reddit, AutoModerator, [deleted] and names ending in 'bot' are not counted. Each review counts as its own user when the platform gives no name. ### Recency Every week in the window counts, and recent weeks count more. Each user-week is weighted 0.5^(age / 4), where age is the number of weeks before the latest full week (the week of 2026-10-12): that week weighs 1, 4 weeks earlier 0.5, 8 weeks earlier 0.25. Customer love, category baselines and area weights use weighted user-weeks; popularity counts each user with the weight of their most recent week. Counts shown (users, user-weeks, praise, complaint) are not weighted. Weekly scores score each week on its own, without weights. ### Popularity Share of voice = the agent's users across all four channels, each weighted by recency, / the sum over all agents. Popularity places share of voice on a log scale: (log share − log of the smallest share) / (log of the largest share − log of the smallest share), so the least-discussed agent is 0 and the most-discussed is 1. Popularity counts every user in the window, so it has no sampling interval. ### Criterion love For each criterion: the agent's peer norm is (positive + 1) / (positive + negative + 2) recency-weighted user-weeks of all other agents, across all channels. The agent's positive share is shrunk toward its peer norm with 200 imaginary weighted user-weeks and compared with it on the log-odds scale. Only first-hand stances count: a separate model pass marks whether each judging post speaks from the author's own use, and a stance that does not, or that the pass refused, is not a rating. 0.5 is the category norm. ### Customer love Customer love = the customer love scores of the 9 areas, averaged on the log-odds scale with weights equal to each area's share of the category's recency-weighted rated user-weeks. ### Feedback Score Feedback Bench 1.1: popularity and customer love count equally. Each is standardised across the agents in the category (z-score of log share of voice; z-score of customer love on the log-odds scale), the two z-scores are averaged, and the average is restated as a standard score: 50 is the average agent and 10 points is one standard deviation. ### Intervals 95% intervals and rank ranges from 1000 bootstrap resamples of users. Popularity is held fixed; customer love and the standardisation are recomputed in each resample. ### Eligibility An agent is ranked once 1 full week (Monday to Sunday) after its launch is in the window. Until then it is not shown, and its posts count toward no score, norm or weight, so its arrival moves no other agent. Posts written before an agent's launch do not count, and its weekly scores start in the week it is first ranked. Shrinkage keeps the customer love of agents with few rated user-weeks near the norm. ### Receipts Up to 5 praise and 5 complaint posts per area and 3 per criterion. Each is a whole post of 60 to 450 characters (700 for areas) with at least 8 words beyond handles and links, that judges the criterion, whose labels are all praise or all complaint, and that contains no slur. Posts that touch the fewest criteria come first, then posts of at most 280 characters, then the newest. A second large language model reads the first 3 candidates per slot and keeps a post only if it clearly praises, or clearly complains about, the agent on that criterion. Links and personal details removed by the source show as marks such as [link]. ### Caveat A subreddit is a proxy for the product. X volume is capped at 1,000 posts a day per feed; no feed in the window reached it on its median day. Labels are automatic. Criteria seen in few posts have wide intervals; those under 30 user-weeks for an agent are shown as too few posts. ### Authors An author is reviewed when, for one agent, they post 40 or more first-hand ratings in the window, post the same text 3 or more times, or write 2 or more posts in a product team's voice ("we shipped", "our users"). A large language model reads a sample of their posts and marks the author as a user, vendor staff, an automated account, or promotion (referral codes, ads for the author's own product, copy-pasted praise). Only users count: staff and promotion posts are left out for that agent, and an automated account is left out for every agent. An author whose posts the model declines to review is left out for that agent, as declined labels are. A post that shares a referral, invitation or redeem code is promotion and is left out, whoever wrote it. ### Levers For each area where an agent's customer love is below the category norm: the Feedback Score and rank it would have if that one area were at the norm (0.5) and everything else stayed the same, computed with the same formula as the ranking. An area's pull follows its weight, so an area many users of the category rate moves the score more than one they rarely mention. ### Head to head Posts that judge two agents. In each post, the agent whose stance (praise minus complaint across all its labels, the overall stance included) is higher is ahead; ties are left out. Shares count the posts where one is ahead, with a 95% Wilson interval. Where: the agent's own subreddit, the other's, or anywhere else. Pairs with at least 30 such posts are shown. ### Failure modes A separate pass sorts complaints into failure modes. For each criterion with at least 30 complaining user-weeks, a large language model reads 150 complaint posts drawn at random and names 4 to 8 failure modes; a second large language model then puts every complaint post on that criterion in one mode, or in Other when it fits none. Modes are fixed once named, and new posts are sorted into them each week. A mode counts distinct user-weeks with a complaint in it, per agent; across agents, one user-week per agent. Modes do not change the labels or the Feedback Score. Examples: up to 3 posts per mode from different users, chosen and checked as receipts are. ### Requests A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. ### Display rules - Reading of a criterion or area: its position. Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. Too few posts under 30 rated user-weeks. - Agent place: its position on Popularity and on Customer love among all 17 agents. The Feedback Score weighs the two equally. - Top quadrant: popularity of at least 0.5 and customer love of at least 0.5; at the edge when the customer love 95% interval still includes 0.5. - Rank is the plain position by Feedback Score. How sure each position is: the rank range, reported in the method and the data files. ### Parameters | Parameter | Value | |---|---| | scoreMean | 50 | | scoreSd | 10 | | priorStrength | 200 | | halfLifeWeeks | 4 | | embargoWeeks | 1 | | bootstrap | 1000 | | seed | 20260927 | ### Area weights in Customer love Each area's share of the category's rated user-weeks. | Area | Weight | |---|---| | Paying and limits | 0.306 | | Setting up and connecting | 0.052 | | Choosing models | 0.104 | | Instructing and context | 0.062 | | Doing the work | 0.247 | | Checking and finishing | 0.024 | | Interface and sessions | 0.065 | | Reliability and speed | 0.106 | | Account and support | 0.034 | ### Category baselines (positive share of user-weeks, %) | Code | Name | Baseline % | |---|---|---| | paying | Paying and limits | 25.6 | | setup | Setting up and connecting | 39.4 | | models | Choosing models | 26.4 | | context | Instructing and context | 36.5 | | work | Doing the work | 51.4 | | checking | Checking and finishing | 50.1 | | interface | Interface and sessions | 40.4 | | reliability | Reliability and speed | 16.3 | | account | Account and support | 10.8 | | limits.plan_value | How much use a plan's price buys | 40.2 | | limits.window_interrupts_work | Short rolling usage window blocks or interrupts work | 14.2 | | limits.burn_rate | Single prompt, model or effort level consumes disproportionate quota | 18.4 | | limits.allowance_change | Price, allowance or plan terms changed | 4.8 | | limits.reset_schedule | Quota reset timing and bonus or banked resets | 17.9 | | limits.usage_meter | Usage meter visibility and accuracy | 8.5 | | limits.prompt_cache | Prompt cache hits, misses and invalidation | 29.1 | | billing.overage_charges | Pay-as-you-go overage, fallback billing and spend caps | 5.7 | | billing.pricing_clarity | Pricing and plan terms stated clearly and consistently | 4.7 | | billing.free_tier | Free tier and free model availability and limits | 58.9 | | billing.subscription_portability | Using an existing subscription across tools | 41.0 | | setup.install_signin | Install, launch and sign-in | 18.5 | | setup.provider_byok_local | Connecting own API keys, local models and custom endpoints | 51.0 | | setup.extensions_mcp | MCP servers, plugins, skills and hooks | 49.9 | | setup.onboarding_docs | Onboarding, discoverability and documentation | 20.7 | | setup.ide_integration | IDE and editor integration | 50.1 | | models.catalog_access | Which models are offered on a plan and when | 24.2 | | models.routing_auto | Automatic model routing and fallback | 25.2 | | models.effort_control | Reasoning effort setting and its defaults | 47.2 | | models.quality_drift | Quality got worse or better over time | 24.5 | | context.instruction_files | Persistent project rules files are read and obeyed | 54.2 | | context.instruction_following | Direct in-prompt instructions and caps are followed | 26.0 | | context.clarifying_questions | Asks the user versus guessing | 39.9 | | context.long_context_decay | Output degrades as the context window fills | 17.4 | | context.compaction | Context compaction keeps what matters, cheaply and quickly | 36.8 | | context.session_memory | Memory and state carried across sessions | 44.6 | | context.codebase_retrieval | Finding the right files in the codebase | 44.6 | | context.attachments | Images, PDFs and file attachments as input | 36.3 | | work.capability | Can do the user's kind of task | 64.5 | | work.frontend_ui | Frontend and visual UI output | 44.5 | | work.bug_diagnosis | Diagnosing and fixing reported bugs | 68.1 | | work.regressions_introduced | Breaks existing code or reintroduces bugs | 6.8 | | work.scope_overreach | Does unrequested work or over-engineers | 5.3 | | work.stuck_loops | Spins, loops or gets stuck without progress | 4.3 | | work.premature_stop | Stops mid-task or answers instead of acting | 7.9 | | work.long_running_autonomy | Long unattended runs and goal/loop mode | 77.4 | | work.multi_agent_orchestration | Subagents, parallel agents and orchestrators | 64.8 | | work.reward_hacking | Games checks instead of fixing the problem | 2.2 | | work.destructive_actions | Risky or irreversible actions without confirmation | 22.3 | | work.git_workflow | Git commits, branches and sync | 34.5 | | work.computer_browser_use | Computer use and browser control | 52.9 | | work.safety_refusals | Safety filters block legitimate coding tasks | 15.3 | | work.permission_prompts | Tool approval prompts and autonomy modes | 23.0 | | work.plan_mode | Plan-before-edit mode | 60.2 | | work.response_verbosity | Length and clarity of replies, summaries and comments | 25.1 | | work.sycophancy_pushback | Caves to or argues with the user's judgement | 19.5 | | verify.false_completion | Claims work is done or fixed when it is not | 3.7 | | verify.self_testing | Builds, tests or runs its own changes | 62.9 | | verify.agent_code_review | Agent-performed code review finds real issues | 78.2 | | verify.change_review_ui | Reviewing and approving the agent's changes | 42.8 | | ui.display_settings | How the interface shows work, and what the user can configure | 34.3 | | ui.session_history | Saving, switching, resuming and rewinding sessions | 26.5 | | ui.interrupt_steer | Stopping and steering a running agent | 42.7 | | surfaces.remote_mobile | Mobile, remote-control and voice access | 56.6 | | surfaces.cloud_sessions | Cloud and remote sandbox execution | 55.4 | | rel.service_errors | Outages, server errors and capacity or rate errors | 6.8 | | rel.response_speed | Latency, throughput and fast mode | 26.7 | | rel.client_failures | Client crashes, freezes and failed tool execution | 7.2 | | rel.update_breakage | Updates break working setups | 11.8 | | account.support | Support, refunds and issue handling | 12.9 | | account.billing_errors | Wrong charges, failed payments and plan provisioning | 1.6 | | account.bans_restrictions | Account bans and access restrictions | 8.3 | | account.data_privacy | Data retention, training use and deployment isolation | 18.4 | ### Criteria 63 criteria in 9 areas. Codebook: every criterion names one product mechanism, visible in a single post. #### Paying and limits (`paying`) What do you pay, and how far does it get you? - [`limits.plan_value`](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) **How much use a plan's price buys.** Whether a plan's weekly or monthly allowance covers the user's normal work, and how its price compares with other plans or agents for the use it gives. Covers running out days early, never hitting the cap, missing tiers and value per dollar. Boundary: Not this: see limits.window_interrupts_work when the short multi-hour window blocks work. Not this: see limits.burn_rate when one prompt, model or effort level drained the quota. Not this: see limits.allowance_change when price or allowance changed over time. Not this: see billing.pricing_clarity when the information itself is unclear. - [`limits.window_interrupts_work`](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) **Short rolling usage window blocks or interrupts work.** The short rolling usage window (a cap of a few hours) blocks the user mid-task or mid-session: forced waits, killed sessions, or a long allowance left unused because the short window ran out. The post is about the window itself, not about what consumed it. Boundary: Not this: see limits.burn_rate when the point is that one prompt, model or effort level used a large share. Not this: see limits.plan_value for the weekly or monthly total. - [`limits.burn_rate`](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) **Single prompt, model or effort level consumes disproportionate quota.** How much quota a specific prompt, task, model version, effort level or feature consumes for the work done. Includes new versions using more than earlier ones and praise for efficient ones. The post names what consumed the quota. Boundary: Not this: see limits.window_interrupts_work when the post is about being blocked by the short window, not about the consumer. Not this: see limits.prompt_cache for cache misses. Not this: see work.multi_agent_orchestration when runaway subagents cause the burn. - [`limits.allowance_change`](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) **Price, allowance or plan terms changed.** The vendor changes a plan's price, quota, multipliers, included models or tiers over time: cuts, raises, removals, silent changes and changes that differ from what was announced. Boundary: Not this: see limits.plan_value for the current level with no change named. Not this: see limits.reset_schedule for when resets happen. - [`limits.reset_schedule`](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) **Quota reset timing and bonus or banked resets.** When and how usage resets happen: scheduled reset dates, surprise or bonus resets, banked resets, and paid resets. Covers resets that wipe saved allowance or land uselessly close to the normal reset. Boundary: Not this: see limits.allowance_change for changes to quota size. Not this: see limits.usage_meter when the reset is only displayed wrongly. - [`limits.usage_meter`](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) **Usage meter visibility and accuracy.** Whether the product shows how much quota and tokens were used and how much remains, per task and per model, and whether that figure is accurate. Covers meters that climb while idle and hidden token counts. Boundary: Not this: see billing.overage_charges for money actually charged. Not this: see limits.burn_rate when the meter is believed and the complaint is the amount consumed. - [`limits.prompt_cache`](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) **Prompt cache hits, misses and invalidation.** Whether prompt caching is kept across turns, resumes and provider routing, and how cache reads are priced. Covers cache drops that inflate usage. Boundary: Not this: see limits.burn_rate for consumption not attributed to caching. - [`billing.overage_charges`](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) **Pay-as-you-go overage, fallback billing and spend caps.** Usage spills from the plan into metered, on-demand or API billing, and whether spend caps and alerts work. Covers overage turning on silently and credit top-ups. Boundary: Not this: see account.billing_errors for wrong charges on the subscription itself. Not this: see models.routing_auto when the cause is auto-selection of a pricier model. - [`billing.pricing_clarity`](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) **Pricing and plan terms stated clearly and consistently.** Whether pricing pages, docs and plan descriptions clearly and consistently state costs, limits and what 'unlimited' means. Boundary: Not this: see limits.allowance_change when terms actually changed. Not this: see limits.usage_meter for in-product usage display. - [`billing.free_tier`](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) **Free tier and free model availability and limits.** Whether free models, free tiers and trial or promo access exist, how long they stay available, and whether their limits are usable. Boundary: Not this: see models.catalog_access for paid model availability. Not this: see rel.service_errors when free models return errors. - [`billing.subscription_portability`](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) **Using an existing subscription across tools.** Whether a paid subscription from one vendor can be used inside another agent or harness, or used as an API outside the vendor's own app. Boundary: Not this: see setup.provider_byok_local for API keys, local models and custom endpoints. #### Setting up and connecting (`setup`) How hard is it to install and connect? - [`setup.install_signin`](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) **Install, launch and sign-in.** Getting the agent installed, started and signed in on the user's platform: installers, updates on first run, login flows, browser redirects and switching accounts. Boundary: Not this: see setup.provider_byok_local for connecting own keys or local models. Not this: see rel.update_breakage when a later update breaks a working setup. - [`setup.provider_byok_local`](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) **Connecting own API keys, local models and custom endpoints.** Whether user-supplied providers work: bring-your-own-key, OpenAI-compatible endpoints, OpenRouter, and local servers such as Ollama or LM Studio. Boundary: Not this: see billing.subscription_portability for reusing a paid subscription. Not this: see rel.tool_call_errors for tool-format failures once connected. - [`setup.extensions_mcp`](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) **MCP servers, plugins, skills and hooks.** Whether external tools and extension points load, authenticate and refresh correctly: MCP servers, plugins, skills, hooks and third-party integrations. Boundary: Not this: see setup.ide_integration for editor extensions. Not this: see context.instruction_files for rules files. - [`setup.onboarding_docs`](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) **Onboarding, discoverability and documentation.** How easily a new user learns the product: quick-start guides, docs explaining features and modes, and how findable features are. Boundary: Not this: see billing.pricing_clarity for pricing docs. Not this: see ui.customization for settings complexity. - [`setup.ide_integration`](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) **IDE and editor integration.** How the agent works inside an editor or IDE, including extension support, editor-native features, autocomplete and feature parity across IDEs. Boundary: Not this: see verify.change_review_ui for diff and approval views. Not this: see ui.terminal_display for CLI rendering. #### Choosing models (`models`) Which models do you get, and do they hold up? - [`models.catalog_access`](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) **Which models are offered on a plan and when.** Whether models are available on the user's plan: same-day support for new releases, deprecations and removals, multi-provider choice, and models missing from a surface. Boundary: Not this: see billing.free_tier for free-model availability. Not this: see models.routing_auto for which model actually runs. - [`models.routing_auto`](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) **Automatic model routing and fallback.** Auto mode, routers or fallbacks pick, switch or hide the model used. This includes subagents running on a model other than the one requested, silent downgrades, and inability to exclude models. Boundary: Not this: see models.effort_control for reasoning level. Not this: see billing.overage_charges for the resulting charge alone. - [`models.effort_control`](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) **Reasoning effort setting and its defaults.** How the reasoning or effort level can be set, whether it stays set, and how outcomes differ by level, such as overthinking at high effort or failing at low effort. Boundary: Not this: see limits.burn_rate when the point is the quota share an effort level consumed. Not this: see work.scope_overreach for over-engineered output. - [`models.quality_drift`](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) **Quality got worse or better over time.** The post compares the agent or one of its models with an earlier time and says it got worse or better: 'nerfed', 'dumber since last week', 'better than at launch'. An explicit comparison with the past is required. Boundary: Not this: see general.unspecific for a verdict with no comparison over time. Not this: see work.capability for what it can do now. Not this: see limits.allowance_change for changed quotas or prices. #### Instructing and context (`context`) Does it follow your instructions and keep the right context? - [`context.instruction_files`](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) **Persistent project rules files are read and obeyed.** Whether the agent reads and follows persistent project instruction files (rules or agent markdown files) across turns. Boundary: Not this: see context.instruction_following for instructions in the current prompt. Not this: see context.session_memory for auto-written memory. - [`context.instruction_following`](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) **Direct in-prompt instructions and caps are followed.** Whether the agent executes explicit instructions, prohibitions and budgets given in the prompt, such as iteration caps and token caps. Boundary: Not this: see context.instruction_files for rules files. Not this: see work.sycophancy_pushback for evaluating the user's claims or opinions. Not this: see work.scope_overreach for unrequested extras. - [`context.clarifying_questions`](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) **Asks the user versus guessing.** Whether the agent stops to ask clarifying questions when it is blocked or the request is ambiguous, instead of inventing assumptions or workarounds. Boundary: Not this: see work.stuck_loops for repetitive failure without any question. - [`context.long_context_decay`](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) **Output degrades as the context window fills.** The agent forgets details, rules or its own statements as the session grows long. Boundary: Not this: see context.compaction for losses caused by summarisation. Not this: see context.session_memory for loss across separate sessions. - [`context.compaction`](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) **Context compaction keeps what matters, cheaply and quickly.** How automatic or manual compaction summarises history: what it drops, how long it takes, what it costs, and whether it is visible. Boundary: Not this: see context.long_context_decay for degradation without compaction. - [`context.session_memory`](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) **Memory and state carried across sessions.** Whether knowledge persists correctly between sessions: memory files, re-discovery cost at session start, stale memories, and leakage between sessions. Boundary: Not this: see ui.session_history for saving and resuming chat transcripts. Not this: see context.instruction_files for user-written rules. - [`context.codebase_retrieval`](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) **Finding the right files in the codebase.** How the agent searches and indexes a repository and pulls in relevant files. Covers over-reading on trivial edits and missing files in large or multi-repo codebases. Boundary: Not this: see limits.burn_rate for consumption not tied to file reading. - [`context.attachments`](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) **Images, PDFs and file attachments as input.** Whether attached images, screenshots, PDFs and other files are accepted and read. Boundary: Not this: see work.computer_browser_use for screen control. #### Doing the work (`work`) How does it behave while it works? - [`work.capability`](https://feedbackbench.com/coding-agents/criteria/work.capability.md) **Can do the user's kind of task.** Whether the agent, with its models, succeeds at the user's kind of task: its size, complexity, language or domain, including building a whole feature in one go or failing at it. The post names the task or its size but no narrower failure behaviour. Boundary: Not this: see work.frontend_ui for visual UI work and work.bug_diagnosis for fixing a reported bug. Not this: see models.quality_drift for a change over time. Not this: see the other work.* and verify.* leaves when a specific behaviour (looping, scope, false claims, breaking code) is named. Not this: see general.unspecific when no task is named. - [`work.frontend_ui`](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) **Frontend and visual UI output.** How the agent handles UI design, layout, visual taste and front-end component wiring. Boundary: Not this: see work.regressions_introduced for UI bugs reintroduced after fixes. - [`work.bug_diagnosis`](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) **Diagnosing and fixing reported bugs.** Whether the agent finds the root cause of a failing behaviour and fixes it without step-by-step guidance. Boundary: Not this: see verify.agent_code_review for reviewing code to find unknown issues. - [`work.regressions_introduced`](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) **Breaks existing code or reintroduces bugs.** Edits break previously working code, reintroduce fixed bugs, or cycle between introducing and fixing bugs. Boundary: Not this: see work.stuck_loops for repetition without code damage. Not this: see work.reward_hacking for deliberately gaming checks. - [`work.scope_overreach`](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) **Does unrequested work or over-engineers.** The agent edits outside the requested scope, adds unasked features, tests or runs, or builds overly complex solutions. Boundary: Not this: see work.destructive_actions for risky irreversible actions. Not this: see work.response_verbosity for padded text or comments. - [`work.stuck_loops`](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) **Spins, loops or gets stuck without progress.** The agent repeats attempts, loops endlessly or spirals on a problem without converging. Boundary: Not this: see work.premature_stop for stopping early. Not this: see context.clarifying_questions when the fix was to ask the user. - [`work.premature_stop`](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) **Stops mid-task or answers instead of acting.** The agent halts before finishing, or replies with explanation or a summary instead of performing the action. Boundary: Not this: see verify.false_completion for stopping while claiming the work is done. Not this: see limits.window_interrupts_work for stops caused by quota. - [`work.long_running_autonomy`](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) **Long unattended runs and goal/loop mode.** Whether the agent sustains hours-long autonomous work toward a goal, and whether a goal or loop mode exists. Boundary: Not this: see work.multi_agent_orchestration for coordination between agents. Not this: see surfaces.cloud_sessions for where the run executes. - [`work.multi_agent_orchestration`](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) **Subagents, parallel agents and orchestrators.** How the agent spawns, delegates to, monitors and coordinates subagents or parallel sessions. Covers polling loops and the wrong choice of subagents. Boundary: Not this: see models.routing_auto for the model a subagent runs on. Not this: see work.git_workflow for merge conflicts between sessions. - [`work.reward_hacking`](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) **Games checks instead of fixing the problem.** The agent disables linters, hardcodes for tests, edits benchmarks or defends bugs with tests so that checks pass without a real fix. Boundary: Not this: see verify.false_completion for plain false claims without manipulated checks. - [`work.destructive_actions`](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) **Risky or irreversible actions without confirmation.** The agent pushes, merges, deletes or wipes changes, or acts on the host outside its sandbox, without asking. Also covers praise for staying in bounds. Boundary: Not this: see work.permission_prompts for approval prompt frequency. Not this: see work.scope_overreach for harmless extra work. - [`work.git_workflow`](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) **Git commits, branches and sync.** How the agent handles commits, attribution, branches, pulls and pushes, and merge conflicts between concurrent agents or teammates. Boundary: Not this: see work.destructive_actions for data-destroying git commands. - [`work.computer_browser_use`](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) **Computer use and browser control.** Whether the agent operates desktop apps and browsers well, without taking over the user's windows. Boundary: Not this: see context.attachments for reading screenshots. - [`work.safety_refusals`](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) **Safety filters block legitimate coding tasks.** Refusals or safety classifiers stop benign or security-remediation work. Boundary: Not this: see work.permission_prompts for tool approval prompts. Not this: see models.routing_auto for safety-triggered model downgrades. - [`work.permission_prompts`](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) **Tool approval prompts and autonomy modes.** How often and when the agent asks permission to run tools or commands, and whether auto or bypass modes behave as configured. Boundary: Not this: see work.destructive_actions for acting without permission. Not this: see work.safety_refusals for content blocks. - [`work.plan_mode`](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) **Plan-before-edit mode.** Whether a read-only planning phase exists, produces useful plans, and actually prevents edits. Boundary: Not this: see work.task_completion for how well an approved plan is executed. - [`work.response_verbosity`](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) **Length and clarity of replies, summaries and comments.** How long and how readable the agent's prose and summaries are, and how many code comments and doc notes it writes. Boundary: Not this: see verify.change_review_ui for the volume of code changes. Not this: see work.scope_overreach for extra code features. - [`work.sycophancy_pushback`](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) **Caves to or argues with the user's judgement.** How the agent handles disagreement: agreeing with everything, reversing correct findings when challenged, failing to push back on wrong claims, or arguing when corrected. Boundary: Not this: see context.instruction_following for refusing a clear instruction. #### Checking and finishing (`checking`) Can you trust that the work is done? - [`verify.false_completion`](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) **Claims work is done or fixed when it is not.** The agent reports success, completion or a finished todo list that turns out to be untrue. Boundary: Not this: see work.reward_hacking when checks were manipulated. Not this: see verify.self_testing for whether checks were run at all. - [`verify.self_testing`](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) **Builds, tests or runs its own changes.** The post says whether the agent built, tested, ran or checked its own change before handing it back: it ran the test suite, skipped tests, did not run the app, or verified in proportion to the change. Boundary: Not this: see verify.false_completion when the agent claims success that did not happen. Not this: see verify.agent_code_review for a separate review mode or review agent. - [`verify.agent_code_review`](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) **Agent-performed code review finds real issues.** A review mode or review agent finds real defects in code or PRs, without re-flagging code that already passed or producing noise. Boundary: Not this: see work.bug_diagnosis for fixing a known symptom. Not this: see work.sycophancy_pushback for reversing findings under pressure. - [`verify.change_review_ui`](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) **Reviewing and approving the agent's changes.** Diff views, per-file approval, edit-acceptance prompts, and how manageable the amount of change is for a human reviewer. Boundary: Not this: see setup.ide_integration for general editor features. Not this: see work.response_verbosity for prose length. #### Interface and sessions (`interface`) How do you operate and steer it? - [`ui.display_settings`](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) **How the interface shows work, and what the user can configure.** How the CLI, TUI, IDE panel or app shows the agent's activity (tool calls, diffs, logs, subagent progress, scrolling, fonts, layout) and which settings, keybindings, themes and presets the user can change. Boundary: Not this: see limits.usage_meter for usage displays. Not this: see ui.session_history for saving and resuming sessions. Not this: see models.effort_control for reasoning-effort settings. - [`ui.session_history`](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) **Saving, switching, resuming and rewinding sessions.** Whether chats and threads persist, can be switched and resumed with their state intact, and can be rewound or reverted. Boundary: Not this: see context.session_memory for knowledge carried into a new session. - [`ui.interrupt_steer`](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) **Stopping and steering a running agent.** Whether the user can stop, interrupt or send guidance mid-run and have the agent respect it and continue. Boundary: Not this: see work.premature_stop for the agent stopping on its own. - [`surfaces.remote_mobile`](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) **Mobile, remote-control and voice access.** Driving or monitoring agents from a phone, another device or by voice, including approvals from the mobile client. Boundary: Not this: see surfaces.cloud_sessions for where the agent executes. - [`surfaces.cloud_sessions`](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) **Cloud and remote sandbox execution.** Agents run in hosted or remote environments. Covers persistence when the laptop closes, access to local files, session caps, and start or resume time. Boundary: Not this: see billing.overage_charges for how cloud runs are billed. Not this: see work.long_running_autonomy for run behaviour. #### Reliability and speed (`reliability`) Does it stay up and respond quickly? - [`rel.service_errors`](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) **Outages, server errors and capacity or rate errors.** Backend unavailability, 5xx or connection errors, capacity errors, and transient 429 errors that do not reflect the user's quota. Boundary: Not this: see limits.window_interrupts_work and limits.allowance_size for plan quota being reached. Not this: see rel.tool_call_errors for harness tool failures. - [`rel.response_speed`](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) **Latency, throughput and fast mode.** How quickly the agent responds and completes work, including paid or fast modes and whether they actually deliver speed. Boundary: Not this: see rel.service_errors for failures. Not this: see rel.client_crash_resources for local slowness from resource use. - [`rel.client_failures`](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) **Client crashes, freezes and failed tool execution.** The local app, CLI or extension crashes, freezes, leaks memory or uses too much CPU, or fails to execute the model's tool calls and shell commands. Boundary: Not this: see rel.service_errors for backend outages and server errors. Not this: see rel.update_breakage when a specific update caused it. - [`rel.update_breakage`](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) **Updates break working setups.** New releases regress features, break compatibility or change behaviour unexpectedly. Also covers praise for smooth migrations. Boundary: Not this: see limits.allowance_change for quota changes delivered in an update. #### Account and support (`account`) How does the vendor treat your account? - [`account.support`](https://feedbackbench.com/coding-agents/criteria/account.support.md) **Support, refunds and issue handling.** Reaching a human, getting refunds and resolutions, and the vendor's responsiveness to bug reports and community issues. Boundary: Not this: see account.billing_errors for the underlying wrong charge. - [`account.billing_errors`](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) **Wrong charges, failed payments and plan provisioning.** Payments are charged incorrectly, plans are not provisioned after payment, proration errors occur, or payments fail. Boundary: Not this: see billing.overage_charges for metered overage. Not this: see account.support for how the vendor handled the error. - [`account.bans_restrictions`](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) **Account bans and access restrictions.** The vendor suspends, bans or restricts accounts, for example for heavy usage or regional embargoes. Boundary: Not this: see limits.allowance_size for normal quota lockouts. - [`account.data_privacy`](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) **Data retention, training use and deployment isolation.** Whether prompts and code are retained or used for training, and whether private or air-gapped deployment is available. Boundary: Not this: see billing.free_tier for free-model availability itself. # Claude Code (Anthropic) Feedback Bench, coding agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/#/agent/claude-code | Measure | Value | |---|---| | Rank | 1 of 17 (rank range 1–1) | | Feedback Score | 68.5 (95% interval 67.4–69.8) | | Popularity | 1.000 (share of voice 31.18%) | | Customer love | 0.573 (95% interval 0.564–0.582) | | Place | Popularity 1st of 17 × Customer love 4th of 17 | | Top quadrant | yes | | Users | 46238 | | Posts counted | 144146 | | Posts that judge the agent | 40710 | | Criteria ranking high / low | 30 / 9 of 63 | ## Top requests What users ask to add or change, most asked first. 6070 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. | Rank | Request | Criterion | User-weeks | Posts | |---|---|---|---|---| | 1 | One-time immediate usage reset | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 288 | 318 | | 2 | Additional bonus usage resets | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 129 | 144 | | 3 | Higher overall usage limits | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 97 | 100 | | 4 | Banked resets usable on demand | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 88 | 90 | | 5 | Remove the 5-hour usage window | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | 61 | 64 | | 6 | Stop silently degrading model quality | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | 59 | 63 | | 7 | Higher-priced tier above current top plan | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 48 | 51 | | 8 | Shorter, less verbose responses by default | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | 48 | 48 | | 9 | Usage reset for model launches and events | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 46 | 49 | | 10 | Compensation reset after outages or bugs | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 46 | 47 | | 11 | Restore previous usage limits | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | 43 | 44 | | 12 | Higher allowance on top-tier plans | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 43 | 43 | ## Ranks highest on Can do the user's kind of task (1st of 13), Quality got worse or better over time (1st of 7), Usage meter visibility and accuracy (1st of 6) ## Ranks lowest on Safety filters block legitimate coding tasks (5th of 5), Cloud and remote sandbox execution (4th of 5), Using an existing subscription across tools (7th of 8) Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. ## All 63 criteria Criterion love: 0.5 is the category norm. n: rated user-weeks. ### Paying and limits: 7th of 13 (customer love 0.525, n 9521) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Single prompt, model or effort level consumes disproportionate quota](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | 5th of 12 | 0.548 | 0.524–0.570 | 3660 | 708 | 2952 | | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 4th of 12 | 0.539 | 0.520–0.558 | 3619 | 1472 | 2147 | | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | 1st of 6 | 0.559 | 0.517–0.601 | 1207 | 187 | 1020 | | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 1st of 5 | 0.552 | 0.520–0.587 | 1165 | 264 | 901 | | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | 2nd of 7 | 0.565 | 0.490–0.635 | 697 | 41 | 656 | | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | 1st of 6 | 0.641 | 0.586–0.696 | 537 | 74 | 463 | | [Prompt cache hits, misses and invalidation](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | 3rd of 4 | 0.497 | 0.454–0.540 | 373 | 107 | 266 | | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | 1st of 6 | 0.544 | 0.471–0.623 | 224 | 12 | 212 | | [Pay-as-you-go overage, fallback billing and spend caps](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | 3rd of 4 | 0.471 | 0.424–0.533 | 165 | 8 | 157 | | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 7th of 8 | 0.453 | 0.424–0.485 | 157 | 44 | 113 | | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | 6th of 7 | 0.470 | 0.442–0.498 | 111 | 52 | 59 | ### Setting up and connecting: 2nd of 12 (customer love 0.603, n 1232) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | 2nd of 9 | 0.576 | 0.544–0.608 | 591 | 342 | 249 | | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | 1st of 7 | 0.587 | 0.556–0.619 | 218 | 139 | 79 | | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | 4th of 8 | 0.524 | 0.487–0.562 | 215 | 48 | 167 | | [Onboarding, discoverability and documentation](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | 3rd of 7 | 0.510 | 0.469–0.553 | 129 | 28 | 101 | | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | 3rd of 9 | 0.517 | 0.492–0.543 | 109 | 63 | 46 | ### Choosing models: 1st of 12 (customer love 0.672, n 2859) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | 1st of 7 | 0.693 | 0.669–0.716 | 2003 | 686 | 1317 | | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | 1st of 4 | 0.580 | 0.547–0.613 | 436 | 238 | 198 | | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | 1st of 8 | 0.574 | 0.537–0.613 | 381 | 122 | 259 | | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 5th of 11 | 0.558 | 0.523–0.592 | 195 | 59 | 136 | ### Instructing and context: 1st of 11 (customer love 0.562, n 3052) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Direct in-prompt instructions and caps are followed](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | 2nd of 5 | 0.515 | 0.474–0.550 | 754 | 203 | 551 | | [Persistent project rules files are read and obeyed](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | 1st of 5 | 0.535 | 0.495–0.576 | 700 | 391 | 309 | | [Context compaction keeps what matters, cheaply and quickly](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | 2nd of 5 | 0.541 | 0.501–0.579 | 658 | 251 | 407 | | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | 1st of 6 | 0.545 | 0.505–0.583 | 621 | 293 | 328 | | [Output degrades as the context window fills](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | 3rd of 5 | 0.502 | 0.453–0.555 | 450 | 73 | 377 | | [Finding the right files in the codebase](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | 2nd of 5 | 0.522 | 0.487–0.557 | 220 | 106 | 114 | | [Asks the user versus guessing](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | 1st of 2 | 0.500 | 0.465–0.536 | 133 | 54 | 79 | | [Images, PDFs and file attachments as input](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | 1st of 4 | 0.552 | 0.520–0.590 | 73 | 38 | 35 | ### Doing the work: 4th of 14 (customer love 0.588, n 9436) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Can do the user's kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | 1st of 13 | 0.698 | 0.682–0.715 | 4998 | 3750 | 1248 | | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | 2nd of 10 | 0.553 | 0.524–0.583 | 1618 | 1081 | 537 | | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | 3rd of 3 | 0.380 | 0.340–0.424 | 729 | 130 | 599 | | [Safety filters block legitimate coding tasks](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | 5th of 5 | 0.341 | 0.304–0.384 | 588 | 50 | 538 | | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | 1st of 5 | 0.574 | 0.526–0.623 | 464 | 125 | 339 | | [Does unrequested work or over-engineers](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | 5th of 5 | 0.429 | 0.368–0.496 | 459 | 17 | 442 | | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | 1st of 8 | 0.557 | 0.519–0.596 | 443 | 359 | 84 | | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | 1st of 5 | 0.563 | 0.512–0.620 | 385 | 98 | 287 | | [Frontend and visual UI output](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | 1st of 5 | 0.562 | 0.528–0.599 | 353 | 180 | 173 | | [Spins, loops or gets stuck without progress](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | 3rd of 6 | 0.497 | 0.431–0.573 | 257 | 11 | 246 | | [Caves to or argues with the user's judgement](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | 1st of 2 | 0.526 | 0.471–0.599 | 235 | 45 | 190 | | [Breaks existing code or reintroduces bugs](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | 1st of 5 | 0.526 | 0.462–0.600 | 218 | 17 | 201 | | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | 4th of 4 | 0.454 | 0.423–0.487 | 177 | 77 | 100 | | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | 1st of 3 | 0.525 | 0.484–0.568 | 168 | 63 | 105 | | [Diagnosing and fixing reported bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | 2nd of 4 | 0.504 | 0.468–0.539 | 152 | 101 | 51 | | [Plan-before-edit mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | 1st of 5 | 0.552 | 0.521–0.581 | 141 | 98 | 43 | | [Stops mid-task or answers instead of acting](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | 1st of 3 | 0.630 | 0.562–0.710 | 118 | 22 | 96 | | [Games checks instead of fixing the problem](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | 3rd of 3 | 0.455 | 0.444–0.465 | 63 | 0 | 63 | ### Checking and finishing: 7th of 9 (customer love 0.490, n 1143) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Agent-performed code review finds real issues](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | 4th of 4 | 0.441 | 0.394–0.490 | 402 | 292 | 110 | | [Builds, tests or runs its own changes](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | 1st of 3 | 0.563 | 0.516–0.607 | 399 | 258 | 141 | | [Claims work is done or fixed when it is not](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | 4th of 4 | 0.431 | 0.369–0.518 | 324 | 9 | 315 | | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | 5th of 5 | 0.447 | 0.417–0.477 | 140 | 45 | 95 | ### Interface and sessions: 5th of 14 (customer love 0.537, n 1823) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 1st of 12 | 0.587 | 0.555–0.617 | 924 | 389 | 535 | | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | 2nd of 7 | 0.550 | 0.515–0.583 | 363 | 231 | 132 | | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | 3rd of 6 | 0.506 | 0.469–0.547 | 314 | 87 | 227 | | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | 4th of 5 | 0.426 | 0.390–0.460 | 279 | 129 | 150 | | [Stopping and steering a running agent](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | 3rd of 3 | 0.473 | 0.452–0.494 | 55 | 15 | 40 | ### Reliability and speed: 2nd of 13 (customer love 0.637, n 1452) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | 1st of 10 | 0.684 | 0.655–0.710 | 555 | 266 | 289 | | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | 1st of 11 | 0.613 | 0.548–0.670 | 468 | 57 | 411 | | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | 4th of 6 | 0.497 | 0.449–0.549 | 269 | 18 | 251 | | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | 6th of 7 | 0.460 | 0.416–0.506 | 208 | 19 | 189 | ### Account and support: 9th of 11 (customer love 0.451, n 1028) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Account bans and access restrictions](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | 3rd of 5 | 0.510 | 0.435–0.584 | 416 | 34 | 382 | | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | 7th of 8 | 0.416 | 0.375–0.462 | 398 | 31 | 367 | | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | 4th of 5 | 0.452 | 0.371–0.576 | 208 | 2 | 206 | | [Data retention, training use and deployment isolation](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | 1st of 4 | 0.502 | 0.466–0.546 | 146 | 27 | 119 | # Devin (Cognition) Feedback Bench, coding agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/#/agent/devin | Measure | Value | |---|---| | Rank | 2 of 17 (rank range 2–4) | | Feedback Score | 64.2 (95% interval 62.4–65.8) | | Popularity | 0.638 (share of voice 2.85%) | | Customer love | 0.611 (95% interval 0.595–0.626) | | Place | Popularity 7th of 17 × Customer love 1st of 17 | | Top quadrant | yes | | Users | 4797 | | Posts counted | 9730 | | Posts that judge the agent | 2583 | | Criteria ranking high / low | 12 / 2 of 63 | ## Top requests What users ask to add or change, most asked first. 753 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. | Rank | Request | Criterion | User-weeks | Posts | |---|---|---|---|---| | 1 | Mid-priced tier between existing plans | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 33 | 40 | | 2 | Higher overall usage limits | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 21 | 27 | | 3 | Official mobile app | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | 21 | 21 | | 4 | Access or trial of max plan | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 11 | 11 | | 5 | Keep free models available permanently | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | 11 | 11 | | 6 | Model choice in fusion mode | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 11 | 11 | | 7 | Auditable reversible memory changes | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | 10 | 10 | | 8 | Cost per completed task metrics | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | 10 | 10 | | 9 | Free trial of paid plans | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | 9 | 10 | | 10 | Keep free model access unlimited | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | 9 | 10 | | 11 | Mobile simulators and Mac VM for testing | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | 9 | 9 | | 12 | Published exact usage limits per plan | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | 8 | 9 | ## Ranks highest on Can do the user's kind of task (2nd of 13), Automatic model routing and fallback (3rd of 8), Single prompt, model or effort level consumes disproportionate quota (2nd of 12) ## Ranks lowest on Latency, throughput and fast mode (7th of 10), Client crashes, freezes and failed tool execution (8th of 11) Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. ## All 63 criteria Criterion love: 0.5 is the category norm. n: rated user-weeks. ### Paying and limits: 3rd of 13 (customer love 0.637, n 661) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 2nd of 12 | 0.556 | 0.529–0.584 | 318 | 164 | 154 | | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | 1st of 7 | 0.548 | 0.520–0.577 | 156 | 113 | 43 | | [Single prompt, model or effort level consumes disproportionate quota](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | 2nd of 12 | 0.570 | 0.535–0.607 | 134 | 44 | 90 | | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | 3rd of 6 | 0.483 | 0.461–0.507 | 52 | 3 | 49 | | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | 4th of 6 | 0.483 | 0.464–0.513 | 40 | 1 | 39 | | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 5th of 8 | 0.492 | 0.470–0.512 | 40 | 14 | 26 | | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | 4th of 6 | 0.497 | 0.470–0.532 | 38 | 3 | 35 | | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | 3rd of 7 | 0.531 | 0.473–0.599 | 36 | 3 | 33 | | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | Too few posts | 0.507 | 0.491–0.524 | 15 | 4 | 11 | | [Pay-as-you-go overage, fallback billing and spend caps](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | Too few posts | 0.492 | 0.486–0.497 | 9 | 0 | 9 | | [Prompt cache hits, misses and invalidation](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | Too few posts | 0.503 | 0.498–0.509 | 3 | 2 | 1 | ### Setting up and connecting: 7th of 12 (customer love 0.500, n 96) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | Too few posts | 0.513 | 0.492–0.536 | 29 | 9 | 20 | | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | Too few posts | 0.504 | 0.489–0.517 | 23 | 12 | 11 | | [Onboarding, discoverability and documentation](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | Too few posts | 0.500 | 0.487–0.517 | 18 | 4 | 14 | | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | Too few posts | 0.507 | 0.497–0.517 | 17 | 11 | 6 | | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | Too few posts | 0.492 | 0.481–0.501 | 12 | 3 | 9 | ### Choosing models: 2nd of 12 (customer love 0.635, n 141) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 2nd of 11 | 0.600 | 0.568–0.632 | 59 | 44 | 15 | | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | 2nd of 7 | 0.551 | 0.526–0.576 | 39 | 26 | 13 | | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | 3rd of 8 | 0.547 | 0.523–0.571 | 37 | 23 | 14 | | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | Too few posts | 0.498 | 0.490–0.506 | 11 | 5 | 6 | ### Instructing and context: 3rd of 11 (customer love 0.522, n 59) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | Too few posts | 0.513 | 0.498–0.528 | 20 | 12 | 8 | | [Finding the right files in the codebase](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | Too few posts | 0.503 | 0.497–0.510 | 11 | 6 | 5 | | [Direct in-prompt instructions and caps are followed](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | Too few posts | 0.510 | 0.499–0.524 | 9 | 5 | 4 | | [Output degrades as the context window fills](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | Too few posts | 0.499 | 0.490–0.511 | 7 | 1 | 6 | | [Persistent project rules files are read and obeyed](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | Too few posts | 0.497 | 0.489–0.504 | 4 | 1 | 3 | | [Context compaction keeps what matters, cheaply and quickly](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | Too few posts | 0.502 | 0.497–0.509 | 4 | 2 | 2 | | [Asks the user versus guessing](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | Too few posts | 0.496 | 0.492–0.500 | 3 | 0 | 3 | | [Images, PDFs and file attachments as input](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | Too few posts | 0.500 | 0.496–0.505 | 2 | 1 | 1 | ### Doing the work: 1st of 14 (customer love 0.670, n 543) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Can do the user's kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | 2nd of 13 | 0.638 | 0.611–0.662 | 396 | 337 | 59 | | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | 4th of 10 | 0.512 | 0.490–0.534 | 76 | 50 | 26 | | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | 2nd of 8 | 0.537 | 0.520–0.553 | 49 | 47 | 2 | | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | Too few posts | 0.495 | 0.484–0.504 | 16 | 6 | 10 | | [Spins, loops or gets stuck without progress](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | Too few posts | 0.521 | 0.488–0.562 | 15 | 2 | 13 | | [Frontend and visual UI output](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | Too few posts | 0.500 | 0.489–0.512 | 13 | 6 | 7 | | [Diagnosing and fixing reported bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | Too few posts | 0.515 | 0.507–0.524 | 12 | 12 | 0 | | [Does unrequested work or over-engineers](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | Too few posts | 0.530 | 0.495–0.572 | 10 | 3 | 7 | | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | Too few posts | 0.500 | 0.490–0.513 | 9 | 2 | 7 | | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | Too few posts | 0.492 | 0.485–0.497 | 9 | 0 | 9 | | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | Too few posts | 0.508 | 0.499–0.519 | 7 | 4 | 3 | | [Stops mid-task or answers instead of acting](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | Too few posts | 0.496 | 0.492–0.499 | 6 | 0 | 6 | | [Breaks existing code or reintroduces bugs](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | Too few posts | 0.506 | 0.496–0.525 | 4 | 1 | 3 | | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | Too few posts | 0.504 | 0.498–0.511 | 3 | 2 | 1 | | [Plan-before-edit mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | Too few posts | 0.504 | 0.500–0.509 | 3 | 3 | 0 | | [Games checks instead of fixing the problem](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | | [Safety filters block legitimate coding tasks](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | Too few posts | 0.506 | 0.500–0.518 | 1 | 1 | 0 | | [Caves to or argues with the user's judgement](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Checking and finishing: 2nd of 9 (customer love 0.526, n 52) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Agent-performed code review finds real issues](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | Too few posts | 0.503 | 0.479–0.523 | 24 | 20 | 4 | | [Builds, tests or runs its own changes](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | Too few posts | 0.509 | 0.498–0.519 | 16 | 13 | 3 | | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | Too few posts | 0.496 | 0.489–0.503 | 8 | 2 | 6 | | [Claims work is done or fixed when it is not](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | Too few posts | 0.495 | 0.491–0.498 | 7 | 0 | 7 | ### Interface and sessions: 1st of 14 (customer love 0.598, n 176) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | 2nd of 5 | 0.569 | 0.545–0.593 | 97 | 80 | 17 | | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 6th of 12 | 0.501 | 0.479–0.522 | 51 | 17 | 34 | | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | 4th of 7 | 0.522 | 0.506–0.540 | 34 | 25 | 9 | | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | Too few posts | 0.493 | 0.485–0.501 | 12 | 1 | 11 | | [Stopping and steering a running agent](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | Too few posts | 0.497 | 0.492–0.500 | 3 | 0 | 3 | ### Reliability and speed: 7th of 13 (customer love 0.515, n 160) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | 7th of 10 | 0.510 | 0.488–0.534 | 74 | 23 | 51 | | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | 8th of 11 | 0.482 | 0.458–0.512 | 59 | 3 | 56 | | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | Too few posts | 0.514 | 0.486–0.545 | 23 | 3 | 20 | | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | Too few posts | 0.511 | 0.496–0.529 | 15 | 5 | 10 | ### Account and support: 4th of 11 (customer love 0.545, n 42) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | Too few posts | 0.529 | 0.496–0.566 | 26 | 7 | 19 | | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | Too few posts | 0.491 | 0.485–0.495 | 12 | 0 | 12 | | [Data retention, training use and deployment isolation](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | Too few posts | 0.513 | 0.499–0.530 | 4 | 3 | 1 | | [Account bans and access restrictions](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | Too few posts | 0.497 | 0.494–0.500 | 3 | 0 | 3 | # OpenCode (Anomaly (open source)) Feedback Bench, coding agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/#/agent/opencode | Measure | Value | |---|---| | Rank | 3 of 17 (rank range 2–4) | | Feedback Score | 63.2 (95% interval 61.7–64.5) | | Popularity | 0.844 (share of voice 11.11%) | | Customer love | 0.562 (95% interval 0.550–0.571) | | Place | Popularity 3rd of 17 × Customer love 6th of 17 | | Top quadrant | yes | | Users | 16979 | | Posts counted | 42214 | | Posts that judge the agent | 12005 | | Criteria ranking high / low | 20 / 8 of 63 | ## Top requests What users ask to add or change, most asked first. 2137 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. | Rank | Request | Criterion | User-weeks | Posts | |---|---|---|---|---| | 1 | DeepSeek V4.1 Flash model support | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 56 | 57 | | 2 | Higher overall usage limits | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 38 | 38 | | 3 | Make a specific model free | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | 35 | 36 | | 4 | Cheaper low-priced entry plan tier | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 34 | 34 | | 5 | Subscriptions usable in third-party harnesses | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 28 | 29 | | 6 | More models on OpenCode Go and Zen plans | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 21 | 21 | | 7 | Mid-priced tier between existing plans | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 20 | 21 | | 8 | Make temporary usage bonuses permanent | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | 18 | 20 | | 9 | More free models | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | 17 | 20 | | 10 | Max or higher reasoning effort level | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | 17 | 19 | | 11 | Stop spurious 429 rate limit errors | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | 17 | 18 | | 12 | Usage scaled to plan price | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 17 | 17 | ## Ranks highest on How much use a plan's price buys (1st of 12), Updates break working setups (1st of 7), Which models are offered on a plan and when (1st of 11) ## Ranks lowest on Plan-before-edit mode (4th of 5), Can do the user's kind of task (9th of 13), Short rolling usage window blocks or interrupts work (5th of 6) Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. ## All 63 criteria Criterion love: 0.5 is the category norm. n: rated user-weeks. ### Paying and limits: 1st of 13 (customer love 0.679, n 2849) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 1st of 12 | 0.664 | 0.641–0.686 | 1233 | 740 | 493 | | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | 2nd of 7 | 0.523 | 0.484–0.560 | 638 | 384 | 254 | | [Single prompt, model or effort level consumes disproportionate quota](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | 7th of 12 | 0.491 | 0.455–0.523 | 519 | 95 | 424 | | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | 2nd of 6 | 0.509 | 0.455–0.566 | 197 | 17 | 180 | | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | 1st of 7 | 0.618 | 0.545–0.681 | 183 | 21 | 162 | | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | 5th of 6 | 0.464 | 0.431–0.501 | 170 | 16 | 154 | | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | 2nd of 6 | 0.533 | 0.464–0.607 | 152 | 9 | 143 | | [Prompt cache hits, misses and invalidation](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | 2nd of 4 | 0.503 | 0.476–0.534 | 125 | 40 | 85 | | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 3rd of 8 | 0.522 | 0.497–0.547 | 103 | 53 | 50 | | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 3rd of 5 | 0.489 | 0.466–0.515 | 58 | 7 | 51 | | [Pay-as-you-go overage, fallback billing and spend caps](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | 2nd of 4 | 0.547 | 0.494–0.616 | 42 | 7 | 35 | ### Setting up and connecting: 6th of 12 (customer love 0.503, n 687) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | 6th of 9 | 0.500 | 0.467–0.531 | 251 | 133 | 118 | | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | 4th of 9 | 0.496 | 0.466–0.528 | 198 | 97 | 101 | | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | 5th of 8 | 0.512 | 0.478–0.549 | 163 | 36 | 127 | | [Onboarding, discoverability and documentation](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | 5th of 7 | 0.492 | 0.464–0.521 | 71 | 13 | 58 | | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | 4th of 7 | 0.498 | 0.478–0.515 | 40 | 19 | 21 | ### Choosing models: 6th of 12 (customer love 0.558, n 930) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 1st of 11 | 0.645 | 0.612–0.676 | 463 | 186 | 277 | | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | 5th of 7 | 0.508 | 0.472–0.539 | 292 | 78 | 214 | | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | 4th of 8 | 0.484 | 0.455–0.511 | 133 | 32 | 101 | | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | 4th of 4 | 0.452 | 0.431–0.473 | 80 | 19 | 61 | ### Instructing and context: 10th of 11 (customer love 0.439, n 452) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Output degrades as the context window fills](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | 4th of 5 | 0.488 | 0.453–0.523 | 114 | 15 | 99 | | [Direct in-prompt instructions and caps are followed](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | 3rd of 5 | 0.480 | 0.453–0.508 | 113 | 26 | 87 | | [Context compaction keeps what matters, cheaply and quickly](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | 3rd of 5 | 0.481 | 0.456–0.510 | 108 | 33 | 75 | | [Finding the right files in the codebase](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | 4th of 5 | 0.483 | 0.467–0.500 | 40 | 13 | 27 | | [Persistent project rules files are read and obeyed](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | 3rd of 5 | 0.500 | 0.483–0.516 | 38 | 22 | 16 | | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | 3rd of 6 | 0.510 | 0.494–0.525 | 34 | 18 | 16 | | [Images, PDFs and file attachments as input](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | 3rd of 4 | 0.492 | 0.477–0.509 | 31 | 10 | 21 | | [Asks the user versus guessing](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | Too few posts | 0.496 | 0.485–0.506 | 19 | 7 | 12 | ### Doing the work: 10th of 14 (customer love 0.472, n 1930) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Can do the user's kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | 9th of 13 | 0.445 | 0.420–0.470 | 1207 | 705 | 502 | | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | 5th of 10 | 0.504 | 0.471–0.536 | 247 | 163 | 84 | | [Spins, loops or gets stuck without progress](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | 1st of 6 | 0.566 | 0.479–0.646 | 226 | 14 | 212 | | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | 7th of 8 | 0.470 | 0.441–0.497 | 68 | 44 | 24 | | [Frontend and visual UI output](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | 2nd of 5 | 0.509 | 0.486–0.533 | 57 | 29 | 28 | | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | 2nd of 5 | 0.514 | 0.487–0.540 | 51 | 13 | 38 | | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | 4th of 5 | 0.476 | 0.458–0.495 | 49 | 6 | 43 | | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | 2nd of 3 | 0.497 | 0.476–0.518 | 49 | 12 | 37 | | [Stops mid-task or answers instead of acting](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | 2nd of 3 | 0.472 | 0.454–0.495 | 48 | 2 | 46 | | [Plan-before-edit mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | 4th of 5 | 0.470 | 0.449–0.492 | 46 | 18 | 28 | | [Breaks existing code or reintroduces bugs](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | 4th of 5 | 0.485 | 0.464–0.514 | 45 | 2 | 43 | | [Diagnosing and fixing reported bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | 1st of 4 | 0.513 | 0.493–0.531 | 42 | 33 | 9 | | [Safety filters block legitimate coding tasks](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | 2nd of 5 | 0.539 | 0.504–0.576 | 34 | 10 | 24 | | [Does unrequested work or over-engineers](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | 1st of 5 | 0.538 | 0.496–0.591 | 30 | 5 | 25 | | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | Too few posts | 0.497 | 0.480–0.513 | 24 | 13 | 11 | | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | Too few posts | 0.500 | 0.490–0.513 | 13 | 4 | 9 | | [Caves to or argues with the user's judgement](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | Too few posts | 0.493 | 0.486–0.501 | 12 | 1 | 11 | | [Games checks instead of fixing the problem](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | Too few posts | 0.494 | 0.490–0.498 | 9 | 0 | 9 | ### Checking and finishing: 6th of 9 (customer love 0.507, n 89) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Agent-performed code review finds real issues](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | 2nd of 4 | 0.503 | 0.483–0.522 | 31 | 25 | 6 | | [Claims work is done or fixed when it is not](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | Too few posts | 0.511 | 0.480–0.559 | 25 | 2 | 23 | | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | Too few posts | 0.510 | 0.496–0.523 | 18 | 11 | 7 | | [Builds, tests or runs its own changes](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | Too few posts | 0.497 | 0.484–0.509 | 15 | 7 | 8 | ### Interface and sessions: 6th of 14 (customer love 0.517, n 555) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 3rd of 12 | 0.521 | 0.489–0.552 | 404 | 150 | 254 | | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | 3rd of 7 | 0.534 | 0.511–0.557 | 92 | 65 | 27 | | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | 2nd of 6 | 0.516 | 0.490–0.543 | 76 | 24 | 52 | | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | Too few posts | 0.513 | 0.501–0.526 | 19 | 15 | 4 | | [Stopping and steering a running agent](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | Too few posts | 0.505 | 0.494–0.519 | 14 | 7 | 7 | ### Reliability and speed: 4th of 13 (customer love 0.570, n 1811) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | 4th of 10 | 0.572 | 0.542–0.601 | 791 | 270 | 521 | | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | 2nd of 6 | 0.514 | 0.456–0.574 | 558 | 44 | 514 | | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | 4th of 11 | 0.581 | 0.523–0.632 | 451 | 48 | 403 | | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | 1st of 7 | 0.635 | 0.573–0.693 | 167 | 43 | 124 | ### Account and support: 7th of 11 (customer love 0.511, n 410) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | 3rd of 8 | 0.530 | 0.485–0.577 | 127 | 22 | 105 | | [Data retention, training use and deployment isolation](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | 4th of 4 | 0.477 | 0.447–0.514 | 125 | 20 | 105 | | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | 5th of 5 | 0.427 | 0.414–0.440 | 109 | 0 | 109 | | [Account bans and access restrictions](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | 2nd of 5 | 0.518 | 0.474–0.573 | 87 | 7 | 80 | # Pi (Earendil Works (open source)) Feedback Bench, coding agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/#/agent/pi | Measure | Value | |---|---| | Rank | 4 of 17 (rank range 2–4) | | Feedback Score | 62.4 (95% interval 60.7–63.8) | | Popularity | 0.641 (share of voice 2.90%) | | Customer love | 0.596 (95% interval 0.582–0.610) | | Place | Popularity 6th of 17 × Customer love 2nd of 17 | | Top quadrant | yes | | Users | 4448 | | Posts counted | 10994 | | Posts that judge the agent | 2710 | | Criteria ranking high / low | 16 / 1 of 63 | ## Top requests What users ask to add or change, most asked first. 593 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. | Rank | Request | Criterion | User-weeks | Posts | |---|---|---|---|---| | 1 | Claude subscription usable in third-party harnesses | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 24 | 24 | | 2 | MCP as optional plugin, not core | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | 14 | 15 | | 3 | Built-in first-class subagent orchestration | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | 12 | 15 | | 4 | Subscriptions usable in third-party harnesses | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 11 | 11 | | 5 | Minimal core with features as optional extensions | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | 10 | 11 | | 6 | Dedicated desktop app | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | 10 | 10 | | 7 | Configurable multi-model agent teams with roles | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | 6 | 6 | | 8 | General MCP server support | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | 6 | 6 | | 9 | Subagent activity visibility | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | 6 | 6 | | 10 | Configurable keyboard shortcuts | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 5 | 6 | | 11 | Multi-provider model support | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 5 | 5 | | 12 | Rewind session to checkpoint with code revert | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | 5 | 5 | ## Ranks highest on Single prompt, model or effort level consumes disproportionate quota (1st of 12), MCP servers, plugins, skills and hooks (1st of 9), Context compaction keeps what matters, cheaply and quickly (1st of 5) ## Ranks lowest on Subagents, parallel agents and orchestrators (9th of 10) Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. ## All 63 criteria Criterion love: 0.5 is the category norm. n: rated user-weeks. ### Paying and limits: 2nd of 13 (customer love 0.650, n 275) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Single prompt, model or effort level consumes disproportionate quota](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | 1st of 12 | 0.639 | 0.604–0.668 | 118 | 66 | 52 | | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 4th of 8 | 0.512 | 0.488–0.537 | 65 | 33 | 32 | | [Prompt cache hits, misses and invalidation](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | 1st of 4 | 0.541 | 0.516–0.570 | 50 | 29 | 21 | | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 3rd of 12 | 0.547 | 0.529–0.566 | 38 | 32 | 6 | | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | Too few posts | 0.528 | 0.505–0.557 | 10 | 5 | 5 | | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | Too few posts | 0.506 | 0.500–0.514 | 7 | 6 | 1 | | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | Too few posts | 0.504 | 0.494–0.522 | 5 | 1 | 4 | | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | Too few posts | 0.496 | 0.492–0.499 | 4 | 0 | 4 | | [Pay-as-you-go overage, fallback billing and spend caps](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | Too few posts | 0.498 | 0.494–0.500 | 2 | 0 | 2 | | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Setting up and connecting: 1st of 12 (customer love 0.626, n 464) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | 1st of 9 | 0.583 | 0.553–0.611 | 306 | 198 | 108 | | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | 1st of 9 | 0.544 | 0.521–0.569 | 75 | 54 | 21 | | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | 2nd of 8 | 0.531 | 0.498–0.565 | 56 | 17 | 39 | | [Onboarding, discoverability and documentation](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | 1st of 7 | 0.537 | 0.510–0.565 | 42 | 18 | 24 | | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | Too few posts | 0.505 | 0.496–0.513 | 9 | 7 | 2 | ### Choosing models: 4th of 12 (customer love 0.568, n 87) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 8th of 11 | 0.541 | 0.518–0.566 | 40 | 22 | 18 | | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | Too few posts | 0.545 | 0.526–0.567 | 23 | 18 | 5 | | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | Too few posts | 0.502 | 0.492–0.512 | 12 | 6 | 6 | | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | Too few posts | 0.492 | 0.483–0.503 | 12 | 1 | 11 | ### Instructing and context: 2nd of 11 (customer love 0.547, n 221) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Context compaction keeps what matters, cheaply and quickly](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | 1st of 5 | 0.543 | 0.518–0.569 | 94 | 49 | 45 | | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | 2nd of 6 | 0.517 | 0.500–0.534 | 44 | 25 | 19 | | [Finding the right files in the codebase](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | Too few posts | 0.496 | 0.483–0.509 | 23 | 9 | 14 | | [Output degrades as the context window fills](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | Too few posts | 0.503 | 0.486–0.525 | 21 | 4 | 17 | | [Direct in-prompt instructions and caps are followed](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | Too few posts | 0.504 | 0.490–0.521 | 20 | 6 | 14 | | [Persistent project rules files are read and obeyed](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | Too few posts | 0.500 | 0.489–0.510 | 19 | 11 | 8 | | [Images, PDFs and file attachments as input](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | Too few posts | 0.502 | 0.494–0.512 | 11 | 5 | 6 | | [Asks the user versus guessing](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | Too few posts | 0.501 | 0.494–0.510 | 8 | 4 | 4 | ### Doing the work: 5th of 14 (customer love 0.558, n 487) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Can do the user's kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | 4th of 13 | 0.562 | 0.535–0.588 | 221 | 170 | 51 | | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | 9th of 10 | 0.476 | 0.447–0.504 | 145 | 84 | 61 | | [Spins, loops or gets stuck without progress](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | 2nd of 6 | 0.558 | 0.508–0.614 | 36 | 6 | 30 | | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | Too few posts | 0.511 | 0.494–0.527 | 26 | 22 | 4 | | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | Too few posts | 0.517 | 0.498–0.539 | 24 | 9 | 15 | | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | Too few posts | 0.518 | 0.502–0.535 | 21 | 11 | 10 | | [Does unrequested work or over-engineers](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | Too few posts | 0.507 | 0.488–0.540 | 15 | 1 | 14 | | [Plan-before-edit mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | Too few posts | 0.491 | 0.480–0.502 | 14 | 5 | 9 | | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | Too few posts | 0.500 | 0.490–0.510 | 11 | 5 | 6 | | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | Too few posts | 0.508 | 0.497–0.520 | 7 | 4 | 3 | | [Breaks existing code or reintroduces bugs](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | Too few posts | 0.505 | 0.495–0.521 | 5 | 1 | 4 | | [Stops mid-task or answers instead of acting](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | Too few posts | 0.495 | 0.490–0.499 | 5 | 0 | 5 | | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | Too few posts | 0.499 | 0.494–0.506 | 4 | 1 | 3 | | [Frontend and visual UI output](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | Too few posts | 0.502 | 0.498–0.507 | 3 | 2 | 1 | | [Diagnosing and fixing reported bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | Too few posts | 0.504 | 0.500–0.508 | 3 | 3 | 0 | | [Safety filters block legitimate coding tasks](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | Too few posts | 0.499 | 0.497–0.500 | 2 | 0 | 2 | | [Caves to or argues with the user's judgement](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | Too few posts | 0.502 | 0.497–0.508 | 2 | 1 | 1 | | [Games checks instead of fixing the problem](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | Too few posts | 0.499 | 0.497–0.500 | 1 | 0 | 1 | ### Checking and finishing: Too few posts (customer love 0.510, n 20) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Agent-performed code review finds real issues](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | Too few posts | 0.505 | 0.495–0.513 | 11 | 10 | 1 | | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | Too few posts | 0.496 | 0.490–0.502 | 5 | 1 | 4 | | [Claims work is done or fixed when it is not](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | Too few posts | 0.516 | 0.499–0.555 | 2 | 1 | 1 | | [Builds, tests or runs its own changes](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | Too few posts | 0.502 | 0.500–0.505 | 2 | 2 | 0 | ### Interface and sessions: 4th of 14 (customer love 0.553, n 211) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 2nd of 12 | 0.537 | 0.508–0.564 | 142 | 63 | 79 | | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | 1st of 6 | 0.559 | 0.538–0.582 | 41 | 31 | 10 | | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | Too few posts | 0.512 | 0.500–0.523 | 16 | 12 | 4 | | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | Too few posts | 0.503 | 0.493–0.513 | 11 | 7 | 4 | | [Stopping and steering a running agent](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | Too few posts | 0.499 | 0.493–0.506 | 5 | 2 | 3 | ### Reliability and speed: 3rd of 13 (customer love 0.616, n 200) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | 3rd of 10 | 0.590 | 0.558–0.619 | 86 | 50 | 36 | | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | 2nd of 11 | 0.587 | 0.532–0.641 | 75 | 16 | 59 | | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | 3rd of 7 | 0.502 | 0.477–0.530 | 33 | 5 | 28 | | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | Too few posts | 0.503 | 0.486–0.522 | 17 | 2 | 15 | ### Account and support: 2nd of 11 (customer love 0.583, n 41) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Account bans and access restrictions](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | Too few posts | 0.540 | 0.501–0.585 | 23 | 5 | 18 | | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | Too few posts | 0.519 | 0.499–0.545 | 9 | 4 | 5 | | [Data retention, training use and deployment isolation](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | Too few posts | 0.527 | 0.505–0.551 | 9 | 6 | 3 | | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | # Amp (Amp) Feedback Bench, coding agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/#/agent/amp | Measure | Value | |---|---| | Rank | 5 of 17 (rank range 5–6) | | Feedback Score | 56.0 (95% interval 54.2–57.8) | | Popularity | 0.482 (share of voice 1.02%) | | Customer love | 0.578 (95% interval 0.563–0.593) | | Place | Popularity 11th of 17 × Customer love 3rd of 17 | | Top quadrant | no | | Users | 1398 | | Posts counted | 5118 | | Posts that judge the agent | 1831 | | Criteria ranking high / low | 11 / 1 of 63 | ## Top requests What users ask to add or change, most asked first. 468 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. | Rank | Request | Criterion | User-weeks | Posts | |---|---|---|---|---| | 1 | Claude subscription usable in third-party harnesses | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 35 | 38 | | 2 | Subscriptions usable in third-party harnesses | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 17 | 19 | | 3 | Multiple subscriptions active at once | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 8 | 10 | | 4 | Run cloud agents on own infrastructure | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | 8 | 8 | | 5 | macOS cloud environments for agents | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | 7 | 8 | | 6 | Group sidebar sessions by project | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 6 | 7 | | 7 | Customizable model picker | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 5 | 6 | | 8 | Custom OpenAI-compatible provider endpoints | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | 5 | 5 | | 9 | GPT-6 Sol and Luna model support | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 5 | 5 | | 10 | Linux desktop app and Linux support | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | 5 | 5 | | 11 | Multi-account switching | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | 5 | 5 | | 12 | Opus 5.5 model support | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 5 | 5 | ## Ranks highest on Support, refunds and issue handling (1st of 8), Can do the user's kind of task (5th of 13), Subagents, parallel agents and orchestrators (1st of 10) ## Ranks lowest on How the interface shows work, and what the user can configure (8th of 12) Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. ## All 63 criteria Criterion love: 0.5 is the category norm. n: rated user-weeks. ### Paying and limits: 4th of 13 (customer love 0.587, n 222) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 2nd of 8 | 0.528 | 0.501–0.559 | 97 | 47 | 50 | | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 6th of 12 | 0.527 | 0.502–0.549 | 66 | 35 | 31 | | [Single prompt, model or effort level consumes disproportionate quota](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | 6th of 12 | 0.503 | 0.482–0.527 | 37 | 7 | 30 | | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | Too few posts | 0.525 | 0.490–0.564 | 24 | 5 | 19 | | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | Too few posts | 0.508 | 0.498–0.519 | 12 | 9 | 3 | | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | Too few posts | 0.496 | 0.488–0.506 | 8 | 1 | 7 | | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | Too few posts | 0.512 | 0.490–0.550 | 8 | 1 | 7 | | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | Too few posts | 0.508 | 0.495–0.525 | 7 | 3 | 4 | | [Pay-as-you-go overage, fallback billing and spend caps](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | Too few posts | 0.497 | 0.494–0.500 | 4 | 0 | 4 | | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | Too few posts | 0.498 | 0.494–0.500 | 3 | 0 | 3 | | [Prompt cache hits, misses and invalidation](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | Too few posts | 0.498 | 0.494–0.500 | 2 | 0 | 2 | ### Setting up and connecting: 3rd of 12 (customer love 0.525, n 170) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | 3rd of 9 | 0.504 | 0.485–0.523 | 49 | 25 | 24 | | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | 5th of 9 | 0.507 | 0.487–0.526 | 48 | 27 | 21 | | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | 1st of 8 | 0.540 | 0.507–0.573 | 44 | 15 | 29 | | [Onboarding, discoverability and documentation](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | 2nd of 7 | 0.521 | 0.497–0.547 | 33 | 11 | 22 | | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | Too few posts | 0.500 | 0.490–0.509 | 10 | 5 | 5 | ### Choosing models: 5th of 12 (customer love 0.559, n 77) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 7th of 11 | 0.551 | 0.524–0.578 | 37 | 21 | 16 | | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | Too few posts | 0.521 | 0.500–0.542 | 24 | 13 | 11 | | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | Too few posts | 0.489 | 0.477–0.501 | 16 | 4 | 12 | | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | Too few posts | 0.514 | 0.502–0.529 | 7 | 5 | 2 | ### Instructing and context: 4th of 11 (customer love 0.511, n 40) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Direct in-prompt instructions and caps are followed](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | Too few posts | 0.510 | 0.498–0.524 | 10 | 6 | 4 | | [Context compaction keeps what matters, cheaply and quickly](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | Too few posts | 0.510 | 0.500–0.520 | 9 | 7 | 2 | | [Output degrades as the context window fills](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | Too few posts | 0.498 | 0.491–0.505 | 7 | 1 | 6 | | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | Too few posts | 0.500 | 0.493–0.507 | 6 | 3 | 3 | | [Finding the right files in the codebase](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | Too few posts | 0.497 | 0.492–0.501 | 4 | 1 | 3 | | [Persistent project rules files are read and obeyed](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | Too few posts | 0.503 | 0.500–0.507 | 3 | 3 | 0 | | [Asks the user versus guessing](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | Too few posts | 0.497 | 0.492–0.500 | 2 | 0 | 2 | | [Images, PDFs and file attachments as input](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | Too few posts | 0.499 | 0.497–0.500 | 1 | 0 | 1 | ### Doing the work: 2nd of 14 (customer love 0.621, n 238) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Can do the user's kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | 5th of 13 | 0.560 | 0.536–0.583 | 104 | 89 | 15 | | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | 1st of 10 | 0.556 | 0.533–0.578 | 87 | 76 | 11 | | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | 4th of 8 | 0.509 | 0.492–0.525 | 36 | 30 | 6 | | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | Too few posts | 0.491 | 0.479–0.502 | 15 | 2 | 13 | | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | Too few posts | 0.505 | 0.496–0.515 | 13 | 8 | 5 | | [Diagnosing and fixing reported bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | Too few posts | 0.501 | 0.491–0.510 | 9 | 7 | 2 | | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | Too few posts | 0.512 | 0.501–0.526 | 7 | 5 | 2 | | [Stops mid-task or answers instead of acting](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | Too few posts | 0.496 | 0.492–0.499 | 6 | 0 | 6 | | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | Too few posts | 0.509 | 0.496–0.523 | 5 | 3 | 2 | | [Frontend and visual UI output](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | Too few posts | 0.501 | 0.496–0.509 | 2 | 1 | 1 | | [Does unrequested work or over-engineers](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | Too few posts | 0.499 | 0.497–0.500 | 2 | 0 | 2 | | [Plan-before-edit mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | Too few posts | 0.500 | 0.494–0.505 | 2 | 1 | 1 | | [Breaks existing code or reintroduces bugs](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | Too few posts | 0.499 | 0.496–0.500 | 1 | 0 | 1 | | [Caves to or argues with the user's judgement](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | | [Spins, loops or gets stuck without progress](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Games checks instead of fixing the problem](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Safety filters block legitimate coding tasks](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Checking and finishing: 4th of 9 (customer love 0.509, n 34) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | Too few posts | 0.505 | 0.490–0.521 | 20 | 10 | 10 | | [Builds, tests or runs its own changes](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | Too few posts | 0.511 | 0.501–0.521 | 10 | 8 | 2 | | [Agent-performed code review finds real issues](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | Too few posts | 0.500 | 0.490–0.507 | 5 | 4 | 1 | | [Claims work is done or fixed when it is not](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | Too few posts | 0.498 | 0.496–0.500 | 2 | 0 | 2 | ### Interface and sessions: 2nd of 14 (customer love 0.595, n 313) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 8th of 12 | 0.480 | 0.455–0.507 | 140 | 43 | 97 | | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | 1st of 5 | 0.590 | 0.563–0.618 | 118 | 96 | 22 | | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | 1st of 7 | 0.559 | 0.538–0.580 | 64 | 54 | 10 | | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | Too few posts | 0.512 | 0.492–0.533 | 29 | 10 | 19 | | [Stopping and steering a running agent](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | Too few posts | 0.503 | 0.498–0.512 | 4 | 2 | 2 | ### Reliability and speed: 6th of 13 (customer love 0.521, n 103) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | 1st of 6 | 0.514 | 0.486–0.542 | 41 | 5 | 36 | | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | 5th of 11 | 0.511 | 0.477–0.549 | 35 | 3 | 32 | | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | Too few posts | 0.513 | 0.493–0.535 | 25 | 9 | 16 | | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | Too few posts | 0.515 | 0.497–0.537 | 8 | 3 | 5 | ### Account and support: 1st of 11 (customer love 0.624, n 50) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | 1st of 8 | 0.612 | 0.575–0.652 | 38 | 29 | 9 | | [Account bans and access restrictions](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | Too few posts | 0.505 | 0.491–0.530 | 8 | 1 | 7 | | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | Too few posts | 0.497 | 0.495–0.499 | 4 | 0 | 4 | | [Data retention, training use and deployment isolation](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | Too few posts | 0.501 | 0.495–0.511 | 4 | 1 | 3 | # Factory (Factory) Feedback Bench, coding agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/#/agent/factory | Measure | Value | |---|---| | Rank | 6 of 17 (rank range 5–7) | | Feedback Score | 55.1 (95% interval 52.8–57.2) | | Popularity | 0.475 (share of voice 0.97%) | | Customer love | 0.572 (95% interval 0.553–0.591) | | Place | Popularity 12th of 17 × Customer love 5th of 17 | | Top quadrant | no | | Users | 1396 | | Posts counted | 3801 | | Posts that judge the agent | 1055 | | Criteria ranking high / low | 9 / 0 of 63 | ## Top requests What users ask to add or change, most asked first. 329 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. | Rank | Request | Criterion | User-weeks | Posts | |---|---|---|---|---| | 1 | Official mobile app | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | 15 | 15 | | 2 | Subscriptions usable in third-party harnesses | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 14 | 14 | | 3 | ChatGPT/Codex subscription usable in other tools | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 12 | 14 | | 4 | DeepSeek V4.1 Flash model support | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 10 | 10 | | 5 | Higher overall usage limits | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 8 | 10 | | 6 | Free trial of paid plans | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | 8 | 9 | | 7 | Honor promised credits and coupons | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | 7 | 9 | | 8 | Linux desktop app and Linux support | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | 7 | 8 | | 9 | Muse / Meta model support | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 7 | 7 | | 10 | Configurable font family and size | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 5 | 5 | | 11 | Control local desktop sessions from mobile app | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | 5 | 5 | | 12 | Remove the 5-hour usage window | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | 5 | 5 | ## Ranks highest on Automatic model routing and fallback (2nd of 8), Can do the user's kind of task (3rd of 13), Long unattended runs and goal/loop mode (3rd of 8) ## Ranks lowest on No criterion where it sits clearly in the bottom half. Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. ## All 63 criteria Criterion love: 0.5 is the category norm. n: rated user-weeks. ### Paying and limits: 5th of 13 (customer love 0.581, n 177) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 5th of 12 | 0.532 | 0.504–0.561 | 105 | 55 | 50 | | [Single prompt, model or effort level consumes disproportionate quota](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | 3rd of 12 | 0.552 | 0.518–0.585 | 41 | 16 | 25 | | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | Too few posts | 0.493 | 0.479–0.512 | 18 | 1 | 17 | | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | Too few posts | 0.494 | 0.483–0.506 | 15 | 4 | 11 | | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | Too few posts | 0.500 | 0.488–0.511 | 10 | 6 | 4 | | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | Too few posts | 0.494 | 0.489–0.498 | 8 | 0 | 8 | | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | Too few posts | 0.500 | 0.491–0.514 | 7 | 1 | 6 | | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | Too few posts | 0.513 | 0.492–0.553 | 7 | 1 | 6 | | [Pay-as-you-go overage, fallback billing and spend caps](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | Too few posts | 0.496 | 0.491–0.499 | 4 | 0 | 4 | | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | Too few posts | 0.524 | 0.497–0.574 | 3 | 1 | 2 | | [Prompt cache hits, misses and invalidation](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | Too few posts | 0.506 | 0.500–0.515 | 2 | 2 | 0 | ### Setting up and connecting: 4th of 12 (customer love 0.514, n 68) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | Too few posts | 0.505 | 0.487–0.522 | 26 | 15 | 11 | | [Onboarding, discoverability and documentation](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | Too few posts | 0.529 | 0.505–0.556 | 17 | 9 | 8 | | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | Too few posts | 0.487 | 0.473–0.499 | 14 | 3 | 11 | | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | Too few posts | 0.496 | 0.485–0.513 | 11 | 1 | 10 | | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | Too few posts | 0.506 | 0.498–0.515 | 7 | 5 | 2 | ### Choosing models: 3rd of 12 (customer love 0.613, n 80) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 3rd of 11 | 0.589 | 0.556–0.619 | 46 | 33 | 13 | | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | 2nd of 8 | 0.552 | 0.530–0.576 | 30 | 21 | 9 | | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | Too few posts | 0.512 | 0.497–0.528 | 11 | 5 | 6 | | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | Too few posts | 0.501 | 0.497–0.507 | 2 | 1 | 1 | ### Instructing and context: Too few posts (customer love 0.514, n 24) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Images, PDFs and file attachments as input](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | Too few posts | 0.501 | 0.493–0.512 | 7 | 2 | 5 | | [Direct in-prompt instructions and caps are followed](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | Too few posts | 0.505 | 0.496–0.516 | 5 | 3 | 2 | | [Context compaction keeps what matters, cheaply and quickly](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | Too few posts | 0.506 | 0.500–0.515 | 3 | 3 | 0 | | [Finding the right files in the codebase](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | Too few posts | 0.505 | 0.500–0.512 | 3 | 3 | 0 | | [Asks the user versus guessing](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | Too few posts | 0.504 | 0.500–0.512 | 2 | 2 | 0 | | [Output degrades as the context window fills](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | Too few posts | 0.498 | 0.494–0.500 | 2 | 0 | 2 | | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | Too few posts | 0.496 | 0.490–0.500 | 2 | 0 | 2 | | [Persistent project rules files are read and obeyed](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Doing the work: 3rd of 14 (customer love 0.616, n 165) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Can do the user's kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | 3rd of 13 | 0.574 | 0.552–0.597 | 95 | 86 | 9 | | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | 3rd of 8 | 0.535 | 0.516–0.552 | 40 | 37 | 3 | | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | 3rd of 10 | 0.513 | 0.495–0.533 | 34 | 25 | 9 | | [Diagnosing and fixing reported bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | Too few posts | 0.510 | 0.502–0.520 | 6 | 6 | 0 | | [Spins, loops or gets stuck without progress](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | Too few posts | 0.495 | 0.490–0.499 | 6 | 0 | 6 | | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | Too few posts | 0.507 | 0.497–0.521 | 6 | 3 | 3 | | [Plan-before-edit mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | Too few posts | 0.509 | 0.502–0.517 | 6 | 6 | 0 | | [Frontend and visual UI output](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | Too few posts | 0.498 | 0.491–0.504 | 5 | 2 | 3 | | [Does unrequested work or over-engineers](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | Too few posts | 0.495 | 0.490–0.499 | 5 | 0 | 5 | | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | Too few posts | 0.513 | 0.499–0.528 | 4 | 3 | 1 | | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | Too few posts | 0.502 | 0.496–0.510 | 3 | 2 | 1 | | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | Too few posts | 0.506 | 0.500–0.514 | 2 | 2 | 0 | | [Breaks existing code or reintroduces bugs](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | Too few posts | 0.499 | 0.496–0.500 | 1 | 0 | 1 | | [Stops mid-task or answers instead of acting](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Games checks instead of fixing the problem](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Safety filters block legitimate coding tasks](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Caves to or argues with the user's judgement](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Checking and finishing: Too few posts (customer love 0.508, n 10) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Agent-performed code review finds real issues](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | Too few posts | 0.499 | 0.485–0.509 | 5 | 4 | 1 | | [Claims work is done or fixed when it is not](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | Too few posts | 0.523 | 0.497–0.580 | 2 | 1 | 1 | | [Builds, tests or runs its own changes](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | Too few posts | 0.504 | 0.500–0.511 | 2 | 2 | 0 | | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | Too few posts | 0.499 | 0.497–0.500 | 1 | 0 | 1 | ### Interface and sessions: 8th of 14 (customer love 0.511, n 68) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 4th of 12 | 0.515 | 0.492–0.540 | 38 | 15 | 23 | | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | Too few posts | 0.483 | 0.465–0.502 | 20 | 6 | 14 | | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | Too few posts | 0.508 | 0.494–0.523 | 17 | 11 | 6 | | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | Too few posts | 0.498 | 0.490–0.509 | 6 | 1 | 5 | | [Stopping and steering a running agent](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | Too few posts | 0.501 | 0.500–0.504 | 1 | 1 | 0 | ### Reliability and speed: 5th of 13 (customer love 0.531, n 50) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | Too few posts | 0.517 | 0.494–0.541 | 27 | 12 | 15 | | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | Too few posts | 0.489 | 0.482–0.495 | 13 | 0 | 13 | | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | Too few posts | 0.509 | 0.487–0.548 | 9 | 1 | 8 | | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | Too few posts | 0.507 | 0.493–0.529 | 4 | 1 | 3 | ### Account and support: 6th of 11 (customer love 0.513, n 45) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | 4th of 8 | 0.507 | 0.478–0.537 | 37 | 6 | 31 | | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | Too few posts | 0.494 | 0.488–0.498 | 9 | 0 | 9 | | [Data retention, training use and deployment isolation](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | Too few posts | 0.501 | 0.497–0.507 | 2 | 1 | 1 | | [Account bans and access restrictions](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | Too few posts | 0.499 | 0.497–0.500 | 1 | 0 | 1 | # Cursor (Anysphere) Feedback Bench, coding agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/#/agent/cursor | Measure | Value | |---|---| | Rank | 7 of 17 (rank range 6–7) | | Feedback Score | 52.9 (95% interval 51.4–54.4) | | Popularity | 0.802 (share of voice 8.42%) | | Customer love | 0.487 (95% interval 0.474–0.500) | | Place | Popularity 5th of 17 × Customer love 14th of 17 | | Top quadrant | no | | Users | 13122 | | Posts counted | 29322 | | Posts that judge the agent | 11328 | | Criteria ranking high / low | 6 / 20 of 63 | ## Top requests What users ask to add or change, most asked first. 2866 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. | Rank | Request | Criterion | User-weeks | Posts | |---|---|---|---|---| | 1 | Native Android app | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | 108 | 122 | | 2 | Fix account linking and unification | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | 63 | 74 | | 3 | One-time immediate usage reset | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 56 | 60 | | 4 | Release next Composer model | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 56 | 60 | | 5 | Honor manually selected model | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | 42 | 45 | | 6 | Unified subscription across Grok, X and Cursor | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 37 | 38 | | 7 | Newer Grok model support | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 36 | 36 | | 8 | Additional bonus usage resets | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 30 | 31 | | 9 | Human support for billing disputes | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | 29 | 34 | | 10 | Higher overall usage limits | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 28 | 28 | | 11 | Astra model availability | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 25 | 26 | | 12 | Human review and appeals for bans | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | 24 | 33 | ## Ranks highest on Reviewing and approving the agent's changes (1st of 5), IDE and editor integration (2nd of 7), Finding the right files in the codebase (1st of 5) ## Ranks lowest on Using an existing subscription across tools (8th of 8), Quality got worse or better over time (6th of 7), Pay-as-you-go overage, fallback billing and spend caps (4th of 4) Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. ## All 63 criteria Criterion love: 0.5 is the category norm. n: rated user-weeks. ### Paying and limits: 10th of 13 (customer love 0.467, n 2779) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 9th of 12 | 0.484 | 0.457–0.512 | 1363 | 535 | 828 | | [Single prompt, model or effort level consumes disproportionate quota](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | 11th of 12 | 0.429 | 0.392–0.463 | 776 | 99 | 677 | | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | 6th of 7 | 0.468 | 0.393–0.535 | 310 | 13 | 297 | | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | 3rd of 6 | 0.491 | 0.430–0.569 | 190 | 9 | 181 | | [Pay-as-you-go overage, fallback billing and spend caps](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | 4th of 4 | 0.415 | 0.381–0.463 | 172 | 3 | 169 | | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | 5th of 6 | 0.493 | 0.446–0.539 | 164 | 12 | 152 | | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 4th of 5 | 0.454 | 0.428–0.483 | 149 | 15 | 134 | | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 8th of 8 | 0.419 | 0.397–0.444 | 100 | 12 | 88 | | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | 2nd of 6 | 0.549 | 0.512–0.586 | 72 | 19 | 53 | | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | 4th of 7 | 0.501 | 0.482–0.519 | 46 | 26 | 20 | | [Prompt cache hits, misses and invalidation](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | Too few posts | 0.495 | 0.482–0.508 | 19 | 4 | 15 | ### Setting up and connecting: 9th of 12 (customer love 0.480, n 545) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | 2nd of 7 | 0.575 | 0.545–0.602 | 154 | 106 | 48 | | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | 7th of 8 | 0.447 | 0.417–0.481 | 145 | 16 | 129 | | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | 6th of 9 | 0.471 | 0.446–0.497 | 127 | 47 | 80 | | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | 9th of 9 | 0.459 | 0.435–0.483 | 83 | 26 | 57 | | [Onboarding, discoverability and documentation](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | 4th of 7 | 0.497 | 0.476–0.519 | 50 | 10 | 40 | ### Choosing models: 10th of 12 (customer love 0.461, n 1126) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | 6th of 7 | 0.408 | 0.377–0.441 | 408 | 59 | 349 | | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | 6th of 8 | 0.475 | 0.441–0.512 | 400 | 87 | 313 | | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 6th of 11 | 0.555 | 0.521–0.589 | 394 | 130 | 264 | | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | Too few posts | 0.499 | 0.486–0.513 | 29 | 14 | 15 | ### Instructing and context: 5th of 11 (customer love 0.507, n 380) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Persistent project rules files are read and obeyed](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | 5th of 5 | 0.473 | 0.448–0.499 | 93 | 42 | 51 | | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | 4th of 6 | 0.492 | 0.470–0.515 | 91 | 39 | 52 | | [Finding the right files in the codebase](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | 1st of 5 | 0.545 | 0.521–0.572 | 83 | 50 | 33 | | [Direct in-prompt instructions and caps are followed](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | 4th of 5 | 0.476 | 0.456–0.498 | 72 | 12 | 60 | | [Output degrades as the context window fills](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | 2nd of 5 | 0.503 | 0.481–0.528 | 36 | 7 | 29 | | [Context compaction keeps what matters, cheaply and quickly](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | Too few posts | 0.499 | 0.487–0.513 | 21 | 7 | 14 | | [Asks the user versus guessing](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | Too few posts | 0.494 | 0.484–0.504 | 15 | 4 | 11 | | [Images, PDFs and file attachments as input](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | Too few posts | 0.492 | 0.482–0.501 | 11 | 2 | 9 | ### Doing the work: 7th of 14 (customer love 0.515, n 1617) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Can do the user's kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | 6th of 13 | 0.522 | 0.496–0.551 | 904 | 625 | 279 | | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | 6th of 10 | 0.500 | 0.470–0.531 | 206 | 132 | 74 | | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | 3rd of 5 | 0.476 | 0.453–0.502 | 74 | 10 | 64 | | [Does unrequested work or over-engineers](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | 4th of 5 | 0.472 | 0.450–0.501 | 69 | 2 | 67 | | [Frontend and visual UI output](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | 3rd of 5 | 0.506 | 0.484–0.530 | 68 | 33 | 35 | | [Plan-before-edit mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | 2nd of 5 | 0.532 | 0.513–0.554 | 62 | 47 | 15 | | [Spins, loops or gets stuck without progress](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | 5th of 6 | 0.460 | 0.450–0.470 | 60 | 0 | 60 | | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | 5th of 8 | 0.488 | 0.461–0.514 | 59 | 43 | 16 | | [Breaks existing code or reintroduces bugs](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | 3rd of 5 | 0.491 | 0.459–0.527 | 55 | 3 | 52 | | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | 3rd of 3 | 0.479 | 0.461–0.498 | 53 | 12 | 41 | | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | 4th of 5 | 0.498 | 0.476–0.522 | 53 | 12 | 41 | | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | 2nd of 4 | 0.483 | 0.467–0.501 | 42 | 16 | 26 | | [Safety filters block legitimate coding tasks](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | 4th of 5 | 0.485 | 0.471–0.503 | 33 | 2 | 31 | | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | Too few posts | 0.490 | 0.475–0.509 | 29 | 4 | 25 | | [Diagnosing and fixing reported bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | Too few posts | 0.494 | 0.473–0.512 | 28 | 18 | 10 | | [Stops mid-task or answers instead of acting](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | Too few posts | 0.503 | 0.486–0.532 | 12 | 1 | 11 | | [Caves to or argues with the user's judgement](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | Too few posts | 0.504 | 0.491–0.520 | 8 | 2 | 6 | | [Games checks instead of fixing the problem](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | Too few posts | 0.497 | 0.493–0.500 | 4 | 0 | 4 | ### Checking and finishing: 3rd of 9 (customer love 0.515, n 176) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | 1st of 5 | 0.567 | 0.538–0.599 | 80 | 51 | 29 | | [Agent-performed code review finds real issues](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | 3rd of 4 | 0.501 | 0.474–0.524 | 44 | 35 | 9 | | [Claims work is done or fixed when it is not](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | 2nd of 4 | 0.500 | 0.464–0.555 | 36 | 2 | 34 | | [Builds, tests or runs its own changes](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | Too few posts | 0.484 | 0.468–0.499 | 25 | 10 | 15 | ### Interface and sessions: 3rd of 14 (customer love 0.556, n 683) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 7th of 12 | 0.497 | 0.467–0.527 | 263 | 85 | 178 | | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | 3rd of 5 | 0.554 | 0.525–0.583 | 213 | 136 | 77 | | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | 5th of 7 | 0.498 | 0.469–0.525 | 149 | 74 | 75 | | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | 5th of 6 | 0.485 | 0.465–0.506 | 65 | 12 | 53 | | [Stopping and steering a running agent](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | 1st of 3 | 0.534 | 0.512–0.559 | 39 | 23 | 16 | ### Reliability and speed: 12th of 13 (customer love 0.477, n 803) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | 5th of 10 | 0.525 | 0.492–0.558 | 277 | 83 | 194 | | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | 9th of 11 | 0.456 | 0.410–0.502 | 248 | 12 | 236 | | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | 5th of 6 | 0.485 | 0.443–0.531 | 235 | 16 | 219 | | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | 5th of 7 | 0.472 | 0.448–0.501 | 83 | 6 | 77 | ### Account and support: 11th of 11 (customer love 0.399, n 804) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | 8th of 8 | 0.409 | 0.365–0.453 | 541 | 50 | 491 | | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | 1st of 5 | 0.555 | 0.400–0.709 | 308 | 6 | 302 | | [Account bans and access restrictions](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | 5th of 5 | 0.432 | 0.405–0.467 | 119 | 2 | 117 | | [Data retention, training use and deployment isolation](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | 2nd of 4 | 0.479 | 0.464–0.496 | 38 | 4 | 34 | # Zed (Zed Industries) Feedback Bench, coding agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/#/agent/zed | Measure | Value | |---|---| | Rank | 8 of 17 (rank range 8–10) | | Feedback Score | 48.7 (95% interval 47.4–50.1) | | Popularity | 0.541 (share of voice 1.50%) | | Customer love | 0.507 (95% interval 0.496–0.519) | | Place | Popularity 9th of 17 × Customer love 8th of 17 | | Top quadrant | yes, at the edge | | Users | 2404 | | Posts counted | 4072 | | Posts that judge the agent | 1301 | | Criteria ranking high / low | 2 / 4 of 63 | ## Top requests What users ask to add or change, most asked first. 712 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. | Rank | Request | Criterion | User-weeks | Posts | |---|---|---|---|---| | 1 | Claude subscription usable in third-party harnesses | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 16 | 17 | | 2 | Jupyter notebook support | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | 15 | 16 | | 3 | Markdown rendering and preview | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 15 | 16 | | 4 | Remote SSH development | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | 14 | 15 | | 5 | Custom OpenAI-compatible provider endpoints | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | 13 | 15 | | 6 | Integrated git panel | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | 13 | 15 | | 7 | Detachable panels and multi-window support | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 11 | 13 | | 8 | Built-in debugger | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | 11 | 11 | | 9 | Per-feature toggles to disable AI features | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 11 | 11 | | 10 | Richer editor extension and UI API | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | 11 | 11 | | 11 | Agent Client Protocol support | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | 10 | 11 | | 12 | Multi-provider model support | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 9 | 9 | ## Ranks highest on Latency, throughput and fast mode (2nd of 10), Client crashes, freezes and failed tool execution (3rd of 11) ## Ranks lowest on IDE and editor integration (6th of 7), Can do the user's kind of task (10th of 13), MCP servers, plugins, skills and hooks (8th of 9) Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. ## All 63 criteria Criterion love: 0.5 is the category norm. n: rated user-weeks. ### Paying and limits: 8th of 13 (customer love 0.517, n 45) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | Too few posts | 0.500 | 0.489–0.512 | 14 | 5 | 9 | | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | Too few posts | 0.502 | 0.494–0.510 | 8 | 6 | 2 | | [Single prompt, model or effort level consumes disproportionate quota](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | Too few posts | 0.507 | 0.497–0.519 | 7 | 3 | 4 | | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | Too few posts | 0.498 | 0.490–0.507 | 7 | 2 | 5 | | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | Too few posts | 0.498 | 0.495–0.500 | 3 | 0 | 3 | | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | Too few posts | 0.498 | 0.496–0.500 | 3 | 0 | 3 | | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | Too few posts | 0.499 | 0.496–0.500 | 2 | 0 | 2 | | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | Too few posts | 0.499 | 0.497–0.500 | 1 | 0 | 1 | | [Prompt cache hits, misses and invalidation](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | Too few posts | 0.499 | 0.497–0.500 | 1 | 0 | 1 | | [Pay-as-you-go overage, fallback billing and spend caps](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Setting up and connecting: 10th of 12 (customer love 0.442, n 193) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | 6th of 7 | 0.457 | 0.433–0.480 | 81 | 26 | 55 | | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | 8th of 9 | 0.456 | 0.436–0.474 | 57 | 11 | 46 | | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | Too few posts | 0.491 | 0.475–0.507 | 25 | 11 | 14 | | [Onboarding, discoverability and documentation](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | Too few posts | 0.487 | 0.478–0.500 | 20 | 1 | 19 | | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | Too few posts | 0.495 | 0.484–0.506 | 18 | 3 | 15 | ### Choosing models: Too few posts (customer love 0.504, n 19) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | Too few posts | 0.500 | 0.489–0.514 | 13 | 3 | 10 | | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | Too few posts | 0.502 | 0.497–0.509 | 2 | 1 | 1 | | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | Too few posts | 0.496 | 0.488–0.500 | 2 | 0 | 2 | | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | Too few posts | 0.507 | 0.500–0.517 | 2 | 2 | 0 | ### Instructing and context: 6th of 11 (customer love 0.499, n 30) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Finding the right files in the codebase](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | Too few posts | 0.497 | 0.485–0.509 | 15 | 6 | 9 | | [Persistent project rules files are read and obeyed](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | Too few posts | 0.496 | 0.490–0.500 | 6 | 1 | 5 | | [Context compaction keeps what matters, cheaply and quickly](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | Too few posts | 0.495 | 0.490–0.499 | 5 | 0 | 5 | | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | Too few posts | 0.505 | 0.500–0.511 | 3 | 3 | 0 | | [Output degrades as the context window fills](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | Too few posts | 0.498 | 0.494–0.500 | 2 | 0 | 2 | | [Direct in-prompt instructions and caps are followed](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | Too few posts | 0.502 | 0.500–0.506 | 1 | 1 | 0 | | [Images, PDFs and file attachments as input](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | Too few posts | 0.499 | 0.496–0.500 | 1 | 0 | 1 | | [Asks the user versus guessing](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Doing the work: 11th of 14 (customer love 0.461, n 136) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Can do the user's kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | 10th of 13 | 0.444 | 0.420–0.469 | 71 | 26 | 45 | | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | Too few posts | 0.507 | 0.491–0.525 | 23 | 9 | 14 | | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | Too few posts | 0.487 | 0.471–0.501 | 17 | 8 | 9 | | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | Too few posts | 0.506 | 0.498–0.517 | 6 | 3 | 3 | | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | Too few posts | 0.507 | 0.496–0.523 | 5 | 2 | 3 | | [Frontend and visual UI output](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | Too few posts | 0.502 | 0.498–0.507 | 4 | 3 | 1 | | [Spins, loops or gets stuck without progress](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | Too few posts | 0.497 | 0.493–0.500 | 3 | 0 | 3 | | [Breaks existing code or reintroduces bugs](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | Too few posts | 0.498 | 0.496–0.500 | 2 | 0 | 2 | | [Plan-before-edit mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | Too few posts | 0.500 | 0.494–0.506 | 2 | 1 | 1 | | [Diagnosing and fixing reported bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | Too few posts | 0.498 | 0.494–0.500 | 1 | 0 | 1 | | [Does unrequested work or over-engineers](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | Too few posts | 0.499 | 0.497–0.500 | 1 | 0 | 1 | | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | Too few posts | 0.497 | 0.491–0.500 | 1 | 0 | 1 | | [Caves to or argues with the user's judgement](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | Too few posts | 0.505 | 0.500–0.519 | 1 | 1 | 0 | | [Stops mid-task or answers instead of acting](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Games checks instead of fixing the problem](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Safety filters block legitimate coding tasks](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Checking and finishing: 8th of 9 (customer love 0.488, n 32) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | 2nd of 5 | 0.492 | 0.476–0.508 | 30 | 11 | 19 | | [Claims work is done or fixed when it is not](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | | [Agent-performed code review finds real issues](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | Too few posts | 0.502 | 0.500–0.505 | 1 | 1 | 0 | | [Builds, tests or runs its own changes](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Interface and sessions: 12th of 14 (customer love 0.441, n 247) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 9th of 12 | 0.474 | 0.447–0.502 | 219 | 66 | 153 | | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | Too few posts | 0.499 | 0.484–0.516 | 25 | 7 | 18 | | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | Too few posts | 0.486 | 0.471–0.497 | 9 | 1 | 8 | | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | Too few posts | 0.492 | 0.482–0.500 | 5 | 1 | 4 | | [Stopping and steering a running agent](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | Too few posts | 0.501 | 0.495–0.507 | 2 | 1 | 1 | ### Reliability and speed: 1st of 13 (customer love 0.668, n 175) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | 2nd of 10 | 0.636 | 0.609–0.662 | 89 | 73 | 16 | | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | 3rd of 11 | 0.585 | 0.530–0.638 | 84 | 17 | 67 | | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | Too few posts | 0.500 | 0.490–0.515 | 9 | 1 | 8 | | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Account and support: Too few posts (customer love 0.492, n 27) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | Too few posts | 0.494 | 0.486–0.504 | 13 | 1 | 12 | | [Data retention, training use and deployment isolation](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | Too few posts | 0.494 | 0.485–0.505 | 12 | 1 | 11 | | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | Too few posts | 0.499 | 0.498–0.500 | 2 | 0 | 2 | | [Account bans and access restrictions](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | # GitHub Copilot (GitHub) Feedback Bench, coding agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/#/agent/copilot | Measure | Value | |---|---| | Rank | 9 of 17 (rank range 8–11) | | Feedback Score | 48.1 (95% interval 46.5–49.9) | | Popularity | 0.557 (share of voice 1.66%) | | Customer love | 0.499 (95% interval 0.485–0.514) | | Place | Popularity 8th of 17 × Customer love 13th of 17 | | Top quadrant | no | | Users | 2612 | | Posts counted | 5413 | | Posts that judge the agent | 1866 | | Criteria ranking high / low | 2 / 7 of 63 | ## Top requests What users ask to add or change, most asked first. 285 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. | Rank | Request | Criterion | User-weeks | Posts | |---|---|---|---|---| | 1 | Exclude specific models from auto routing | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | 10 | 12 | | 2 | Per-file and per-hunk accept/reject of changes | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | 9 | 9 | | 3 | Honor manually selected model | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | 7 | 7 | | 4 | More open-weight models | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 4 | 5 | | 5 | Subagent token usage visibility | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | 4 | 5 | | 6 | More model options in catalog | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 4 | 4 | | 7 | Per-subagent model selection | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | 4 | 4 | | 8 | Organization-scale BYOK deployment via SSO | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | 3 | 7 | | 9 | Choose the orchestrator model | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | 3 | 3 | | 10 | Confirmation before risky actions | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | 3 | 3 | | 11 | DeepSeek V4.1 Flash model support | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 3 | 3 | | 12 | Faster addition of newly released models | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 3 | 3 | ## Ranks highest on Single prompt, model or effort level consumes disproportionate quota (4th of 12), Which models are offered on a plan and when (4th of 11) ## Ranks lowest on Can do the user's kind of task (11th of 13), Automatic model routing and fallback (5th of 8), Latency, throughput and fast mode (8th of 10) Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. ## All 63 criteria Criterion love: 0.5 is the category norm. n: rated user-weeks. ### Paying and limits: 6th of 13 (customer love 0.539, n 515) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 8th of 12 | 0.488 | 0.458–0.517 | 276 | 108 | 168 | | [Single prompt, model or effort level consumes disproportionate quota](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | 4th of 12 | 0.551 | 0.512–0.586 | 163 | 42 | 121 | | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | 4th of 7 | 0.507 | 0.469–0.552 | 50 | 3 | 47 | | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | Too few posts | 0.509 | 0.484–0.542 | 21 | 3 | 18 | | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | Too few posts | 0.534 | 0.488–0.586 | 18 | 3 | 15 | | [Pay-as-you-go overage, fallback billing and spend caps](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | Too few posts | 0.541 | 0.492–0.599 | 13 | 3 | 10 | | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | Too few posts | 0.498 | 0.488–0.509 | 9 | 1 | 8 | | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | Too few posts | 0.498 | 0.492–0.505 | 9 | 1 | 8 | | [Prompt cache hits, misses and invalidation](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | Too few posts | 0.503 | 0.494–0.516 | 8 | 3 | 5 | | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | Too few posts | 0.502 | 0.493–0.510 | 8 | 5 | 3 | | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | Too few posts | 0.504 | 0.495–0.517 | 7 | 4 | 3 | ### Setting up and connecting: 5th of 12 (customer love 0.513, n 158) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | 3rd of 7 | 0.509 | 0.488–0.530 | 58 | 32 | 26 | | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | 5th of 9 | 0.472 | 0.451–0.493 | 53 | 17 | 36 | | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | 7th of 9 | 0.500 | 0.481–0.519 | 37 | 19 | 18 | | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | Too few posts | 0.500 | 0.490–0.514 | 8 | 2 | 6 | | [Onboarding, discoverability and documentation](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | Too few posts | 0.493 | 0.486–0.498 | 7 | 0 | 7 | ### Choosing models: 7th of 12 (customer love 0.541, n 226) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 4th of 11 | 0.578 | 0.546–0.610 | 90 | 46 | 44 | | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | 5th of 8 | 0.477 | 0.452–0.504 | 84 | 16 | 68 | | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | 4th of 7 | 0.515 | 0.487–0.540 | 50 | 14 | 36 | | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | Too few posts | 0.494 | 0.483–0.505 | 21 | 7 | 14 | ### Instructing and context: 8th of 11 (customer love 0.490, n 80) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Persistent project rules files are read and obeyed](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | Too few posts | 0.488 | 0.472–0.502 | 21 | 9 | 12 | | [Direct in-prompt instructions and caps are followed](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | Too few posts | 0.494 | 0.481–0.509 | 21 | 4 | 17 | | [Finding the right files in the codebase](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | Too few posts | 0.495 | 0.484–0.506 | 14 | 5 | 9 | | [Context compaction keeps what matters, cheaply and quickly](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | Too few posts | 0.492 | 0.477–0.507 | 11 | 2 | 9 | | [Output degrades as the context window fills](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | Too few posts | 0.503 | 0.494–0.515 | 7 | 2 | 5 | | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | Too few posts | 0.506 | 0.497–0.515 | 7 | 5 | 2 | | [Images, PDFs and file attachments as input](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | Too few posts | 0.499 | 0.493–0.504 | 3 | 1 | 2 | | [Asks the user versus guessing](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | Too few posts | 0.500 | 0.495–0.507 | 2 | 1 | 1 | ### Doing the work: 13th of 14 (customer love 0.433, n 356) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Can do the user's kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | 11th of 13 | 0.408 | 0.380–0.440 | 234 | 106 | 128 | | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | 7th of 10 | 0.488 | 0.465–0.507 | 33 | 18 | 15 | | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | Too few posts | 0.498 | 0.486–0.512 | 18 | 4 | 14 | | [Spins, loops or gets stuck without progress](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | Too few posts | 0.513 | 0.486–0.567 | 15 | 1 | 14 | | [Does unrequested work or over-engineers](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | Too few posts | 0.504 | 0.486–0.535 | 13 | 1 | 12 | | [Plan-before-edit mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | Too few posts | 0.486 | 0.471–0.498 | 11 | 3 | 8 | | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | Too few posts | 0.502 | 0.493–0.510 | 9 | 6 | 3 | | [Breaks existing code or reintroduces bugs](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | Too few posts | 0.494 | 0.488–0.498 | 8 | 0 | 8 | | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | Too few posts | 0.508 | 0.502–0.514 | 8 | 8 | 0 | | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | Too few posts | 0.497 | 0.490–0.507 | 8 | 1 | 7 | | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | Too few posts | 0.494 | 0.488–0.500 | 7 | 1 | 6 | | [Diagnosing and fixing reported bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | Too few posts | 0.499 | 0.492–0.506 | 6 | 4 | 2 | | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | Too few posts | 0.502 | 0.494–0.515 | 5 | 1 | 4 | | [Stops mid-task or answers instead of acting](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | Too few posts | 0.497 | 0.494–0.500 | 4 | 0 | 4 | | [Caves to or argues with the user's judgement](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | Too few posts | 0.499 | 0.494–0.506 | 4 | 1 | 3 | | [Frontend and visual UI output](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | Too few posts | 0.502 | 0.498–0.508 | 3 | 2 | 1 | | [Safety filters block legitimate coding tasks](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | Too few posts | 0.498 | 0.494–0.500 | 2 | 0 | 2 | | [Games checks instead of fixing the problem](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | ### Checking and finishing: 5th of 9 (customer love 0.508, n 61) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Agent-performed code review finds real issues](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | Too few posts | 0.480 | 0.454–0.502 | 27 | 17 | 10 | | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | Too few posts | 0.515 | 0.498–0.532 | 26 | 16 | 10 | | [Claims work is done or fixed when it is not](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | Too few posts | 0.495 | 0.491–0.499 | 6 | 0 | 6 | | [Builds, tests or runs its own changes](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | Too few posts | 0.501 | 0.497–0.506 | 3 | 2 | 1 | ### Interface and sessions: 11th of 14 (customer love 0.461, n 75) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 10th of 12 | 0.471 | 0.449–0.494 | 60 | 12 | 48 | | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | Too few posts | 0.501 | 0.493–0.511 | 13 | 4 | 9 | | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | Too few posts | 0.499 | 0.491–0.506 | 5 | 3 | 2 | | [Stopping and steering a running agent](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | Too few posts | 0.500 | 0.496–0.504 | 2 | 1 | 1 | | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | Too few posts | 0.499 | 0.491–0.505 | 2 | 1 | 1 | ### Reliability and speed: 9th of 13 (customer love 0.501, n 132) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | 6th of 11 | 0.509 | 0.472–0.550 | 54 | 5 | 49 | | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | 8th of 10 | 0.510 | 0.487–0.533 | 45 | 16 | 29 | | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | 4th of 7 | 0.497 | 0.474–0.520 | 31 | 5 | 26 | | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | Too few posts | 0.504 | 0.487–0.531 | 10 | 1 | 9 | ### Account and support: 3rd of 11 (customer love 0.558, n 39) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | Too few posts | 0.517 | 0.496–0.542 | 15 | 4 | 11 | | [Data retention, training use and deployment isolation](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | Too few posts | 0.528 | 0.506–0.555 | 14 | 7 | 7 | | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | Too few posts | 0.493 | 0.488–0.497 | 12 | 0 | 12 | | [Account bans and access restrictions](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | Too few posts | 0.497 | 0.494–0.500 | 3 | 0 | 3 | # OpenAI Codex (OpenAI) Feedback Bench, coding agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/#/agent/codex | Measure | Value | |---|---| | Rank | 10 of 17 (rank range 8–11) | | Feedback Score | 47.7 (95% interval 46.6–48.8) | | Popularity | 0.980 (share of voice 27.35%) | | Customer love | 0.409 (95% interval 0.400–0.417) | | Place | Popularity 2nd of 17 × Customer love 16th of 17 | | Top quadrant | no | | Users | 42663 | | Posts counted | 188968 | | Posts that judge the agent | 60310 | | Criteria ranking high / low | 10 / 27 of 63 | ## Top requests What users ask to add or change, most asked first. 7525 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. | Rank | Request | Criterion | User-weeks | Posts | |---|---|---|---|---| | 1 | Banked resets usable on demand | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 150 | 156 | | 2 | Higher overall usage limits | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 147 | 155 | | 3 | Remove the 5-hour usage window | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | 140 | 153 | | 4 | Opt out of forced resets | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 125 | 140 | | 5 | Additional bonus usage resets | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 125 | 130 | | 6 | One-time immediate usage reset | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 116 | 127 | | 7 | Restore previous usage limits | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | 111 | 128 | | 8 | Faster overall response speed | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | 95 | 101 | | 9 | More token-efficient models | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 94 | 97 | | 10 | Resets that keep the normal reset date | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 92 | 102 | | 11 | Compensation reset after outages or bugs | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 86 | 90 | | 12 | Predictable fixed reset schedule without surprises | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 69 | 79 | ## Ranks highest on Using an existing subscription across tools (1st of 8), Safety filters block legitimate coding tasks (1st of 5), Computer use and browser control (1st of 4) ## Ranks lowest on Usage meter visibility and accuracy (6th of 6), Automatic model routing and fallback (8th of 8), Quality got worse or better over time (7th of 7) Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. ## All 63 criteria Criterion love: 0.5 is the category norm. n: rated user-weeks. ### Paying and limits: 13th of 13 (customer love 0.398, n 15635) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Single prompt, model or effort level consumes disproportionate quota](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | 9th of 12 | 0.482 | 0.462–0.507 | 6763 | 1129 | 5634 | | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 12th of 12 | 0.416 | 0.399–0.435 | 5816 | 2051 | 3765 | | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 5th of 5 | 0.448 | 0.413–0.485 | 3653 | 606 | 3047 | | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | 7th of 7 | 0.390 | 0.331–0.455 | 1838 | 69 | 1769 | | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | 4th of 6 | 0.481 | 0.440–0.529 | 1754 | 229 | 1525 | | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | 6th of 6 | 0.335 | 0.291–0.383 | 1137 | 48 | 1089 | | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | 6th of 6 | 0.413 | 0.363–0.472 | 336 | 10 | 326 | | [Prompt cache hits, misses and invalidation](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | 4th of 4 | 0.461 | 0.428–0.495 | 228 | 52 | 176 | | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 1st of 8 | 0.597 | 0.564–0.630 | 198 | 123 | 75 | | [Pay-as-you-go overage, fallback billing and spend caps](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | 1st of 4 | 0.556 | 0.483–0.643 | 146 | 14 | 132 | | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | 3rd of 7 | 0.509 | 0.480–0.537 | 136 | 84 | 52 | ### Setting up and connecting: 11th of 12 (customer love 0.438, n 1415) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | 3rd of 8 | 0.524 | 0.477–0.567 | 531 | 108 | 423 | | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | 7th of 9 | 0.463 | 0.434–0.494 | 392 | 176 | 216 | | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | 5th of 7 | 0.468 | 0.437–0.500 | 226 | 97 | 129 | | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | 2nd of 9 | 0.520 | 0.492–0.549 | 170 | 95 | 75 | | [Onboarding, discoverability and documentation](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | 7th of 7 | 0.461 | 0.433–0.494 | 133 | 18 | 115 | ### Choosing models: 11th of 12 (customer love 0.355, n 4611) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | 7th of 7 | 0.313 | 0.289–0.336 | 3117 | 527 | 2590 | | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 10th of 11 | 0.408 | 0.376–0.441 | 787 | 130 | 657 | | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | 2nd of 4 | 0.492 | 0.458–0.529 | 624 | 302 | 322 | | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | 8th of 8 | 0.388 | 0.359–0.419 | 447 | 65 | 382 | ### Instructing and context: 9th of 11 (customer love 0.470, n 2137) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Direct in-prompt instructions and caps are followed](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | 1st of 5 | 0.517 | 0.478–0.557 | 730 | 200 | 530 | | [Context compaction keeps what matters, cheaply and quickly](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | 5th of 5 | 0.453 | 0.418–0.487 | 434 | 138 | 296 | | [Persistent project rules files are read and obeyed](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | 2nd of 5 | 0.508 | 0.477–0.541 | 350 | 194 | 156 | | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | 6th of 6 | 0.457 | 0.425–0.487 | 281 | 108 | 173 | | [Output degrades as the context window fills](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | 1st of 5 | 0.525 | 0.481–0.570 | 276 | 54 | 222 | | [Finding the right files in the codebase](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | 3rd of 5 | 0.491 | 0.461–0.520 | 146 | 59 | 87 | | [Asks the user versus guessing](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | 2nd of 2 | 0.493 | 0.463–0.523 | 117 | 45 | 72 | | [Images, PDFs and file attachments as input](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | 2nd of 4 | 0.525 | 0.496–0.554 | 78 | 35 | 43 | ### Doing the work: 12th of 14 (customer love 0.435, n 9910) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Can do the user's kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | 12th of 13 | 0.382 | 0.364–0.399 | 5857 | 3404 | 2453 | | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | 10th of 10 | 0.438 | 0.409–0.470 | 1450 | 859 | 591 | | [Does unrequested work or over-engineers](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | 2nd of 5 | 0.528 | 0.455–0.609 | 824 | 48 | 776 | | [Spins, loops or gets stuck without progress](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | 6th of 6 | 0.423 | 0.350–0.502 | 572 | 19 | 553 | | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | 8th of 8 | 0.422 | 0.378–0.469 | 552 | 399 | 153 | | [Frontend and visual UI output](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | 5th of 5 | 0.421 | 0.384–0.456 | 524 | 197 | 327 | | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | 1st of 4 | 0.593 | 0.550–0.635 | 495 | 294 | 201 | | [Safety filters block legitimate coding tasks](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | 1st of 5 | 0.636 | 0.578–0.691 | 376 | 83 | 293 | | [Breaks existing code or reintroduces bugs](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | 5th of 5 | 0.473 | 0.411–0.546 | 337 | 19 | 318 | | [Stops mid-task or answers instead of acting](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | 3rd of 3 | 0.428 | 0.380–0.490 | 329 | 19 | 310 | | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | 3rd of 5 | 0.510 | 0.469–0.549 | 313 | 79 | 234 | | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | 1st of 3 | 0.574 | 0.538–0.612 | 209 | 80 | 129 | | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | 2nd of 5 | 0.487 | 0.450–0.525 | 207 | 43 | 164 | | [Diagnosing and fixing reported bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | 3rd of 4 | 0.485 | 0.446–0.522 | 204 | 139 | 65 | | [Caves to or argues with the user's judgement](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | 2nd of 2 | 0.476 | 0.450–0.506 | 105 | 18 | 87 | | [Plan-before-edit mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | 3rd of 5 | 0.480 | 0.456–0.505 | 84 | 42 | 42 | | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | 2nd of 3 | 0.497 | 0.475–0.522 | 62 | 20 | 42 | | [Games checks instead of fixing the problem](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | 2nd of 3 | 0.498 | 0.460–0.582 | 47 | 1 | 46 | ### Checking and finishing: 1st of 9 (customer love 0.541, n 788) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Agent-performed code review finds real issues](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | 1st of 4 | 0.585 | 0.543–0.621 | 371 | 310 | 61 | | [Claims work is done or fixed when it is not](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | 1st of 4 | 0.565 | 0.478–0.671 | 238 | 14 | 224 | | [Builds, tests or runs its own changes](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | 3rd of 3 | 0.461 | 0.429–0.491 | 138 | 71 | 67 | | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | 4th of 5 | 0.481 | 0.458–0.505 | 67 | 21 | 46 | ### Interface and sessions: 13th of 14 (customer love 0.407, n 2214) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 11th of 12 | 0.445 | 0.417–0.477 | 1157 | 343 | 814 | | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | 7th of 7 | 0.396 | 0.359–0.430 | 526 | 249 | 277 | | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | 4th of 6 | 0.490 | 0.451–0.528 | 323 | 76 | 247 | | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | 5th of 5 | 0.370 | 0.338–0.401 | 205 | 69 | 136 | | [Stopping and steering a running agent](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | 2nd of 3 | 0.492 | 0.459–0.528 | 107 | 43 | 64 | ### Reliability and speed: 13th of 13 (customer love 0.323, n 5291) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | 10th of 10 | 0.282 | 0.260–0.305 | 2493 | 463 | 2030 | | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | 11th of 11 | 0.353 | 0.308–0.404 | 1582 | 75 | 1507 | | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | 3rd of 6 | 0.498 | 0.440–0.559 | 1082 | 81 | 1001 | | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | 7th of 7 | 0.408 | 0.364–0.459 | 537 | 45 | 492 | ### Account and support: 5th of 11 (customer love 0.540, n 802) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | 2nd of 8 | 0.560 | 0.513–0.608 | 391 | 74 | 317 | | [Account bans and access restrictions](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | 1st of 5 | 0.552 | 0.495–0.624 | 218 | 25 | 193 | | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | 2nd of 5 | 0.541 | 0.430–0.691 | 187 | 5 | 182 | | [Data retention, training use and deployment isolation](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | 3rd of 4 | 0.479 | 0.453–0.509 | 96 | 16 | 80 | # Cline (Cline (open source)) Feedback Bench, coding agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/#/agent/cline | Measure | Value | |---|---| | Rank | 11 of 17 (rank range 9–11) | | Feedback Score | 46.7 (95% interval 45.2–48.3) | | Popularity | 0.497 (share of voice 1.12%) | | Customer love | 0.500 (95% interval 0.487–0.514) | | Place | Popularity 10th of 17 × Customer love 12th of 17 | | Top quadrant | no | | Users | 1885 | | Posts counted | 3121 | | Posts that judge the agent | 834 | | Criteria ranking high / low | 1 / 6 of 63 | ## Top requests What users ask to add or change, most asked first. 361 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. | Rank | Request | Criterion | User-weeks | Posts | |---|---|---|---|---| | 1 | Linux desktop app and Linux support | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | 56 | 63 | | 2 | Higher free tier usage limits | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | 8 | 9 | | 3 | Higher overall usage limits | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 6 | 6 | | 4 | DeepSeek V4.1 Flash model support | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 5 | 5 | | 5 | Faster, more responsive customer support | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | 5 | 5 | | 6 | Faster addition of newly released models | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 4 | 5 | | 7 | Additional free usage credits | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | 4 | 4 | | 8 | Clear free tier limits | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | 4 | 4 | | 9 | Control local desktop sessions from mobile app | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | 4 | 4 | | 10 | Cost per completed task metrics | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | 4 | 4 | | 11 | General desktop app stability fixes | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | 4 | 4 | | 12 | Make a specific model free | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | 4 | 4 | ## Ranks highest on Connecting own API keys, local models and custom endpoints (4th of 9) ## Ranks lowest on Single prompt, model or effort level consumes disproportionate quota (8th of 12), Can do the user's kind of task (8th of 13), Client crashes, freezes and failed tool execution (7th of 11) Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. ## All 63 criteria Criterion love: 0.5 is the category norm. n: rated user-weeks. ### Paying and limits: 9th of 13 (customer love 0.511, n 196) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | 7th of 7 | 0.459 | 0.438–0.482 | 78 | 30 | 48 | | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 7th of 12 | 0.492 | 0.476–0.511 | 56 | 20 | 36 | | [Single prompt, model or effort level consumes disproportionate quota](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | 8th of 12 | 0.487 | 0.471–0.504 | 30 | 3 | 27 | | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | Too few posts | 0.487 | 0.481–0.493 | 17 | 0 | 17 | | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | Too few posts | 0.495 | 0.491–0.498 | 8 | 0 | 8 | | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | Too few posts | 0.495 | 0.491–0.498 | 6 | 0 | 6 | | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | Too few posts | 0.518 | 0.496–0.552 | 6 | 2 | 4 | | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | Too few posts | 0.500 | 0.492–0.508 | 6 | 3 | 3 | | [Prompt cache hits, misses and invalidation](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | Too few posts | 0.501 | 0.495–0.509 | 5 | 2 | 3 | | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | Too few posts | 0.498 | 0.495–0.500 | 3 | 0 | 3 | | [Pay-as-you-go overage, fallback billing and spend caps](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | Too few posts | 0.498 | 0.496–0.500 | 2 | 0 | 2 | ### Setting up and connecting: 8th of 12 (customer love 0.497, n 102) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | 6th of 8 | 0.503 | 0.483–0.524 | 43 | 8 | 35 | | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | 4th of 9 | 0.509 | 0.493–0.525 | 30 | 19 | 11 | | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | Too few posts | 0.503 | 0.491–0.516 | 14 | 8 | 6 | | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | Too few posts | 0.488 | 0.479–0.497 | 13 | 2 | 11 | | [Onboarding, discoverability and documentation](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | Too few posts | 0.505 | 0.493–0.519 | 8 | 3 | 5 | ### Choosing models: 8th of 12 (customer love 0.502, n 54) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | Too few posts | 0.508 | 0.493–0.524 | 24 | 8 | 16 | | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | Too few posts | 0.500 | 0.491–0.510 | 11 | 3 | 8 | | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | Too few posts | 0.490 | 0.482–0.497 | 11 | 1 | 10 | | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | Too few posts | 0.501 | 0.492–0.511 | 11 | 3 | 8 | ### Instructing and context: 7th of 11 (customer love 0.498, n 41) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Finding the right files in the codebase](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | Too few posts | 0.499 | 0.491–0.508 | 10 | 4 | 6 | | [Context compaction keeps what matters, cheaply and quickly](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | Too few posts | 0.493 | 0.487–0.500 | 9 | 1 | 8 | | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | Too few posts | 0.501 | 0.496–0.507 | 5 | 3 | 2 | | [Images, PDFs and file attachments as input](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | Too few posts | 0.494 | 0.488–0.499 | 5 | 0 | 5 | | [Direct in-prompt instructions and caps are followed](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | Too few posts | 0.499 | 0.493–0.506 | 4 | 1 | 3 | | [Persistent project rules files are read and obeyed](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | Too few posts | 0.504 | 0.500–0.510 | 3 | 3 | 0 | | [Output degrades as the context window fills](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | Too few posts | 0.503 | 0.497–0.513 | 3 | 1 | 2 | | [Asks the user versus guessing](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | Too few posts | 0.502 | 0.497–0.508 | 2 | 1 | 1 | ### Doing the work: 9th of 14 (customer love 0.480, n 125) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Can do the user's kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | 8th of 13 | 0.483 | 0.460–0.505 | 66 | 36 | 30 | | [Spins, loops or gets stuck without progress](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | Too few posts | 0.502 | 0.484–0.535 | 17 | 1 | 16 | | [Plan-before-edit mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | Too few posts | 0.497 | 0.483–0.511 | 15 | 7 | 8 | | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | Too few posts | 0.497 | 0.484–0.507 | 9 | 7 | 2 | | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | Too few posts | 0.494 | 0.484–0.504 | 9 | 4 | 5 | | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | Too few posts | 0.503 | 0.493–0.514 | 6 | 2 | 4 | | [Breaks existing code or reintroduces bugs](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | Too few posts | 0.506 | 0.493–0.527 | 4 | 1 | 3 | | [Diagnosing and fixing reported bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | Too few posts | 0.502 | 0.497–0.508 | 3 | 2 | 1 | | [Stops mid-task or answers instead of acting](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | Too few posts | 0.498 | 0.494–0.500 | 3 | 0 | 3 | | [Games checks instead of fixing the problem](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | Too few posts | 0.498 | 0.496–0.500 | 2 | 0 | 2 | | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | Too few posts | 0.498 | 0.495–0.500 | 2 | 0 | 2 | | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | Too few posts | 0.501 | 0.497–0.507 | 2 | 1 | 1 | | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | Too few posts | 0.499 | 0.494–0.503 | 2 | 1 | 1 | | [Safety filters block legitimate coding tasks](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | Too few posts | 0.506 | 0.498–0.522 | 2 | 1 | 1 | | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | Too few posts | 0.506 | 0.500–0.517 | 2 | 2 | 0 | | [Does unrequested work or over-engineers](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | Too few posts | 0.499 | 0.497–0.500 | 1 | 0 | 1 | | [Frontend and visual UI output](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Caves to or argues with the user's judgement](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Checking and finishing: Too few posts (customer love 0.499, n 9) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Builds, tests or runs its own changes](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | Too few posts | 0.502 | 0.496–0.507 | 4 | 3 | 1 | | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | Too few posts | 0.502 | 0.496–0.514 | 4 | 2 | 2 | | [Claims work is done or fixed when it is not](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | Too few posts | 0.498 | 0.495–0.500 | 2 | 0 | 2 | | [Agent-performed code review finds real issues](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | Too few posts | 0.501 | 0.500–0.502 | 1 | 1 | 0 | ### Interface and sessions: 7th of 14 (customer love 0.514, n 45) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | Too few posts | 0.515 | 0.502–0.530 | 28 | 16 | 12 | | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | Too few posts | 0.496 | 0.487–0.505 | 11 | 2 | 9 | | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | Too few posts | 0.505 | 0.501–0.511 | 4 | 4 | 0 | | [Stopping and steering a running agent](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | Too few posts | 0.500 | 0.496–0.503 | 2 | 1 | 1 | | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | Too few posts | 0.503 | 0.500–0.508 | 2 | 2 | 0 | ### Reliability and speed: 8th of 13 (customer love 0.510, n 136) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | 7th of 11 | 0.484 | 0.456–0.517 | 66 | 3 | 63 | | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | 6th of 10 | 0.524 | 0.502–0.546 | 57 | 24 | 33 | | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | Too few posts | 0.492 | 0.481–0.507 | 19 | 1 | 18 | | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | Too few posts | 0.494 | 0.491–0.498 | 8 | 0 | 8 | ### Account and support: Too few posts (customer love 0.482, n 25) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | Too few posts | 0.489 | 0.482–0.495 | 16 | 0 | 16 | | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | Too few posts | 0.492 | 0.488–0.496 | 11 | 0 | 11 | | [Account bans and access restrictions](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | | [Data retention, training use and deployment isolation](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | # Conductor (Melty Labs) Feedback Bench, coding agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/#/agent/conductor | Measure | Value | |---|---| | Rank | 12 of 17 (rank range 12–13) | | Feedback Score | 43.3 (95% interval 42.4–44.3) | | Popularity | 0.305 (share of voice 0.32%) | | Customer love | 0.512 (95% interval 0.504–0.521) | | Place | Popularity 14th of 17 × Customer love 7th of 17 | | Top quadrant | no | | Users | 535 | | Posts counted | 792 | | Posts that judge the agent | 326 | | Criteria ranking high / low | 1 / 0 of 63 | ## Top requests What users ask to add or change, most asked first. 146 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. | Rank | Request | Criterion | User-weeks | Posts | |---|---|---|---|---| | 1 | Customizable model picker | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 6 | 6 | | 2 | Control local desktop sessions from mobile app | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | 4 | 5 | | 3 | Native iOS app | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | 4 | 4 | | 4 | Official mobile app | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | 4 | 4 | | 5 | Restore removed features | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | 4 | 4 | | 6 | Windows support and Windows desktop app | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | 4 | 4 | | 7 | Cloud agents controlled from mobile | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | 3 | 3 | | 8 | New Haiku model release and support | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 3 | 3 | | 9 | Newer Grok model support | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 3 | 3 | | 10 | Agent completion and attention notifications | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 2 | 2 | | 11 | Cloud agent execution support | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | 2 | 2 | | 12 | Confirmation before archiving a workspace | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | 2 | 2 | ## Ranks highest on How the interface shows work, and what the user can configure (5th of 12) ## Ranks lowest on No criterion where it sits clearly in the bottom half. Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. ## All 63 criteria Criterion love: 0.5 is the category norm. n: rated user-weeks. ### Paying and limits: Too few posts (customer love 0.521, n 23) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | Too few posts | 0.504 | 0.494–0.516 | 12 | 6 | 6 | | [Single prompt, model or effort level consumes disproportionate quota](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | Too few posts | 0.496 | 0.493–0.499 | 4 | 0 | 4 | | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | Too few posts | 0.500 | 0.495–0.504 | 3 | 2 | 1 | | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | Too few posts | 0.505 | 0.500–0.513 | 2 | 2 | 0 | | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | Too few posts | 0.499 | 0.496–0.500 | 1 | 0 | 1 | | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | Too few posts | 0.522 | 0.500–0.568 | 1 | 1 | 0 | | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Prompt cache hits, misses and invalidation](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Pay-as-you-go overage, fallback billing and spend caps](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Setting up and connecting: Too few posts (customer love 0.483, n 22) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | Too few posts | 0.493 | 0.486–0.500 | 8 | 2 | 6 | | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | Too few posts | 0.496 | 0.489–0.503 | 7 | 1 | 6 | | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | Too few posts | 0.494 | 0.486–0.500 | 6 | 1 | 5 | | [Onboarding, discoverability and documentation](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Choosing models: Too few posts (customer love 0.513, n 29) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | Too few posts | 0.509 | 0.494–0.526 | 17 | 7 | 10 | | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | Too few posts | 0.505 | 0.492–0.520 | 10 | 4 | 6 | | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | Too few posts | 0.499 | 0.496–0.500 | 1 | 0 | 1 | | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | Too few posts | 0.504 | 0.500–0.511 | 1 | 1 | 0 | ### Instructing and context: Too few posts (customer love 0.501, n 9) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | Too few posts | 0.500 | 0.495–0.504 | 4 | 2 | 2 | | [Finding the right files in the codebase](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | Too few posts | 0.501 | 0.498–0.505 | 2 | 1 | 1 | | [Direct in-prompt instructions and caps are followed](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | Too few posts | 0.499 | 0.497–0.500 | 1 | 0 | 1 | | [Context compaction keeps what matters, cheaply and quickly](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | Too few posts | 0.502 | 0.500–0.505 | 1 | 1 | 0 | | [Images, PDFs and file attachments as input](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | Too few posts | 0.499 | 0.495–0.500 | 1 | 0 | 1 | | [Persistent project rules files are read and obeyed](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Asks the user versus guessing](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Output degrades as the context window fills](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Doing the work: 6th of 14 (customer love 0.522, n 44) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | Too few posts | 0.510 | 0.501–0.519 | 19 | 16 | 3 | | [Can do the user's kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | Too few posts | 0.504 | 0.496–0.512 | 10 | 7 | 3 | | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | Too few posts | 0.509 | 0.501–0.518 | 6 | 5 | 1 | | [Frontend and visual UI output](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | Too few posts | 0.501 | 0.497–0.506 | 3 | 2 | 1 | | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | Too few posts | 0.499 | 0.492–0.504 | 3 | 2 | 1 | | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | Too few posts | 0.497 | 0.493–0.500 | 3 | 0 | 3 | | [Diagnosing and fixing reported bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | Too few posts | 0.502 | 0.500–0.504 | 2 | 2 | 0 | | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | Too few posts | 0.500 | 0.497–0.504 | 2 | 1 | 1 | | [Plan-before-edit mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | Too few posts | 0.501 | 0.500–0.504 | 1 | 1 | 0 | | [Breaks existing code or reintroduces bugs](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Does unrequested work or over-engineers](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Spins, loops or gets stuck without progress](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Stops mid-task or answers instead of acting](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Games checks instead of fixing the problem](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Safety filters block legitimate coding tasks](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Caves to or argues with the user's judgement](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Checking and finishing: Too few posts (customer love 0.503, n 3) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | Too few posts | 0.504 | 0.498–0.513 | 3 | 2 | 1 | | [Claims work is done or fixed when it is not](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Builds, tests or runs its own changes](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Agent-performed code review finds real issues](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Interface and sessions: 10th of 14 (customer love 0.506, n 76) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 5th of 12 | 0.502 | 0.486–0.519 | 35 | 13 | 22 | | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | Too few posts | 0.498 | 0.483–0.514 | 28 | 16 | 12 | | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | Too few posts | 0.490 | 0.476–0.504 | 22 | 8 | 14 | | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | Too few posts | 0.503 | 0.495–0.514 | 2 | 1 | 1 | | [Stopping and steering a running agent](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Reliability and speed: Too few posts (customer love 0.492, n 17) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | Too few posts | 0.493 | 0.489–0.497 | 11 | 0 | 11 | | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | Too few posts | 0.496 | 0.492–0.499 | 5 | 0 | 5 | | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | Too few posts | 0.502 | 0.498–0.507 | 2 | 1 | 1 | | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Account and support: Too few posts (customer love 0.510, n 8) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | Too few posts | 0.509 | 0.493–0.529 | 6 | 2 | 4 | | [Account bans and access restrictions](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | Too few posts | 0.498 | 0.494–0.500 | 2 | 0 | 2 | | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | | [Data retention, training use and deployment isolation](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | # Kiro (AWS) Feedback Bench, coding agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/#/agent/kiro | Measure | Value | |---|---| | Rank | 13 of 17 (rank range 12–15) | | Feedback Score | 41.6 (95% interval 40.4–43.1) | | Popularity | 0.404 (share of voice 0.61%) | | Customer love | 0.478 (95% interval 0.467–0.490) | | Place | Popularity 13th of 17 × Customer love 15th of 17 | | Top quadrant | no | | Users | 919 | | Posts counted | 1947 | | Posts that judge the agent | 609 | | Criteria ranking high / low | 0 / 4 of 63 | ## Top requests What users ask to add or change, most asked first. 239 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. | Rank | Request | Criterion | User-weeks | Posts | |---|---|---|---|---| | 1 | Opus 5.5 model support | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 29 | 37 | | 2 | Expand student program to more universities and countries | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | 21 | 21 | | 3 | Faster addition of newly released models | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 13 | 14 | | 4 | Fix declined cards and failed payments | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | 11 | 12 | | 5 | Astra model availability | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 9 | 9 | | 6 | More open-weight models | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 8 | 8 | | 7 | DeepSeek V4.1 Flash model support | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 6 | 7 | | 8 | GPT-6 Sol and Luna model support | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 6 | 7 | | 9 | Fable model availability | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 6 | 6 | | 10 | Transparent, consistent model cost multipliers | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | 6 | 6 | | 11 | Refund for accidental or mistaken purchase | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | 5 | 7 | | 12 | Sonnet 5.5 model support | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 5 | 5 | ## Ranks highest on No criterion where it sits clearly in the top half. ## Ranks lowest on Which models are offered on a plan and when (9th of 11), How much use a plan's price buys (10th of 12), Single prompt, model or effort level consumes disproportionate quota (10th of 12) Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. ## All 63 criteria Criterion love: 0.5 is the category norm. n: rated user-weeks. ### Paying and limits: 11th of 13 (customer love 0.454, n 142) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Single prompt, model or effort level consumes disproportionate quota](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | 10th of 12 | 0.478 | 0.457–0.499 | 59 | 6 | 53 | | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 10th of 12 | 0.484 | 0.465–0.501 | 50 | 16 | 34 | | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | Too few posts | 0.481 | 0.474–0.487 | 27 | 0 | 27 | | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | Too few posts | 0.488 | 0.479–0.496 | 10 | 1 | 9 | | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | Too few posts | 0.494 | 0.490–0.498 | 8 | 0 | 8 | | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | Too few posts | 0.518 | 0.501–0.539 | 6 | 4 | 2 | | [Pay-as-you-go overage, fallback billing and spend caps](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | Too few posts | 0.495 | 0.490–0.499 | 6 | 0 | 6 | | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | Too few posts | 0.497 | 0.493–0.500 | 4 | 0 | 4 | | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | Too few posts | 0.498 | 0.495–0.500 | 2 | 0 | 2 | | [Prompt cache hits, misses and invalidation](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | Too few posts | 0.499 | 0.497–0.500 | 1 | 0 | 1 | | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Setting up and connecting: Too few posts (customer love 0.487, n 28) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | Too few posts | 0.491 | 0.482–0.500 | 11 | 3 | 8 | | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | Too few posts | 0.500 | 0.492–0.508 | 8 | 4 | 4 | | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | Too few posts | 0.495 | 0.492–0.498 | 7 | 0 | 7 | | [Onboarding, discoverability and documentation](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | Too few posts | 0.496 | 0.492–0.499 | 5 | 0 | 5 | | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | Too few posts | 0.500 | 0.495–0.504 | 2 | 1 | 1 | ### Choosing models: 9th of 12 (customer love 0.480, n 69) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 9th of 11 | 0.478 | 0.460–0.497 | 56 | 7 | 49 | | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | Too few posts | 0.508 | 0.495–0.523 | 7 | 3 | 4 | | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | Too few posts | 0.500 | 0.493–0.510 | 4 | 1 | 3 | | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | Too few posts | 0.495 | 0.488–0.500 | 3 | 0 | 3 | ### Instructing and context: Too few posts (customer love 0.506, n 22) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Persistent project rules files are read and obeyed](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | Too few posts | 0.504 | 0.500–0.510 | 6 | 5 | 1 | | [Context compaction keeps what matters, cheaply and quickly](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | Too few posts | 0.503 | 0.496–0.513 | 5 | 2 | 3 | | [Output degrades as the context window fills](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | Too few posts | 0.507 | 0.498–0.521 | 4 | 2 | 2 | | [Images, PDFs and file attachments as input](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | Too few posts | 0.501 | 0.494–0.510 | 3 | 1 | 2 | | [Direct in-prompt instructions and caps are followed](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | Too few posts | 0.498 | 0.495–0.500 | 2 | 0 | 2 | | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | Too few posts | 0.500 | 0.495–0.505 | 2 | 1 | 1 | | [Finding the right files in the codebase](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | Too few posts | 0.498 | 0.494–0.500 | 1 | 0 | 1 | | [Asks the user versus guessing](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Doing the work: 8th of 14 (customer love 0.486, n 100) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Can do the user's kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | 7th of 13 | 0.485 | 0.461–0.507 | 56 | 31 | 25 | | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | Too few posts | 0.492 | 0.481–0.505 | 12 | 1 | 11 | | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | Too few posts | 0.500 | 0.490–0.509 | 10 | 6 | 4 | | [Plan-before-edit mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | Too few posts | 0.512 | 0.504–0.521 | 9 | 9 | 0 | | [Does unrequested work or over-engineers](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | Too few posts | 0.494 | 0.490–0.498 | 7 | 0 | 7 | | [Breaks existing code or reintroduces bugs](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | Too few posts | 0.496 | 0.493–0.499 | 6 | 0 | 6 | | [Spins, loops or gets stuck without progress](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | Too few posts | 0.494 | 0.489–0.498 | 6 | 0 | 6 | | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | Too few posts | 0.499 | 0.492–0.505 | 3 | 2 | 1 | | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | Too few posts | 0.499 | 0.493–0.505 | 3 | 1 | 2 | | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | Too few posts | 0.498 | 0.495–0.500 | 2 | 0 | 2 | | [Safety filters block legitimate coding tasks](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | Too few posts | 0.504 | 0.498–0.515 | 2 | 1 | 1 | | [Diagnosing and fixing reported bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | Too few posts | 0.501 | 0.500–0.506 | 1 | 1 | 0 | | [Stops mid-task or answers instead of acting](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | | [Frontend and visual UI output](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Games checks instead of fixing the problem](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Caves to or argues with the user's judgement](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Checking and finishing: Too few posts (customer love 0.501, n 4) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Claims work is done or fixed when it is not](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | Too few posts | 0.499 | 0.499–0.500 | 1 | 0 | 1 | | [Builds, tests or runs its own changes](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | Too few posts | 0.502 | 0.500–0.505 | 1 | 1 | 0 | | [Agent-performed code review finds real issues](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | Too few posts | 0.501 | 0.500–0.504 | 1 | 1 | 0 | | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | Too few posts | 0.498 | 0.494–0.500 | 1 | 0 | 1 | ### Interface and sessions: Too few posts (customer love 0.497, n 17) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | Too few posts | 0.497 | 0.488–0.507 | 10 | 2 | 8 | | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | Too few posts | 0.503 | 0.497–0.511 | 3 | 2 | 1 | | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | Too few posts | 0.499 | 0.495–0.503 | 2 | 1 | 1 | | [Stopping and steering a running agent](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | Too few posts | 0.499 | 0.496–0.500 | 1 | 0 | 1 | | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | Too few posts | 0.501 | 0.500–0.502 | 1 | 1 | 0 | ### Reliability and speed: 11th of 13 (customer love 0.488, n 46) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | Too few posts | 0.495 | 0.484–0.511 | 18 | 4 | 14 | | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | Too few posts | 0.503 | 0.484–0.539 | 16 | 1 | 15 | | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | Too few posts | 0.509 | 0.483–0.547 | 14 | 2 | 12 | | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | Too few posts | 0.498 | 0.494–0.500 | 2 | 0 | 2 | ### Account and support: 8th of 11 (customer love 0.492, n 86) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | 3rd of 5 | 0.510 | 0.460–0.605 | 46 | 1 | 45 | | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | 6th of 8 | 0.481 | 0.462–0.504 | 45 | 3 | 42 | | [Account bans and access restrictions](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | Too few posts | 0.488 | 0.482–0.494 | 15 | 0 | 15 | | [Data retention, training use and deployment isolation](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | Too few posts | 0.515 | 0.497–0.534 | 6 | 3 | 3 | # Google Antigravity (Google) Feedback Bench, coding agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/#/agent/antigravity | Measure | Value | |---|---| | Rank | 14 of 17 (rank range 13–15) | | Feedback Score | 41.3 (95% interval 40.0–42.7) | | Popularity | 0.807 (share of voice 8.69%) | | Customer love | 0.394 (95% interval 0.382–0.405) | | Place | Popularity 4th of 17 × Customer love 17th of 17 | | Top quadrant | no | | Users | 12223 | | Posts counted | 30817 | | Posts that judge the agent | 10752 | | Criteria ranking high / low | 2 / 30 of 63 | ## Top requests What users ask to add or change, most asked first. 2873 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. | Rank | Request | Criterion | User-weeks | Posts | |---|---|---|---|---| | 1 | Newest flagship model on the Pro plan | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 183 | 208 | | 2 | Release Gemini 4 / Argon models | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 79 | 86 | | 3 | Update outdated third-party model versions | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 61 | 64 | | 4 | Subscriptions usable in third-party harnesses | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 53 | 53 | | 5 | Higher overall usage limits | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 52 | 56 | | 6 | Equal model access for promo and student subscribers | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 44 | 47 | | 7 | One-time immediate usage reset | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 40 | 41 | | 8 | Higher limits for specific models | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 35 | 35 | | 9 | Keep older models available | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 33 | 34 | | 10 | Restore removed models | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 32 | 34 | | 11 | Opus 5.5 model support | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 27 | 29 | | 12 | Google/Gemini subscription usable in other harnesses | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 24 | 25 | ## Ranks highest on Quota reset timing and bonus or banked resets (2nd of 5), Quality got worse or better over time (3rd of 7) ## Ranks lowest on IDE and editor integration (7th of 7), Which models are offered on a plan and when (11th of 11), Images, PDFs and file attachments as input (4th of 4) Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. ## All 63 criteria Criterion love: 0.5 is the category norm. n: rated user-weeks. ### Paying and limits: 12th of 13 (customer love 0.418, n 2124) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 11th of 12 | 0.426 | 0.398–0.455 | 908 | 328 | 580 | | [Single prompt, model or effort level consumes disproportionate quota](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | 12th of 12 | 0.368 | 0.329–0.402 | 861 | 91 | 770 | | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | 5th of 7 | 0.483 | 0.411–0.554 | 183 | 7 | 176 | | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | 6th of 6 | 0.420 | 0.392–0.451 | 174 | 10 | 164 | | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | 2nd of 5 | 0.551 | 0.508–0.590 | 117 | 25 | 92 | | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | 5th of 7 | 0.476 | 0.446–0.504 | 102 | 58 | 44 | | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | 3rd of 6 | 0.504 | 0.461–0.551 | 76 | 8 | 68 | | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | 6th of 8 | 0.470 | 0.451–0.489 | 64 | 13 | 51 | | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | 5th of 6 | 0.470 | 0.437–0.513 | 61 | 2 | 59 | | [Prompt cache hits, misses and invalidation](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | Too few posts | 0.495 | 0.483–0.506 | 8 | 1 | 7 | | [Pay-as-you-go overage, fallback billing and spend caps](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | Too few posts | 0.493 | 0.487–0.498 | 8 | 0 | 8 | ### Setting up and connecting: 12th of 12 (customer love 0.356, n 627) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | 7th of 7 | 0.394 | 0.366–0.420 | 208 | 59 | 149 | | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | 8th of 8 | 0.422 | 0.394–0.452 | 204 | 18 | 186 | | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | 9th of 9 | 0.445 | 0.419–0.472 | 148 | 48 | 100 | | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | 8th of 9 | 0.463 | 0.443–0.483 | 50 | 13 | 37 | | [Onboarding, discoverability and documentation](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | 6th of 7 | 0.474 | 0.459–0.490 | 42 | 4 | 38 | ### Choosing models: 12th of 12 (customer love 0.338, n 1345) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | 11th of 11 | 0.241 | 0.219–0.268 | 790 | 71 | 719 | | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | 3rd of 7 | 0.535 | 0.500–0.570 | 490 | 152 | 338 | | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | 7th of 8 | 0.461 | 0.440–0.483 | 67 | 10 | 57 | | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | 3rd of 4 | 0.485 | 0.465–0.507 | 65 | 28 | 37 | ### Instructing and context: 11th of 11 (customer love 0.388, n 526) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Direct in-prompt instructions and caps are followed](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | 5th of 5 | 0.469 | 0.440–0.499 | 162 | 34 | 128 | | [Output degrades as the context window fills](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | 5th of 5 | 0.467 | 0.442–0.498 | 94 | 10 | 84 | | [Finding the right files in the codebase](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | 5th of 5 | 0.440 | 0.416–0.461 | 74 | 15 | 59 | | [Persistent project rules files are read and obeyed](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | 4th of 5 | 0.482 | 0.461–0.504 | 71 | 32 | 39 | | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | 5th of 6 | 0.461 | 0.441–0.481 | 62 | 12 | 50 | | [Images, PDFs and file attachments as input](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | 4th of 4 | 0.453 | 0.434–0.471 | 53 | 7 | 46 | | [Context compaction keeps what matters, cheaply and quickly](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | 4th of 5 | 0.480 | 0.464–0.499 | 45 | 9 | 36 | | [Asks the user versus guessing](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | Too few posts | 0.513 | 0.499–0.530 | 19 | 11 | 8 | ### Doing the work: 14th of 14 (customer love 0.352, n 2380) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Can do the user's kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | 13th of 13 | 0.325 | 0.303–0.346 | 1419 | 657 | 762 | | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | 5th of 5 | 0.379 | 0.353–0.406 | 253 | 19 | 234 | | [Spins, loops or gets stuck without progress](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | 4th of 6 | 0.473 | 0.410–0.539 | 188 | 6 | 182 | | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | 8th of 10 | 0.480 | 0.452–0.509 | 166 | 96 | 70 | | [Plan-before-edit mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | 5th of 5 | 0.466 | 0.436–0.494 | 116 | 56 | 60 | | [Does unrequested work or over-engineers](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | 3rd of 5 | 0.512 | 0.456–0.574 | 99 | 7 | 92 | | [Breaks existing code or reintroduces bugs](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | 2nd of 5 | 0.506 | 0.467–0.553 | 81 | 7 | 74 | | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | 5th of 5 | 0.467 | 0.448–0.486 | 75 | 8 | 67 | | [Frontend and visual UI output](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | 4th of 5 | 0.498 | 0.473–0.525 | 72 | 33 | 39 | | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | 3rd of 4 | 0.474 | 0.452–0.497 | 62 | 24 | 38 | | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | 6th of 8 | 0.485 | 0.456–0.513 | 60 | 41 | 19 | | [Diagnosing and fixing reported bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | 4th of 4 | 0.470 | 0.443–0.495 | 45 | 22 | 23 | | [Safety filters block legitimate coding tasks](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | 3rd of 5 | 0.509 | 0.484–0.537 | 33 | 6 | 27 | | [Games checks instead of fixing the problem](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | 1st of 3 | 0.511 | 0.470–0.602 | 32 | 1 | 31 | | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | Too few posts | 0.523 | 0.506–0.544 | 25 | 14 | 11 | | [Stops mid-task or answers instead of acting](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | Too few posts | 0.481 | 0.472–0.489 | 21 | 0 | 21 | | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | Too few posts | 0.493 | 0.482–0.504 | 17 | 4 | 13 | | [Caves to or argues with the user's judgement](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | Too few posts | 0.500 | 0.487–0.517 | 17 | 4 | 13 | ### Checking and finishing: 9th of 9 (customer love 0.387, n 176) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Claims work is done or fixed when it is not](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | 3rd of 4 | 0.451 | 0.426–0.490 | 91 | 1 | 90 | | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | 3rd of 5 | 0.486 | 0.468–0.503 | 35 | 10 | 25 | | [Builds, tests or runs its own changes](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | 2nd of 3 | 0.481 | 0.463–0.498 | 31 | 13 | 18 | | [Agent-performed code review finds real issues](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | Too few posts | 0.467 | 0.439–0.493 | 26 | 12 | 14 | ### Interface and sessions: 14th of 14 (customer love 0.399, n 472) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 12th of 12 | 0.414 | 0.386–0.444 | 254 | 49 | 205 | | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | 6th of 7 | 0.492 | 0.464–0.519 | 111 | 59 | 52 | | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | 6th of 6 | 0.441 | 0.421–0.463 | 89 | 9 | 80 | | [Stopping and steering a running agent](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | Too few posts | 0.495 | 0.483–0.506 | 22 | 9 | 13 | | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | Too few posts | 0.498 | 0.483–0.512 | 22 | 11 | 11 | ### Reliability and speed: 10th of 13 (customer love 0.493, n 1274) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | 9th of 10 | 0.464 | 0.429–0.496 | 760 | 206 | 554 | | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | 10th of 11 | 0.433 | 0.384–0.483 | 315 | 13 | 302 | | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | 6th of 6 | 0.454 | 0.408–0.502 | 194 | 8 | 186 | | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | 2nd of 7 | 0.516 | 0.474–0.560 | 110 | 17 | 93 | ### Account and support: 10th of 11 (customer love 0.430, n 258) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Account bans and access restrictions](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | 4th of 5 | 0.449 | 0.428–0.479 | 122 | 4 | 118 | | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | 5th of 8 | 0.483 | 0.452–0.515 | 99 | 12 | 87 | | [Data retention, training use and deployment isolation](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | Too few posts | 0.484 | 0.473–0.496 | 27 | 1 | 26 | | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | Too few posts | 0.477 | 0.468–0.486 | 26 | 0 | 26 | # Warp (Warp) Feedback Bench, coding agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/#/agent/warp | Measure | Value | |---|---| | Rank | 15 of 17 (rank range 13–15) | | Feedback Score | 40.6 (95% interval 39.9–41.3) | | Popularity | 0.246 (share of voice 0.21%) | | Customer love | 0.502 (95% interval 0.497–0.508) | | Place | Popularity 15th of 17 × Customer love 10th of 17 | | Top quadrant | no | | Users | 374 | | Posts counted | 586 | | Posts that judge the agent | 194 | | Criteria ranking high / low | 0 / 0 of 63 | ## Top requests What users ask to add or change, most asked first. 45 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. | Rank | Request | Criterion | User-weeks | Posts | |---|---|---|---|---| | 1 | BYOK without paid plan or credit gating | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | 2 | 2 | | 2 | Built-in first-class subagent orchestration | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | 2 | 2 | | 3 | Cheaper low-priced entry plan tier | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | 2 | 2 | | 4 | Confirmation before quitting with running work | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | 2 | 2 | | 5 | Native iOS app | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | 2 | 2 | | 6 | Reproducible replay of tasks | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | 2 | 2 | ## Ranks highest on No criterion where it sits clearly in the top half. ## Ranks lowest on No criterion where it sits clearly in the bottom half. Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. ## All 63 criteria Criterion love: 0.5 is the category norm. n: rated user-weeks. ### Paying and limits: Too few posts (customer love 0.492, n 24) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Single prompt, model or effort level consumes disproportionate quota](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | Too few posts | 0.495 | 0.489–0.503 | 10 | 1 | 9 | | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | Too few posts | 0.497 | 0.490–0.503 | 7 | 2 | 5 | | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | Too few posts | 0.497 | 0.494–0.500 | 4 | 0 | 4 | | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | Too few posts | 0.498 | 0.496–0.500 | 2 | 0 | 2 | | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | Too few posts | 0.497 | 0.494–0.500 | 2 | 0 | 2 | | [Prompt cache hits, misses and invalidation](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | Too few posts | 0.501 | 0.498–0.506 | 2 | 1 | 1 | | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | Too few posts | 0.510 | 0.499–0.536 | 2 | 1 | 1 | | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | Too few posts | 0.497 | 0.492–0.500 | 2 | 0 | 2 | | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Pay-as-you-go overage, fallback billing and spend caps](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Setting up and connecting: Too few posts (customer love 0.485, n 18) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | Too few posts | 0.493 | 0.484–0.499 | 9 | 2 | 7 | | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | Too few posts | 0.496 | 0.493–0.499 | 4 | 0 | 4 | | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | Too few posts | 0.498 | 0.494–0.500 | 2 | 0 | 2 | | [Onboarding, discoverability and documentation](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | Too few posts | 0.498 | 0.494–0.500 | 2 | 0 | 2 | | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | Too few posts | 0.499 | 0.496–0.500 | 1 | 0 | 1 | ### Choosing models: Too few posts (customer love 0.498, n 5) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | Too few posts | 0.498 | 0.496–0.500 | 2 | 0 | 2 | | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | Too few posts | 0.502 | 0.500–0.506 | 1 | 1 | 0 | | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | Too few posts | 0.499 | 0.497–0.500 | 1 | 0 | 1 | | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | Too few posts | 0.498 | 0.494–0.500 | 1 | 0 | 1 | ### Instructing and context: Too few posts (customer love 0.502, n 3) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | Too few posts | 0.502 | 0.500–0.506 | 2 | 2 | 0 | | [Output degrades as the context window fills](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | | [Persistent project rules files are read and obeyed](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Direct in-prompt instructions and caps are followed](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Asks the user versus guessing](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Context compaction keeps what matters, cheaply and quickly](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Finding the right files in the codebase](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Images, PDFs and file attachments as input](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Doing the work: Too few posts (customer love 0.516, n 20) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Can do the user's kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | Too few posts | 0.509 | 0.503–0.516 | 10 | 9 | 1 | | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | Too few posts | 0.502 | 0.495–0.510 | 8 | 5 | 3 | | [Frontend and visual UI output](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | Too few posts | 0.500 | 0.497–0.504 | 2 | 1 | 1 | | [Diagnosing and fixing reported bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | Too few posts | 0.502 | 0.500–0.506 | 1 | 1 | 0 | | [Does unrequested work or over-engineers](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | Too few posts | 0.499 | 0.499–0.500 | 1 | 0 | 1 | | [Spins, loops or gets stuck without progress](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | Too few posts | 0.499 | 0.499–0.500 | 1 | 0 | 1 | | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | Too few posts | 0.499 | 0.496–0.500 | 1 | 0 | 1 | | [Breaks existing code or reintroduces bugs](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Stops mid-task or answers instead of acting](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Games checks instead of fixing the problem](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Safety filters block legitimate coding tasks](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Plan-before-edit mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Caves to or argues with the user's judgement](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Checking and finishing: Too few posts (customer love 0.500, n 2) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Claims work is done or fixed when it is not](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | Too few posts | 0.501 | 0.500–0.504 | 1 | 1 | 0 | | [Builds, tests or runs its own changes](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Agent-performed code review finds real issues](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Interface and sessions: 9th of 14 (customer love 0.507, n 33) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | Too few posts | 0.508 | 0.494–0.523 | 23 | 10 | 13 | | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | Too few posts | 0.500 | 0.494–0.507 | 7 | 2 | 5 | | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | Too few posts | 0.504 | 0.501–0.507 | 4 | 4 | 0 | | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | Too few posts | 0.501 | 0.500–0.502 | 1 | 1 | 0 | | [Stopping and steering a running agent](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Reliability and speed: Too few posts (customer love 0.511, n 24) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | Too few posts | 0.490 | 0.483–0.496 | 14 | 0 | 14 | | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | Too few posts | 0.514 | 0.502–0.529 | 8 | 6 | 2 | | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | Too few posts | 0.499 | 0.497–0.500 | 1 | 0 | 1 | ### Account and support: Too few posts (customer love 0.505, n 10) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | Too few posts | 0.504 | 0.493–0.518 | 9 | 2 | 7 | | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | Too few posts | 0.499 | 0.497–0.500 | 1 | 0 | 1 | | [Account bans and access restrictions](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Data retention, training use and deployment isolation](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | # Grok Build (xAI) Feedback Bench, coding agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/#/agent/grok-build | Measure | Value | |---|---| | Rank | 16 of 17 (rank range 16–16) | | Feedback Score | 35.6 (95% interval 35.1–36.2) | | Popularity | 0.030 (share of voice 0.05%) | | Customer love | 0.507 (95% interval 0.502–0.512) | | Place | Popularity 16th of 17 × Customer love 8th of 17 | | Top quadrant | no | | Users | 92 | | Posts counted | 105 | | Posts that judge the agent | 50 | | Criteria ranking high / low | 0 / 0 of 63 | ## Top requests What users ask to add or change, most asked first. 9 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. No request is asked for in enough user-weeks to show. ## Ranks highest on No criterion where it sits clearly in the top half. ## Ranks lowest on No criterion where it sits clearly in the bottom half. Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. ## All 63 criteria Criterion love: 0.5 is the category norm. n: rated user-weeks. ### Paying and limits: Too few posts (customer love 0.509, n 12) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | Too few posts | 0.501 | 0.494–0.509 | 8 | 4 | 4 | | [Single prompt, model or effort level consumes disproportionate quota](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | Too few posts | 0.507 | 0.498–0.517 | 3 | 2 | 1 | | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | Too few posts | 0.498 | 0.496–0.500 | 2 | 0 | 2 | | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Prompt cache hits, misses and invalidation](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Pay-as-you-go overage, fallback billing and spend caps](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Setting up and connecting: Too few posts (customer love 0.498, n 2) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | Too few posts | 0.498 | 0.495–0.500 | 2 | 0 | 2 | | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Onboarding, discoverability and documentation](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Choosing models: Too few posts (customer love 0.505, n 2) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | Too few posts | 0.502 | 0.500–0.507 | 1 | 1 | 0 | | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | Too few posts | 0.503 | 0.500–0.509 | 1 | 1 | 0 | | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Instructing and context: Too few posts (customer love 0.503, n 3) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Direct in-prompt instructions and caps are followed](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | Too few posts | 0.506 | 0.500–0.514 | 2 | 2 | 0 | | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | Too few posts | 0.499 | 0.497–0.500 | 1 | 0 | 1 | | [Persistent project rules files are read and obeyed](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Asks the user versus guessing](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Output degrades as the context window fills](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Context compaction keeps what matters, cheaply and quickly](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Finding the right files in the codebase](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Images, PDFs and file attachments as input](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Doing the work: Too few posts (customer love 0.514, n 17) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Can do the user's kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | Too few posts | 0.507 | 0.499–0.514 | 12 | 10 | 2 | | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | Too few posts | 0.503 | 0.500–0.506 | 3 | 3 | 0 | | [Does unrequested work or over-engineers](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | | [Spins, loops or gets stuck without progress](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | | [Safety filters block legitimate coding tasks](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | Too few posts | 0.503 | 0.500–0.509 | 1 | 1 | 0 | | [Frontend and visual UI output](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Diagnosing and fixing reported bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Breaks existing code or reintroduces bugs](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Stops mid-task or answers instead of acting](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Games checks instead of fixing the problem](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Plan-before-edit mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Caves to or argues with the user's judgement](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Checking and finishing: Too few posts (customer love 0.502, n 2) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Agent-performed code review finds real issues](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | Too few posts | 0.502 | 0.500–0.504 | 2 | 2 | 0 | | [Claims work is done or fixed when it is not](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Builds, tests or runs its own changes](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Interface and sessions: Too few posts (customer love 0.502, n 1) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | Too few posts | 0.503 | 0.500–0.508 | 1 | 1 | 0 | | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Stopping and steering a running agent](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Reliability and speed: Too few posts (customer love 0.500, n 4) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | Too few posts | 0.499 | 0.494–0.503 | 4 | 1 | 3 | | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Account and support: Too few posts (customer love 0.498, n 2) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Data retention, training use and deployment isolation](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | Too few posts | 0.498 | 0.496–0.500 | 2 | 0 | 2 | | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Account bans and access restrictions](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | # Augment Code (Augment) Feedback Bench, coding agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/coding-agents/#/agent/augment | Measure | Value | |---|---| | Rank | 17 of 17 (rank range 17–17) | | Feedback Score | 34.1 (95% interval 33.6–34.6) | | Popularity | 0.000 (share of voice 0.04%) | | Customer love | 0.501 (95% interval 0.497–0.504) | | Place | Popularity 17th of 17 × Customer love 11th of 17 | | Top quadrant | no | | Users | 67 | | Posts counted | 81 | | Posts that judge the agent | 35 | | Criteria ranking high / low | 0 / 0 of 63 | ## Top requests What users ask to add or change, most asked first. 7 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest. No request is asked for in enough user-weeks to show. ## Ranks highest on No criterion where it sits clearly in the top half. ## Ranks lowest on No criterion where it sits clearly in the bottom half. Position on a criterion: its place among the agents with at least 30 rated user-weeks there, ordered by customer love. It ranks high when both its position and its 95% interval put it in the top half, low when both put it in the bottom half; neither otherwise, or when fewer than 4 agents qualify. ## All 63 criteria Criterion love: 0.5 is the category norm. n: rated user-weeks. ### Paying and limits: Too few posts (customer love 0.496, n 10) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [How much use a plan's price buys](https://feedbackbench.com/coding-agents/criteria/limits.plan_value.md) | Too few posts | 0.498 | 0.494–0.502 | 5 | 1 | 4 | | [Single prompt, model or effort level consumes disproportionate quota](https://feedbackbench.com/coding-agents/criteria/limits.burn_rate.md) | Too few posts | 0.495 | 0.491–0.499 | 5 | 0 | 5 | | [Price, allowance or plan terms changed](https://feedbackbench.com/coding-agents/criteria/limits.allowance_change.md) | Too few posts | 0.499 | 0.497–0.500 | 2 | 0 | 2 | | [Usage meter visibility and accuracy](https://feedbackbench.com/coding-agents/criteria/limits.usage_meter.md) | Too few posts | 0.498 | 0.496–0.500 | 2 | 0 | 2 | | [Pricing and plan terms stated clearly and consistently](https://feedbackbench.com/coding-agents/criteria/billing.pricing_clarity.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | | [Free tier and free model availability and limits](https://feedbackbench.com/coding-agents/criteria/billing.free_tier.md) | Too few posts | 0.502 | 0.500–0.506 | 1 | 1 | 0 | | [Short rolling usage window blocks or interrupts work](https://feedbackbench.com/coding-agents/criteria/limits.window_interrupts_work.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Quota reset timing and bonus or banked resets](https://feedbackbench.com/coding-agents/criteria/limits.reset_schedule.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Prompt cache hits, misses and invalidation](https://feedbackbench.com/coding-agents/criteria/limits.prompt_cache.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Pay-as-you-go overage, fallback billing and spend caps](https://feedbackbench.com/coding-agents/criteria/billing.overage_charges.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Using an existing subscription across tools](https://feedbackbench.com/coding-agents/criteria/billing.subscription_portability.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Setting up and connecting: Too few posts (customer love 0.505, n 8) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [IDE and editor integration](https://feedbackbench.com/coding-agents/criteria/setup.ide_integration.md) | Too few posts | 0.504 | 0.500–0.508 | 5 | 4 | 1 | | [Onboarding, discoverability and documentation](https://feedbackbench.com/coding-agents/criteria/setup.onboarding_docs.md) | Too few posts | 0.500 | 0.495–0.508 | 4 | 1 | 3 | | [Install, launch and sign-in](https://feedbackbench.com/coding-agents/criteria/setup.install_signin.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Connecting own API keys, local models and custom endpoints](https://feedbackbench.com/coding-agents/criteria/setup.provider_byok_local.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [MCP servers, plugins, skills and hooks](https://feedbackbench.com/coding-agents/criteria/setup.extensions_mcp.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Choosing models: Too few posts (customer love 0.499, n 1) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Quality got worse or better over time](https://feedbackbench.com/coding-agents/criteria/models.quality_drift.md) | Too few posts | 0.499 | 0.498–0.500 | 1 | 0 | 1 | | [Which models are offered on a plan and when](https://feedbackbench.com/coding-agents/criteria/models.catalog_access.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Automatic model routing and fallback](https://feedbackbench.com/coding-agents/criteria/models.routing_auto.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Reasoning effort setting and its defaults](https://feedbackbench.com/coding-agents/criteria/models.effort_control.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Instructing and context: Too few posts (customer love 0.532, n 15) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Finding the right files in the codebase](https://feedbackbench.com/coding-agents/criteria/context.codebase_retrieval.md) | Too few posts | 0.527 | 0.516–0.539 | 15 | 15 | 0 | | [Persistent project rules files are read and obeyed](https://feedbackbench.com/coding-agents/criteria/context.instruction_files.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Direct in-prompt instructions and caps are followed](https://feedbackbench.com/coding-agents/criteria/context.instruction_following.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Asks the user versus guessing](https://feedbackbench.com/coding-agents/criteria/context.clarifying_questions.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Output degrades as the context window fills](https://feedbackbench.com/coding-agents/criteria/context.long_context_decay.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Context compaction keeps what matters, cheaply and quickly](https://feedbackbench.com/coding-agents/criteria/context.compaction.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Memory and state carried across sessions](https://feedbackbench.com/coding-agents/criteria/context.session_memory.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Images, PDFs and file attachments as input](https://feedbackbench.com/coding-agents/criteria/context.attachments.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Doing the work: Too few posts (customer love 0.497, n 7) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Can do the user's kind of task](https://feedbackbench.com/coding-agents/criteria/work.capability.md) | Too few posts | 0.499 | 0.492–0.505 | 5 | 3 | 2 | | [Does unrequested work or over-engineers](https://feedbackbench.com/coding-agents/criteria/work.scope_overreach.md) | Too few posts | 0.498 | 0.495–0.500 | 2 | 0 | 2 | | [Frontend and visual UI output](https://feedbackbench.com/coding-agents/criteria/work.frontend_ui.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Diagnosing and fixing reported bugs](https://feedbackbench.com/coding-agents/criteria/work.bug_diagnosis.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Breaks existing code or reintroduces bugs](https://feedbackbench.com/coding-agents/criteria/work.regressions_introduced.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Spins, loops or gets stuck without progress](https://feedbackbench.com/coding-agents/criteria/work.stuck_loops.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Stops mid-task or answers instead of acting](https://feedbackbench.com/coding-agents/criteria/work.premature_stop.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Long unattended runs and goal/loop mode](https://feedbackbench.com/coding-agents/criteria/work.long_running_autonomy.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Subagents, parallel agents and orchestrators](https://feedbackbench.com/coding-agents/criteria/work.multi_agent_orchestration.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Games checks instead of fixing the problem](https://feedbackbench.com/coding-agents/criteria/work.reward_hacking.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Risky or irreversible actions without confirmation](https://feedbackbench.com/coding-agents/criteria/work.destructive_actions.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Git commits, branches and sync](https://feedbackbench.com/coding-agents/criteria/work.git_workflow.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Computer use and browser control](https://feedbackbench.com/coding-agents/criteria/work.computer_browser_use.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Safety filters block legitimate coding tasks](https://feedbackbench.com/coding-agents/criteria/work.safety_refusals.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Tool approval prompts and autonomy modes](https://feedbackbench.com/coding-agents/criteria/work.permission_prompts.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Plan-before-edit mode](https://feedbackbench.com/coding-agents/criteria/work.plan_mode.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Length and clarity of replies, summaries and comments](https://feedbackbench.com/coding-agents/criteria/work.response_verbosity.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Caves to or argues with the user's judgement](https://feedbackbench.com/coding-agents/criteria/work.sycophancy_pushback.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Checking and finishing: Too few posts (customer love 0.501, n 1) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Agent-performed code review finds real issues](https://feedbackbench.com/coding-agents/criteria/verify.agent_code_review.md) | Too few posts | 0.501 | 0.500–0.503 | 1 | 1 | 0 | | [Claims work is done or fixed when it is not](https://feedbackbench.com/coding-agents/criteria/verify.false_completion.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Builds, tests or runs its own changes](https://feedbackbench.com/coding-agents/criteria/verify.self_testing.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Reviewing and approving the agent's changes](https://feedbackbench.com/coding-agents/criteria/verify.change_review_ui.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Interface and sessions: Too few posts (customer love 0.500, n 2) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Mobile, remote-control and voice access](https://feedbackbench.com/coding-agents/criteria/surfaces.remote_mobile.md) | Too few posts | 0.501 | 0.500–0.502 | 1 | 1 | 0 | | [Cloud and remote sandbox execution](https://feedbackbench.com/coding-agents/criteria/surfaces.cloud_sessions.md) | Too few posts | 0.499 | 0.496–0.500 | 1 | 0 | 1 | | [How the interface shows work, and what the user can configure](https://feedbackbench.com/coding-agents/criteria/ui.display_settings.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Saving, switching, resuming and rewinding sessions](https://feedbackbench.com/coding-agents/criteria/ui.session_history.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Stopping and steering a running agent](https://feedbackbench.com/coding-agents/criteria/ui.interrupt_steer.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Reliability and speed: Too few posts (customer love 0.504, n 3) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Latency, throughput and fast mode](https://feedbackbench.com/coding-agents/criteria/rel.response_speed.md) | Too few posts | 0.503 | 0.496–0.512 | 2 | 1 | 1 | | [Updates break working setups](https://feedbackbench.com/coding-agents/criteria/rel.update_breakage.md) | Too few posts | 0.499 | 0.497–0.500 | 1 | 0 | 1 | | [Outages, server errors and capacity or rate errors](https://feedbackbench.com/coding-agents/criteria/rel.service_errors.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Client crashes, freezes and failed tool execution](https://feedbackbench.com/coding-agents/criteria/rel.client_failures.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | ### Account and support: Too few posts (customer love 0.497, n 4) | Criterion | Reading | Customer love | 95% interval | n | Praise | Complaint | |---|---|---|---|---|---|---| | [Support, refunds and issue handling](https://feedbackbench.com/coding-agents/criteria/account.support.md) | Too few posts | 0.497 | 0.494–0.499 | 4 | 0 | 4 | | [Wrong charges, failed payments and plan provisioning](https://feedbackbench.com/coding-agents/criteria/account.billing_errors.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Account bans and access restrictions](https://feedbackbench.com/coding-agents/criteria/account.bans_restrictions.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 | | [Data retention, training use and deployment isolation](https://feedbackbench.com/coding-agents/criteria/account.data_privacy.md) | Too few posts | 0.500 | 0.500–0.500 | 0 | 0 | 0 |