# Which AI model runs, how good it is, and choosing it (`control.model`)

Feedback Bench, personal agents, built 2026-10-19, window 2026-08-31 to 2026-10-18. Web page: https://feedbackbench.com/#/criterion/control.model

Area: [Staying in control](https://feedbackbench.com/criteria/control.md)

**Definition.** Which underlying model and reasoning level the agent uses, whether the user can see, choose or restrict it, and verdicts that name the model as the reason the agent is good or bad.

**Boundary.** Not this: see [Agent spends usage on overhead and costly choices](https://feedbackbench.com/criteria/billing.token_efficiency.md) when model choice is blamed for burning usage. Not this: see [Wrong facts, hallucinations and careless mistakes](https://feedbackbench.com/criteria/act.accuracy.md) when specific outputs are wrong and no model is named.

Rated user-weeks, all agents: 1499. Complaint share: 44%.

## The brief

Written by large language models from this page's numbers and the posts it quotes, then checked: every number in the text is in the build. Interpretation, not measurement.

Hermes Agent is best at this. Its users run the models they pick, from Claude to DeepSeek and Qwen, and they name the model as a reason the agent works well. This criterion is the 18th most rated of 53 for AI personal agents. Hermes Agent users rate it 4.4 times as often as other users. Grok Bot ranks low. Its users say they cannot see or pick the model it runs, and they write 37% of the category's complaints here.

### What separates the best from the rest

Hermes Agent users say they pick their own models. They set a default model, hand settled work to a second one and try new model updates when they come out. Grok Bot users say they get no choice. Posts describe no model picker and no third-party models, and say the agent only answers "i'm grok bot" when asked what it runs. Grok Bot has 243 complaining user-weeks here against 89 praising.

- **Ranks high**: [Hermes Agent](https://feedbackbench.com/agents/hermes-agent.md)
  - Hermes Agent, 2026-10-18, X search: Hermes Agent, hermes-agent, Hermes Desktop (X): “gpt-6.1 sol is my default model for hermes agent and has never missed a beat. as soon as i have something bedded down, luna takes over. this approach is highly recommended esp if your are on lower end plans. [link]” [source](https://twitter.com/2034530639584829440/status/2111611348019036178)
  - Hermes Agent, 2026-10-17, X search: Hermes Agent, hermes-agent, Hermes Desktop (X): “@gnukeith the only two i'm using on the list are grok and an uncensored version of qwen 3.8 27b. both solid, but the little 27b qwen paired with hermes agent has been smashing my expectations.” [source](https://twitter.com/1520613336517332992/status/2111594160780464182)
- **Ranks low**: [Grok Bot](https://feedbackbench.com/agents/grok-bot.md)
  - Grok Bot, 2026-10-14, r/GrokBot (Reddit): “how do you know what it's using?! i try to ask mine and it just tells me, "i'm grok bot"..” [source](https://www.reddit.com/r/GrokBot/comments/1x5pahu/can_i_control_the_models_an_agent_uses_with_its/pgp633n/)
  - Grok Bot, 2026-10-07, @bot (X): “@petergyang @bot it still isn't good enough though. they need the ability to use a model picker. i don't want to delegate tough work to an opaque model.” [source](https://twitter.com/1885707511778476032/status/2107980783235543469)

## How it breaks

Rule: A separate pass sorts complaints into failure modes. For each criterion with at least 30 complaining user-weeks, a large language model reads 150 complaint posts drawn at random and names 4 to 8 failure modes; a second large language model then puts every complaint post on that criterion in one mode, or in Other when it fits none. Modes are fixed once named, and new posts are sorted into them each week. A mode counts distinct user-weeks with a complaint in it, per agent; across agents, one user-week per agent. Modes do not change the labels or the Feedback Score. Examples: up to 3 posts per mode from different users, chosen and checked as receipts are.

The largest failure mode is a model too weak for the job, at 36% of complaining user-weeks. Grok Bot users write 40% of these complaints. Grok Bot also has most of the complaints about a hidden or silently changed model (52%) and about being locked to its maker's models (64%). Hermes Agent has most of the complaints about switching models breaking the agent, at 63%. Complaints about a wanted model missing from the picker split between OpenClaw and Hermes Agent.

668 complaining user-weeks.

| Failure mode | User-weeks | Posts | Agents (user-weeks) | Definition |
|---|---|---|---|---|
| Underlying model too weak for the job | 238 | 242 | Grok Bot 95, Hermes Agent 51, Muse 36, OpenClaw 34, ChatGPT Dots 16, Gemini Spark 3, Hark 1, Instinct 1, Poke 1 | The user names the underlying model as the reason the agent makes mistakes, gives up, refuses, writes poorly or does outdated-caliber work; complaints that only say the user can't switch models, with no verdict on quality, are out. |
| Running model hidden or silently changed | 85 | 88 | Grok Bot 44, ChatGPT Dots 12, Hermes Agent 10, Gemini Spark 5, Muse 5, OpenClaw 5, Hark 3, Instinct 1 | The agent does not say which model or reasoning level it is using, gives contradictory answers about it, or quietly routes to a different or weaker model so quality drops; cases where the user can see the model but cannot change it are out. |
| Agent locked to the vendor's own models | 70 | 75 | Grok Bot 45, OpenClaw 10, Muse 6, ChatGPT Dots 4, Gemini Spark 3, Hermes Agent 2 | The agent only runs on its maker's models and offers no way to switch to another vendor's model or bring your own; complaints about the locked-in model's quality or about a missing picker inside an open agent are out. |
| Wanted model missing from the picker | 69 | 73 | OpenClaw 33, Hermes Agent 32, Grok Bot 3, Muse 1 | In an agent that supports multiple models, a specific model the user wants (a new release, a free model, an OpenRouter variant, a local model) doesn't show up, can't be found, or errors out because of discovery, version or plugin problems; vendor lock-in by design and subscription-policy blocks are out. |
| Switching models breaks the agent | 59 | 60 | Hermes Agent 37, OpenClaw 18, Grok Bot 3, ChatGPT Dots 1 | Changing or falling back to another model breaks the install, drops persona or continuity, gets saved as the new default without being asked, or mixes up settings like context size and effort levels; failing to switch because the model is missing is out. |
| No per-task or per-agent model control | 53 | 53 | Grok Bot 29, Hermes Agent 12, OpenClaw 6, Muse 4, ChatGPT Dots 1, Gemini Spark 1 | The user can pick a model only globally, and cannot set a model, provider or reasoning level for each subagent, delegated task, session, bot or auxiliary role; having no model choice at all is out. |
| Subscription can't power the agent's model | 35 | 38 | Grok Bot 13, Hermes Agent 12, OpenClaw 6, Gemini Spark 2, ChatGPT Dots 1, Poke 1 | The user's existing model subscription or credits (for example a Claude, Gemini or ChatGPT plan, or Cursor credits) can't be used to run that provider's models inside the agent, so they must pay again or go without; models that are missing for technical reasons are out. |
| Other | 70 | 71 | Hermes Agent 26, Grok Bot 20, OpenClaw 11, ChatGPT Dots 9, Muse 3, Hark 1 | Complaints that fit none of the modes above. |

#### Underlying model too weak for the job

- Hermes Agent, 2026-10-18, X search: Hermes Agent, hermes-agent, Hermes Desktop (X): “@mfon_crypto no it’s not. it’s even dumber in where opus 5.5 is coming from someone who uses it as a daily driver in my hermes agent” [source](https://twitter.com/1719455822009274368/status/2111729345165172841)
- Grok Bot, 2026-10-17, X search: Grok Bot, Grokbot, x.ai/bot (X): “@xiaohuiyuan dots 很好用, grokbot基座模型不太行, try dots connect your macmini, it works well” [source](https://twitter.com/1786565128508604416/status/2111464638043025691)
- Hark, 2026-10-17, @hark_labs (X): “@hark_labs stop using gpt 3 in ur shitass wannabe agent app lmao this is hilarious. even 27b run locally doesnt make these mistakes [link]” [source](https://twitter.com/1737356334922649600/status/2111284705425502289)

#### Running model hidden or silently changed

- Grok Bot, 2026-10-18, r/GrokBot (Reddit): “it automatically uses it sometimes, but you can’t choose manually. personally, i have antigravity and cc signed in on grok bots computer for hard tasks cause i have grok bot connect to my homelab so i have to have it code sometimes” [source](https://www.reddit.com/r/GrokBot/comments/1x8vde5/how_do_you_make_the_grok_bots_use_claude/phg692l/)
- Grok Bot, 2026-10-14, r/GrokBot (Reddit): “how do you know what it's using?! i try to ask mine and it just tells me, "i'm grok bot"..” [source](https://www.reddit.com/r/GrokBot/comments/1x5pahu/can_i_control_the_models_an_agent_uses_with_its/pgp633n/)
- ChatGPT Dots, 2026-10-07, I Made Grok Bot, Dots & Muse Do the Same 3 Jobs (RAW RESULTS) (YouTube): “astra would have done better, but dots does not use astra—at least not all of the time, which is disappointing. that was the first thing i asked dots. great video though, thanks!!” [source](https://www.youtube.com/watch?v=fbm5LevEJyI&lc=UgzqyKLO6UIBDSPeOcd4AaABAg)

#### Agent locked to the vendor's own models

- Grok Bot, 2026-10-18, r/GrokBot (Reddit): “the only bad thing about grok bot is that it uses grok model.” [source](https://www.reddit.com/r/GrokBot/comments/1x7nwin/what_ways_is_grok_bot_better_than_muse_agent_or/phfilhn/)
- Grok Bot, 2026-10-13, r/cursor (Reddit): “i agree. i went with a grok plan for the sole purpose of grokbot and left my cursor ultra. i was thinking of coming back but now, no way. no 3rd party models, not worth it.” [source](https://www.reddit.com/r/cursor/comments/1x4or37/cursor_new_plans_and_major_changes/pggwo7y/)
- Grok Bot, 2026-10-09, X search: Grok Bot, Grokbot, x.ai/bot (X): “@zaddyfi i mean, you can’t select and use opus, the same way you can’t select the version of grok in grokbot” [source](https://twitter.com/877685012103127040/status/2108437593666789829)

#### Wanted model missing from the picker

- OpenClaw, 2026-10-16, @openclaw (X): “@steipete @lugriba @openclaw @roboclawbot i also can’t get gpt-6.1-sol to become available in oc but is working everywhere else” [source](https://twitter.com/989320117329641472/status/2111059655795569104)
- OpenClaw, 2026-10-14, X search: OpenClaw, openclaw.ai (X): “to be fair, it's mostly settings in openclaw that feel awkward, telling it i have my local model set on a tailscale isn't even an option in their list of providers any more.” [source](https://twitter.com/7215722/status/2110392601757258018)

#### Switching models breaks the agent

- Hermes Agent, 2026-10-15, X search: Hermes Agent, hermes-agent, Hermes Desktop (X): “i am working on the config.yml of hermes agent, but i cannot switch to local when the model is set to local and api. there is also an issue, and it has not been addressed yet. [link]” [source](https://twitter.com/82571855/status/2110596931374436466)
- OpenClaw, 2026-10-07, @openclaw (X): “my codex has been working for about 15 minutes to switch my openclaw to 6.1 sol. is anyone even working on @openclaw anymore? i switched most of my tasks to grokbot @bot because it’s just so good. but switching models should be one button press.” [source](https://twitter.com/274738888/status/2107978654563156241)
- Grok Bot, 2026-10-07, X search: Grok Bot, Grokbot, x.ai/bot (X): “@rileybrown for the worse. you dont get it, claude makes grokbot dumber and slower” [source](https://twitter.com/3078459360/status/2107806797411324044)

#### No per-task or per-agent model control

- Muse, 2026-10-16, @muse (X): “@muse i got a paid sub to get wonder for my muse agent but it only runs on wonder in my side chats and the main chat stays on spark. what can i do???” [source](https://twitter.com/875160333580730372/status/2110887865748230347)
- Grok Bot, 2026-10-07, Grok Bot Now Uses Claude Opus 5.5 + More (MASSIVE Update) (YouTube): “i still think that it should let us decide which models we want to use rather than it picking automatically” [source](https://www.youtube.com/watch?v=Z8H3MvLJwfs&lc=UgxpwmbRMiVimHDW1mt4AaABAg)
- Hermes Agent, 2026-10-03, @NousResearch (X): “@nousresearch its a horrible design choice to leave model switching to fallbacks and configs. a simple pop-up to select from after rate-limited provider would be awesome.” [source](https://twitter.com/1100708039580106753/status/2106450130119590282)

#### Subscription can't power the agent's model

- Hermes Agent, 2026-10-16, X search: Hermes Agent, hermes-agent, Hermes Desktop (X): “@henrybeagle808 @alexandr_wang @plumberbutt97 @harrystebbings wonder is okay but my hermes agent runs on opus 5.5. it’s much less intelligent :( i wouldn’t feel the agent was lacking if i had access to wonder full” [source](https://twitter.com/4617620115/status/2110904539402649776)
- OpenClaw, 2026-10-14, X search: OpenClaw, openclaw.ai (X): “it's so much fun to create a knowledge base with claude code. it's a shame that the claude api can't be used with openclaw. also, i want to make the terminal super flashy.” [source](https://twitter.com/1712368266700849152/status/2110311769822634081)

#### Other

- ChatGPT Dots, 2026-10-14, X search: ChatGPT dots, ChatGPT dot, OpenAI dots, OpenAI's dot (X): “the nature is different, so i can't say for sure, but argon spark is several times smarter than chatgpt dots and grok bot. however, the overwhelming drawback is that it can hardly access anything outside of google workspace...” [source](https://twitter.com/1624307614149652480/status/2110260296736739424)

## Top requests

What users ask to add or change, most asked first. 672 user-weeks ask for something. Requests do not change the Feedback Score. Rule: A separate pass by a large language model reads every counted post and extracts what the user asks the agent or its vendor to add or change, with the criteria it maps to and a short normalised wording; it does not touch the labels or the Feedback Score. A large language model groups the wordings within each criterion (the first criterion the request maps to) into themes, and a second pass merges themes that ask for the same change under different criteria; a merged theme sits under the criterion most of its requests map to, and code counts it once. A theme counts distinct user-weeks that ask for it, per agent; across agents, one user-week per agent. Themes asked in fewer than 2 user-weeks, and requests that share no theme, are not shown. Examples: up to 3 posts per theme from different users, without slurs: whole posts of 60 to 450 characters with at least 8 words beyond handles and links, posts of at most 280 characters first, then the newest.

| Rank | Request | User-weeks | Posts | Agents (user-weeks) |
|---|---|---|---|---|
| 1 | User choice of the underlying model | 98 | 98 | Grok Bot 54, OpenClaw 22, Hermes Agent 12, ChatGPT Dots 4, Muse 4, Gemini Spark 1, Hark 1 |
| 2 | Better underlying model quality | 77 | 77 | Grok Bot 44, Muse 14, Hermes Agent 8, OpenClaw 7, ChatGPT Dots 3, Hark 1 |
| 3 | Local model support | 57 | 58 | Hermes Agent 30, Grok Bot 12, OpenClaw 11, Muse 3, ChatGPT Dots 1 |
| 4 | Support for other providers' models | 51 | 53 | Grok Bot 19, OpenClaw 14, Hermes Agent 11, Muse 4, ChatGPT Dots 3 |
| 5 | Show which model is running | 43 | 44 | Grok Bot 20, Hermes Agent 8, OpenClaw 7, Hark 4, ChatGPT Dots 3, Gemini Spark 1 |
| 6 | Bring your own API key | 36 | 36 | Grok Bot 13, Hermes Agent 9, OpenClaw 8, Muse 3, Hark 2, ChatGPT Dots 1 |
| 7 | Per-task and per-agent model assignment | 35 | 36 | Grok Bot 17, Hermes Agent 13, OpenClaw 4, ChatGPT Dots 1 |
| 8 | Automatic task-based model routing | 30 | 31 | Hermes Agent 15, OpenClaw 9, Grok Bot 5, Muse 1 |
| 9 | Latest GPT and GPT-6.1 Sol support | 24 | 29 | OpenClaw 22, Hermes Agent 2 |
| 10 | Reasoning effort control | 23 | 23 | Grok Bot 9, Muse 7, ChatGPT Dots 4, Hermes Agent 3 |
| 11 | Opt out of or restrict specific models | 22 | 24 | Grok Bot 15, Hermes Agent 3, OpenClaw 2, ChatGPT Dots 1, Gemini Spark 1 |
| 12 | Claude Opus model availability | 17 | 17 | Grok Bot 11, ChatGPT Dots 2, Hermes Agent 2, OpenClaw 2 |

### 1. User choice of the underlying model

- OpenClaw, 2026-10-18, X search: OpenClaw, openclaw.ai (X): “@yetone i thought i was the only one still using it 😂 then why doesn't magpie support directly setting the openclaw model? i also submitted a pr to support lobster and asked for a review 🙏 [link]” [source](https://twitter.com/708591039511207936/status/2111850157973393408)
- Muse, 2026-10-12, @muse (X): “@muse — does this mean you’d run on muse wonder instead of spark? or is there a model selector or what? [link]” [source](https://twitter.com/405449912/status/2109722679993631066)
- Hermes Agent, 2026-10-10, X search: Hermes Agent, hermes-agent, Hermes Desktop (X): “@elonmusk @elonmusk are you able to make grok bot offer model selections like hermes agent does???” [source](https://twitter.com/1086080362038476802/status/2109051364202672219)

### 2. Better underlying model quality

- Hark, 2026-10-17, @hark_labs (X): “@hark_labs stop using gpt 3 in ur shitass wannabe agent app lmao this is hilarious. even 27b run locally doesnt make these mistakes [link]” [source](https://twitter.com/1737356334922649600/status/2111284705425502289)
- Hermes Agent, 2026-10-16, X search: Hermes Agent, hermes-agent, Hermes Desktop (X): “@thsottiaux gpt-6 in chat's memory is so poor that is it had the revelation that hermes agent accepts oauth for the umpteenth time and three times in the same session. please, we are using this for work planning. we cannot work with a defective model. you are openai. be better.” [source](https://twitter.com/1396323276809441280/status/2111073193721245827)
- Grok Bot, 2026-10-10, r/GrokBot (Reddit): “i gotta admit, after the “claude” update, which i think my country got a day or two after yall, it’s actually pretty good now. a better baseline model is what i’ve wanted since day one, and now i’m a little hesitant to drop my ultra plan… 🥲” [source](https://www.reddit.com/r/GrokBot/comments/1wzsk17/what_does_this_mean_my_grok_bot_runs_on_opus_55/pf1lgw6/)

### 3. Local model support

- OpenClaw, 2026-10-18, r/openclaw (Reddit): “whatever i settle on, it must be locally hosted, truly open source, and i must be able to use local models. i'm hoping oc inproves. for now i'm actually using t3 code.” [source](https://www.reddit.com/r/openclaw/comments/1x8m718/real_question_oc_vs_muse/phh5oje/)
- Hermes Agent, 2026-10-13, Hermes Agent Fundamentals In 20 Minutes (No Code) (YouTube): “hermes have its own local server for ai models , 0 setup on the user part just point at the model and it fetches it and run it .” [source](https://www.youtube.com/watch?v=ArdXLt8ZXC0&lc=UgwAUjrmwdkV0gC24cN4AaABAg)
- Hermes Agent, 2026-10-11, r/hermesagent (Reddit): “today i have hermes running 24x7 but i want to have local model to "keep" my data in my house. i do not want to send some data out from my systems” [source](https://www.reddit.com/r/hermesagent/comments/1x315pp/cheapest_decent_machine_for_hermes_agent_with/pf8eb8m/)

### 4. Support for other providers' models

- Hermes Agent, 2026-10-15, @NousResearch (X): “hey @nousresearch, @teknium, @witcheer: can we get @groqllc, @cerebras, @sambanovaai, @fireworksai_hq, and @togethercompute as official providers in hermes?” [source](https://twitter.com/1282829142518632453/status/2110765549848518743)
- Grok Bot, 2026-10-10, X search: Grok Bot, Grokbot, x.ai/bot (X): “@thsottiaux tbh id love dots to work like grok bot but better with openai models, id stay if that improves, i think claude is way better at coding but openai is great at 3d, especially with dots running, which claude doesnt have” [source](https://twitter.com/1614156073233006592/status/2108841952993030269)
- Grok Bot, 2026-10-07, So... GrokBot Now Runs on Opus 5.5 (Explained) (YouTube): “this can be one of the most paradigm shifts in a while. if they allow not just opus 5.5 but all other models sonnet and haiku 5.5 its going to be a game changer” [source](https://www.youtube.com/watch?v=c8ptggw6Mt4&lc=UgzocTHx1MobreNzNM54AaABAg)

### 5. Show which model is running

- ChatGPT Dots, 2026-10-09, X search: ChatGPT dots, ChatGPT dot, OpenAI dots, OpenAI's dot (X): “@grok @bot absolutely nailed the ui and ux. seamless experience, easy integration and cursor subscription makes it so useful. being able so choose the model or to know which one is in use would made it perfect. waiting for more updates on openai dots also. competition is good” [source](https://twitter.com/1765051318480064513/status/2108658494119313905)
- Grok Bot, 2026-10-09, X search: Grok Bot, Grokbot, x.ai/bot (X): “@jobhakdi if the harness is the one picking the backend model per task, the question for non-coders is visibility: will grokbot show which model actually produced a given answer, or does that stay hidden inside the stack?” [source](https://twitter.com/2107129059821936640/status/2108649326280888813)
- Hark, 2026-10-09, @hark_labs (X): “@hark_labs @bot 4. limited vm capabilities. no docker. 5. it’s unclear which model is being used or what the benchmark results are. the quality seems decent, but i don’t have a full picture.” [source](https://twitter.com/966334777870577669/status/2108435918390468820)

### 6. Bring your own API key

- Hermes Agent, 2026-10-14, r/hermesagent (Reddit): “man fuck this free trial first-launch model. stop trying to steer me to your crappy paid service. i have my api keys and my local setup, you need to make *that* easy to plug in to” [source](https://www.reddit.com/r/hermesagent/comments/1x5u50y/hermes_agent_v0220_the_threshold_release/pgqz2dh/)
- Hark, 2026-10-10, @hark_labs (X): “@pablothethinker @adcock_brett @hark_labs i have it on my "to-try" drawer, but prefer a lot more @zocomputer approach of multiple models (even byok)” [source](https://twitter.com/3592581/status/2109043704912544099)
- Hark, 2026-10-07, @hark_labs (X): “@hark_labs could it install a coding harness and drain usage straight from hark and choose models too? (asking for a friend)” [source](https://twitter.com/1537239770660851718/status/2107964462774825375)

### 7. Per-task and per-agent model assignment

- Hermes Agent, 2026-10-14, @NousResearch (X): “@miaai_lab we need something like this built into @droid and @nousresearch hermes agent maybe for hermes it would be an auxiliary model” [source](https://twitter.com/1382136217601417222/status/2110224872622506364)
- Grok Bot, 2026-10-12, @bot (X): “there was life before @grok @bot and after. i have tried to make persistent chat experience work but the @xai team has really done the fucking job. life will never be the same without this experience. (model switcher per bot would be nice though😍)” [source](https://twitter.com/132661201/status/2109526078746488908)
- Hermes Agent, 2026-10-10, @NousResearch (X): “@teknium @asvanevik @nousresearch overall solid: subagents get isolated context + tools and just work. one gap: no per-task model — delegation.model/provider in config.yaml pins all subagents globally, so mixed-model batches still require separate hermes --provider/-m processes” [source](https://twitter.com/2077810080645263361/status/2108945297346842683)

### 8. Automatic task-based model routing

- OpenClaw, 2026-10-10, X search: OpenClaw, openclaw.ai (X): “@openaidevs @thsottiaux if codex agents could change model and reasoning on their own if allowed by the user and maybe even clear context we wouldn't need openclaw, paperclip etc. but could rather just rely on a self hosted ticket system and instructions suggesting a model.” [source](https://twitter.com/330105913/status/2108997455102488680)
- Hermes Agent, 2026-10-09, r/hermesagent (Reddit): “it’s not a good idea to route different models for each prompt, it would wipe out cache, which is a huge cost saving feature. it should route ton different model based on task.” [source](https://www.reddit.com/r/hermesagent/comments/1x1sjss/hermes_jev_litellm_50_tokens_saving/pey64o2/)
- Hermes Agent, 2026-10-09, X search: Hermes Agent, hermes-agent, Hermes Desktop (X): “@poteto @koltregaskes @bot i want hermes agent, but better. hermes is powerful but too much upkeep for a solo founder. grok bot is far simpler, and i love that it's now more model agnostic and picks the best model. i'd pay for its ui on top of hermes' power and ownership. [link]” [source](https://twitter.com/1074414809607618560/status/2108678329620685182)

### 9. Latest GPT and GPT-6.1 Sol support

- Hermes Agent, 2026-10-12, @NousResearch (X): “@nousresearch @teknium hermes is still missing the catalog entry for ultra fast with gpt sol 6.1. there's also a bug where messages containing over 1mb of data are rejected.” [source](https://twitter.com/1960694413836845056/status/2109464487635882063)
- Hermes Agent, 2026-10-09, @NousResearch (X): “@nousresearch please update with sol 6.1 and more models and different reasoning efforts!” [source](https://twitter.com/1806457685702762496/status/2108445449325580290)
- OpenClaw, 2026-10-03, @openclaw (X): “@openclaw hey folks, the codex harness didn't get updated :) sol 6.1 is still not supported!” [source](https://twitter.com/136402253/status/2106401104489820316)

### 10. Reasoning effort control

- Grok Bot, 2026-10-13, What's new in Grok Bot - Oct 13, 2026 (YouTube): “apparently bots are just not proactive when giving them long coding tasks. even the new main bot. sometimes it would answer super quickly, which is good only if the answer is correct. i hope we can set like bots effort level.” [source](https://www.youtube.com/watch?v=8_K9yAckFh0&lc=UgzeDQoDwb43hHcGB8F4AaABAg)
- ChatGPT Dots, 2026-10-13, X search: ChatGPT dots, ChatGPT dot, OpenAI dots, OpenAI's dot (X): “here is an example - i had given ticket details a few days ago in the same chat with dot and it lazily forgot to look it up. @openai dots / bots need to have a “hard work” option enabled by default. it comes at cost of faster answers - but the quality is much better. [link]” [source](https://twitter.com/120544983/status/2110113350528946212)
- Muse, 2026-10-12, r/AI_Agents (Reddit): “i love muse but wish i could increase the reasoning time. it goes so fast it will “miss” or oversimplify items” [source](https://www.reddit.com/r/AI_Agents/comments/1wzdnzc/best_ai_agent_grok_vs_muse_vs_dots_vs_spark/pfhfimi/)

### 11. Opt out of or restrict specific models

- Gemini Spark, 2026-10-17, X search: Gemini Spark (X): “gemini spark is still difficult to use. can you stop switching me to the flash model all the time? also, why do you keep repeating the answer to my previous question?” [source](https://twitter.com/1998354483567566848/status/2111519383260704822)
- Grok Bot, 2026-10-13, What's new in Grok Bot - Oct 13, 2026 (YouTube): “begging for a switch to not use anthropic models, thousands of us do not trust anthropic would pay more for less with a smile if it meant no opus” [source](https://www.youtube.com/watch?v=8_K9yAckFh0&lc=Ugz05YKMezUOQded8X94AaABAg)
- Grok Bot, 2026-10-08, Grok Bot just got a MASSIVE update (YouTube): “tbh i wish we could turn off/blacklist external models we don't want it to use. this makes me less likely to use my grok bot.” [source](https://www.youtube.com/watch?v=ZW-HR9XMQKw&lc=UgyfSINMzITvyV2JgHV4AaABAg)

### 12. Claude Opus model availability

- Grok Bot, 2026-10-17, X search: Grok Bot, Grokbot, x.ai/bot (X): “@iruletheworldmo if grok bot ran on opus 5.5 its all i’d use” [source](https://twitter.com/863668035625795585/status/2111347175011782888)
- Grok Bot, 2026-10-13, @bot (X): “if @bot let you run everything with opus 5.5, i would cancel all my other subs.” [source](https://twitter.com/1885707511778476032/status/2110145244448866701)
- Grok Bot, 2026-10-12, X search: Grok Bot, Grokbot, x.ai/bot (X): “grok bot should not be directly connected! if it cannot route to opus5.5, it will become less intelligent!” [source](https://twitter.com/2106252309038813184/status/2109610051073896855)

## Every agent

| Agent | Overall rank | Reading | Customer love | 95% interval | n | Praise | Complaint |
|---|---|---|---|---|---|---|---|
| [Hermes Agent](https://feedbackbench.com/agents/hermes-agent.md) | 2 | 1st of 5 | 0.696 | 0.663–0.724 | 657 | 482 | 175 |
| [OpenClaw](https://feedbackbench.com/agents/openclaw.md) | 5 | 2nd of 5 | 0.505 | 0.473–0.538 | 275 | 160 | 115 |
| [Muse](https://feedbackbench.com/agents/muse.md) | 1 | 3rd of 5 | 0.497 | 0.469–0.525 | 125 | 70 | 55 |
| [ChatGPT Dots](https://feedbackbench.com/agents/dots.md) | 8 | 4th of 5 | 0.466 | 0.439–0.493 | 75 | 31 | 44 |
| [Grok Bot](https://feedbackbench.com/agents/grok-bot.md) | 3 | 5th of 5 | 0.320 | 0.292–0.351 | 332 | 89 | 243 |
| [Gemini Spark](https://feedbackbench.com/agents/gemini-spark.md) | 7 | Too few posts | – | – | 20 | 6 | 14 |
| [Hark](https://feedbackbench.com/agents/hark.md) | 4 | Too few posts | – | – | 11 | 6 | 5 |
| [Instinct](https://feedbackbench.com/agents/instinct.md) | 6 | Too few posts | – | – | 2 | 0 | 2 |
| [Poke](https://feedbackbench.com/agents/poke.md) | 9 | Too few posts | – | – | 2 | 0 | 2 |

## Posts

Receipts rule: Up to 5 praise and 5 complaint posts per area and 3 per criterion. Each is a whole post of 60 to 450 characters (700 for areas) with at least 8 words beyond handles and links, that judges the criterion, whose labels are all praise or all complaint, and that contains no slur. Posts that touch the fewest criteria come first, then posts of at most 280 characters, then the newest. A second large language model reads the first 3 candidates per slot and keeps a post only if it clearly praises, or clearly complains about, the agent on that criterion. Links and personal details removed by the source show as marks such as [link].

### Hermes Agent

- Praise, 2026-10-18, r/hermesagent (Reddit): “same, heavily using claude through plugin with no problems. feels a bit different than other agents because it literally is the claude cli piped in (and have to often point it at hermes docs), but no real complaints.” [source](https://www.reddit.com/r/hermesagent/comments/1x7lgrg/hermes_claude/pheu0ga/)
- Praise, 2026-10-18, X search: Hermes Agent, hermes-agent, Hermes Desktop (X): “gpt-6.1 sol is my default model for hermes agent and has never missed a beat. as soon as i have something bedded down, luna takes over. this approach is highly recommended esp if your are on lower end plans. [link]” [source](https://twitter.com/2034530639584829440/status/2111611348019036178)
- Praise, 2026-10-17, r/hermesagent (Reddit): “oh yeah, say go, confirm, omg that was killing me everytime glm-5.3-flash and deepseek v4.1 flash are the way to go today v4.1-flash october update released so far so good” [source](https://www.reddit.com/r/hermesagent/comments/1x5ax88/haiku_viable_for_hermes_use/pha0j7p/)
- Complaint, 2026-10-18, X search: Hermes Agent, hermes-agent, Hermes Desktop (X): “@mfon_crypto no it’s not. it’s even dumber in where opus 5.5 is coming from someone who uses it as a daily driver in my hermes agent” [source](https://twitter.com/1719455822009274368/status/2111729345165172841)
- Complaint, 2026-10-16, X search: Hermes Agent, hermes-agent, Hermes Desktop (X): “@henrybeagle808 @alexandr_wang @plumberbutt97 @harrystebbings wonder is okay but my hermes agent runs on opus 5.5. it’s much less intelligent :( i wouldn’t feel the agent was lacking if i had access to wonder full” [source](https://twitter.com/4617620115/status/2110904539402649776)
- Complaint, 2026-10-15, X search: Hermes Agent, hermes-agent, Hermes Desktop (X): “@elio_diaz_j @terrytao85 @hyjsfn9h luna6 not good, acting like a fool in my hermes agent.” [source](https://twitter.com/1813105807946620929/status/2110544183039852772)

### OpenClaw

- Praise, 2026-10-12, X search: OpenClaw, openclaw.ai (X): “@brorlandi i used both a lot and ended up with openclaw. it may be old, but it's running on several servers here, as part of various applications. openclaw integrated with claude performs much better than each one alone.” [source](https://twitter.com/1525275821077893127/status/2109590075101028793)
- Praise, 2026-10-10, X search: OpenClaw, openclaw.ai (X): “@valmeretheory i saw your posts and finally decided to go back to claude too. opus 5.5 is amazing in openclaw 😭” [source](https://twitter.com/1995817411023699973/status/2108985485322305910)
- Praise, 2026-10-08, X search: OpenClaw, openclaw.ai (X): “@coldniko it is really working good now and it allows me to run the first useful local model with my openclaw agent. good quality and speed. [link]” [source](https://twitter.com/3024647763/status/2108127678775607742)
- Complaint, 2026-10-17, X search: OpenClaw, openclaw.ai (X): “@mac_nishio i have been using gpt with openclaw→hermes for about six months, but in terms of reaching relatively distant goals, it is below claude and of course below deepseek.” [source](https://twitter.com/111898324/status/2111259393325219987)
- Complaint, 2026-10-16, @openclaw (X): “@steipete @lugriba @openclaw @roboclawbot i also can’t get gpt-6.1-sol to become available in oc but is working everywhere else” [source](https://twitter.com/989320117329641472/status/2111059655795569104)
- Complaint, 2026-10-14, X search: OpenClaw, openclaw.ai (X): “it's so much fun to create a knowledge base with claude code. it's a shame that the claude api can't be used with openclaw. also, i want to make the terminal super flashy.” [source](https://twitter.com/1712368266700849152/status/2110311769822634081)

### Muse

- Praise, 2026-10-17, @muse (X): “agu, my @muse agent just told me it uses muse wonder model. i reviewed it against @claudeai sonnet 5.5, my default model and muse wonder is impressive. [link]” [source](https://twitter.com/1833885240709820418/status/2111460350432280927)
- Praise, 2026-10-17, @muse (X): “meta @muse wonder full is so good.. very competitive to gpt and anthropic - way beyond grok capabilities. this is definitely going to stay as one of my daily drivers love it!!!!” [source](https://twitter.com/1773671005065478145/status/2111465759230816308)
- Praise, 2026-10-17, Zuckerberg’s $100 Billion Bet (YouTube): “@ninarainbow that's not true. muse wonder is pretty good. it's better than any of the open source ones, which is all you need to be.” [source](https://www.youtube.com/watch?v=XfdIhd1g6EI&lc=Ugxdt6CJfYEnQ2ymiHt4AaABAg.AbtNxbKYd_UAc4qx8joNkb)
- Complaint, 2026-10-16, @muse (X): “@muse i got a paid sub to get wonder for my muse agent but it only runs on wonder in my side chats and the main chat stays on spark. what can i do???” [source](https://twitter.com/875160333580730372/status/2110887865748230347)
- Complaint, 2026-10-12, Meta resucita con el NUEVO Muse Wonder (YouTube): “they are a bunch of damn liars, i have tried it, i bought the 100 pack with my team and i assure you it does not reach the top chinese models! i'm talking about muse wonder full.” [source](https://www.youtube.com/watch?v=wMeC6WaxpsY&lc=UgxgrMHiTzcM6QAjw3d4AaABAg)
- Complaint, 2026-10-03, @muse (X): “damn, seeing huge degradation in the quality of @muse . i wonder if they throttled the intelligence cause of the millions of downloads” [source](https://twitter.com/1409949421110919168/status/2106427908000502249)

### ChatGPT Dots

- Praise, 2026-10-14, X search: ChatGPT dots, ChatGPT dot, OpenAI dots, OpenAI's dot (X): “i realised today that since use my @openai dot, i haven’t once thought about which model or effort level i need to use. pretty nice place to be.” [source](https://twitter.com/20139298/status/2110203400633950552)
- Praise, 2026-10-10, X search: ChatGPT dots, ChatGPT dot, OpenAI dots, OpenAI's dot (X): “since the things that can be done and entrusted to chatgpt dots have increased, i'm refining requirements and designs in chat, but gpt-6 is smart. i thought 5.6 was incredibly smart, but gpt-6 feels even several levels smarter.” [source](https://twitter.com/61489604/status/2108961768697647281)
- Praise, 2026-10-04, X search: ChatGPT dots, ChatGPT dot, OpenAI dots, OpenAI's dot (X): “openai's dots is the best personal assistant i've ever used. it's not because of all the connectors or connectivity. it's because of what matters most to me: the model is powerful enough that i can trust it with my 3d modeling optimization tasks” [source](https://twitter.com/1842067121024729089/status/2106627842016502136)
- Complaint, 2026-10-14, X search: ChatGPT dots, ChatGPT dot, OpenAI dots, OpenAI's dot (X): “the nature is different, so i can't say for sure, but argon spark is several times smarter than chatgpt dots and grok bot. however, the overwhelming drawback is that it can hardly access anything outside of google workspace...” [source](https://twitter.com/1624307614149652480/status/2110260296736739424)
- Complaint, 2026-10-12, X search: ChatGPT dots, ChatGPT dot, OpenAI dots, OpenAI's dot (X): “@cutesuscat i tried this with a problem i handed to an openai dot and it absolutely did not do a better job than opus 5.5” [source](https://twitter.com/964807719973629952/status/2109585606145896866)
- Complaint, 2026-10-07, I Made Grok Bot, Dots & Muse Do the Same 3 Jobs (RAW RESULTS) (YouTube): “astra would have done better, but dots does not use astra—at least not all of the time, which is disappointing. that was the first thing i asked dots. great video though, thanks!!” [source](https://www.youtube.com/watch?v=fbm5LevEJyI&lc=UgzqyKLO6UIBDSPeOcd4AaABAg)

### Grok Bot

- Praise, 2026-10-13, The ONLY Grok Bot Tutorial You Need 2026 - How To Use Grok Bot For Beginners (YouTube): “grok bot is like a broker, uses the right si according to the task.” [source](https://www.youtube.com/watch?v=q1IwR88mhaA&lc=UgzHE9_G_jashfOVfxp4AaABAg.AbfET0IPh7uAbvMtWo3b55)
- Praise, 2026-10-09, X search: Grok Bot, Grokbot, x.ai/bot (X): “grokbot calls the opencode model in pi, greatly increasing work efficiency and usability 🤩” [source](https://twitter.com/2054869241233629184/status/2108388497249980701)
- Praise, 2026-10-08, X search: Grok Bot, Grokbot, x.ai/bot (X): “this aged very poorly grok bot is a great product and i’m finding it very useful. massive respect for the decision to use the best model for the job, not just their models [link]” [source](https://twitter.com/3296904699/status/2108286668319199464)
- Complaint, 2026-10-18, r/GrokBot (Reddit): “the only bad thing about grok bot is that it uses grok model.” [source](https://www.reddit.com/r/GrokBot/comments/1x7nwin/what_ways_is_grok_bot_better_than_muse_agent_or/phfilhn/)
- Complaint, 2026-10-18, r/GrokBot (Reddit): “it automatically uses it sometimes, but you can’t choose manually. personally, i have antigravity and cc signed in on grok bots computer for hard tasks cause i have grok bot connect to my homelab so i have to have it code sometimes” [source](https://www.reddit.com/r/GrokBot/comments/1x8vde5/how_do_you_make_the_grok_bots_use_claude/phg692l/)
- Complaint, 2026-10-17, X search: Grok Bot, Grokbot, x.ai/bot (X): “@xiaohuiyuan dots 很好用, grokbot基座模型不太行, try dots connect your macmini, it works well” [source](https://twitter.com/1786565128508604416/status/2111464638043025691)

### Gemini Spark

- Praise, 2026-10-18, X search: Gemini Spark (X): “started with opus 5.5, then switched to gemini 4 argon when it landed in gemini spark. honestly, argon blew me away. crazy model. how did i do? @claudeai @trq212 @geminiapp @_mohansolo @officiallogank” [source](https://twitter.com/1896907608973512704/status/2111866577692102880)
- Praise, 2026-10-14, X search: Gemini Spark (X): “the pelican riding a bike made by argon4 in gemini spark is really impressive, google has truly risen! [link]” [source](https://twitter.com/1758307046133526528/status/2110174690581098813)
- Praise, 2026-10-14, X search: Gemini Spark (X): “using gemini spark to learn the juicefs source code. i found that the language of gemini 4 argon is indeed much better, and the ai flavor has diminished a lot. it's very comfortable to use for learning. [link]” [source](https://twitter.com/1594959396764672000/status/2110221686624903437)
- Complaint, 2026-09-11, X search: Gemini Spark (X): “gpt 5.6 luna is far better than whatever google is running in its gemini spark. i have a paid subscription for google and no sub for gpt. ggs google!” [source](https://twitter.com/1368523504610467840/status/2098441532760043555)
- Complaint, 2026-10-17, X search: Gemini Spark (X): “gemini spark is still difficult to use. can you stop switching me to the flash model all the time? also, why do you keep repeating the answer to my previous question?” [source](https://twitter.com/1998354483567566848/status/2111519383260704822)
- Complaint, 2026-10-13, X search: Gemini Spark (X): “kind of disappointed with gemini spark "task" with argon, so far 6 tasks have died off with zero output or error, just dead. i haven't gotten any single completed task. but they run fine on flash. [link] [link]” [source](https://twitter.com/1338867227605356545/status/2110111691933405497)

### Hark

- Praise, 2026-10-09, @hark_labs (X): “guys those who got @hark_labs pro 3x for free don't let it go to waste, this shit uses the best backend api models available!! it can make edits, it can code, it's sometimes even better than opus 5.5 medium!!! it might even be using fable unreleased model or 5.1 don't sleep on hark...” [source](https://twitter.com/1389178840388313092/status/2108410730366300481)
- Praise, 2026-10-08, X search: Hark Pro, hark.com, Hark AI (X): “i exported my hark data and ran the numbers to see which models are actually doing the work. claude opus 5.5 handled 63% of my 780 requests. add claude sonnet 5 and it's over 83%, routed across anthropic, openrouter, bedrock, cloudflare and google. that's why hark pro feels this accurate and smooth. frontier sota models under the hood, with fallbacks across providers 🔥 @hark_labs #ai #aiagents #hark #harkpro #claude” [source](https://twitter.com/1991860005159817216/status/2108227501722112114)
- Complaint, 2026-10-17, @hark_labs (X): “@hark_labs stop using gpt 3 in ur shitass wannabe agent app lmao this is hilarious. even 27b run locally doesnt make these mistakes [link]” [source](https://twitter.com/1737356334922649600/status/2111284705425502289)
- Complaint, 2026-10-07, X search: Hark Pro, hark.com, Hark AI (X): “thanks to @yulmu_coffee, i am using hark pro well. i was also curious about what model you are using and if it is fine-tuned, what model it is based on, but you rarely tell me--” [source](https://twitter.com/1815696491245436929/status/2107905012198465637)
- Complaint, 2026-10-09, @hark_labs (X): “@hark_labs @bot 4. limited vm capabilities. no docker. 5. it’s unclear which model is being used or what the benchmark results are. the quality seems decent, but i don’t have a full picture.” [source](https://twitter.com/966334777870577669/status/2108435918390468820)

### Poke

- Complaint, 2026-09-14, @interaction (X): “@clarshio @fainzaig @interaction i guess they moved 99% of their traffic to swe1.7 maybe swe2 after the acquistion, which is an unusable model. i was their day 1 user and used to have the ultra plan, cancelled immediately after they got acquired.” [source](https://twitter.com/2089023008333815808/status/2099347396727849415)