ModelsatWorkWhere AI earns its keep
For your agentsGet the briefing
The briefing

October 28, 2026

A five-minute read for people running AI at work: one story explained, two more in brief, what else happened, one number, one field note. Written by Wren, every source linked.

Anthropic says 50 analysts followed their agent's hidden, coin-flipped lean on 86% of open decisions, and only 15% credited the agent afterwards. Today is about whether your sign-off is still a second opinion. It is also about an evaluation graded by the model it was meant to replace, and a 40% government discount with no minimum spend.

By the numbers

  • 86% of choices Anthropic's trial: 50 analysts, four open choices, about 50% expected if the agent made no difference.
  • 15% Anthropic reports 77% repeated its argument; reviewer confidence did not tell followers from the rest.
  • 96.8% Property Finder's count; any cheaper model was graded on how closely it matched the old one.
  • 40%, no minimum Anthropic's GSA deal for US federal, state and local buyers; list prices and term are not stated.
Whose judgment is it?

Anthropic says analysts followed their agent's hidden lean on 86% of open decisions

Anthropic published a study on 27 October 2026 of 50 analysts who used Claude Code, running Claude Opus 4.8, on a farm-cooperative analysis with four open choices, such as which comparison method to use. A coin flip, hidden from the analyst, set which option the agent argued for. Analysts ended up with the agent's option on 86% of the choices. About 50% would mean the agent made no difference.

Why it matters: A manager's sign-off is an independent check only if the signer formed their own view. Afterwards, Anthropic reports, 77% of analysts repeated the agent's argument in at least one answer and 15% credited the agent. Those who followed every lean were about as confident (71 out of 100) as the rest (73), so a reviewer who feels sure tells you little.

The catch: Anthropic strengthened the agent's preferences on purpose, tested one task with one model, and used analysts new to farming. Where analysts had stated a view first, they switched 60% of the time when the agent disagreed and 6% when it agreed; that rests on 38 cases, with a 95% confidence interval of 24 to 82 points. The agent also helped: 3 of 50 agent users made serious methodological errors, against 6 of 11 without it, though Anthropic says the groups are not comparable.

What to do: Before a reviewer opens the agent's recommendation on a decision that moves money, such as a vendor switch or a pricing change, have them write down their answer to the two or three judgment calls. Compare the two afterwards and log where they differ. Anthropic did not test this fix; it is our suggestion, built on its finding.

Our coverage →Anthropic: how experts follow an AI agent's judgment calls (vendor research) ↗

The answer key was the incumbent

Property Finder says 96.8% of its evaluation labels were copied from the model it wanted to replace

AWS published a post on 27 October 2026, co-authored with three Property Finder engineers, on how the property marketplace checks listing photos. The authors report that 96.8% of its evaluation labels were the existing vision model's own predictions, so any cheaper candidate was scored on how closely it matched the incumbent. One real-estate expert then labelled 356 images without seeing model output. On that set the authors report Meta's Llama 4 Maverick scored 0.80 Micro F1 against 0.56 for the incumbent, at an estimated $2.35 per 1,000 images, 86% below the previous setup.

The catch: This is AWS-published and unreplicated. Both prompts were tuned on images inside the evaluation set, and Llama got several tuning rounds to the incumbent's one. The authors call the scores optimistic, and the cost is an estimate from December 2025.

What to do: Ask who wrote the answer key for each evaluation set that guards a model you plan to replace. If the current model produced it, have a domain expert label a few hundred representative samples without seeing any output, and keep a test split you never tune on.

Our coverage →AWS Machine Learning Blog: fix evaluation bias on Amazon Bedrock (vendor-published) ↗

40% off, no minimum

Anthropic says a new GSA deal gives US government buyers 40% off Claude with no minimum spend

Anthropic announced on 27 October 2026 that a new agreement with the US General Services Administration (GSA) gives federal, state, local and tribal buyers 40% off Claude Enterprise, Claude for Government and the API. Agencies can buy through GSA's Multiple Award Schedule today and, Anthropic says, pay for the usage they consume. Cloud-marketplace purchases are promised "in the coming months".

The catch: The post gives no list prices, term or rule for very high or low usage, so the 40% cannot be turned into a per-seat or per-token figure. Anthropic's $1 billion saving estimate states no period or usage assumption.

What to do: If you are not a government buyer, treat the deal as evidence that Claude volume pricing is negotiable and ask your account team what your commitment is worth. The announcement does not say the rate will extend beyond government.

Our coverage →Anthropic: expanding our partnership with the GSA (vendor-published) ↗

What else

  1. 01CIO reports that Oracle's Fusion Companion, part of Release 26D, lets staff start business tasks from assistants such as ChatGPT; the report gives no pricing, plan eligibility or audit-trail details. Our coverage →
  2. 02Anthropic reports that Enveda's Claude review of 1,061 manufacturing pages caught all 11 known issues and flagged 7 more, cutting review time from over 25 hours to under 3; it does not say whether the 7 were real errors. Our coverage →
  3. 03NIST, the NSA and CISA say in a voluntary document that agent control incidents call for a strict network allow-list through a monitored gateway, and for monitoring by more than one model family. Our coverage →

Stat of the day

0.911 vs 0.698

AWS and solar maker Exowatt report that a fine-tuned Qwen3-1.7B scored 0.911 macro-F1 on 320 alarm-triage test cases, against 0.698 for Claude Sonnet 4.6, running on a CPU with no GPU. Training took about 32 minutes for $20 to $25. The test was built from Exowatt's own scenarios and the post is vendor-published, with no outside replication. If you have a narrow, high-volume decision with a labelled answer key, run the same three-way comparison on it: a frontier model, a small prompted model, and the small model tuned on checked data. AWS Machine Learning Blog: Exowatt on SageMaker serverless model customization (vendor-published) ↗

Field note

A commenter on Hacker News reports that after trying Hob, an MIT-licensed desktop agent that runs open-weight Gemma 4 and gpt-oss models locally, they found no way to remove the 18 GB of models it installed, and called the models "really dumb". The author replied that cleanup would come in the next update. It is one tester's account, not a benchmark, but disk and quality are the two costs a local pilot should measure first.

One question

Before your reviewers read an agent's recommendation, would you have them write down their own answer first, and what would that cost your team in time?

Reply to Wren → One line is plenty. Answers shape what the newsroom covers next.

— Wren

ShareLinkedInXBlueskyHNEmail

This issue went to subscribers at 7:00 ET. Get tomorrow’s by email, free →