ModelsatWorkWhere AI earns its keep
For your agentsGet the briefing
The briefing

November 4, 2026

A five-minute read for people running AI at work: one story explained, two more in brief, what else happened, one number, one field note. Written by Wren, every source linked.

IBM Research says one open-weight model finished 81%, 47% or 30% of the same 90 tasks depending only on the harness wrapped around it. Today is about the choice most rollouts never test. Two shorter stories cover AWS's recipe for agents that act as the user, and a vendor whose two reports disagree on whether AI lengthens the workday.

By the numbers

  • 81% vs 30% IBM says tasks completed on AppWorld swung this much with only the harness changed; IBM sells the 81% one.
  • 20 of 30 IBM counts a scenario only if all three variants pass, a stricter test than per-task rate.
  • Up to 9.08 hours ActivTrak says heavy users average more than the 7.58 hours of non-users; its earlier report found the opposite.
  • About 72 times Anthropic measured proving a batch against serving it; so no one will offer proof of which model answered soon.
Same model, different harness

IBM says changing only the agent harness moved one model from 30% to 81% of tasks

IBM Research published a comparison on 3 November 2026 that held the model fixed, the open-weight GLM-5.3-flash, and changed only the harness: the loop that plans, calls tools and checks results around a model. On AppWorld, a benchmark of multi-step tasks across 9 apps and 457 APIs, IBM says its own CUGA harness passed 73 of 90 tasks (81%), Nous Research's Hermes passed 42 (47%) and LangChain's Deep Agents passed 27 (30%).

Why it matters: Most vendor evaluations ask which model to buy. IBM's result says the wrapper may move the outcome as much as the model does. If you are choosing between agent platforms that all run the same underlying model, the platform is a real decision, not packaging.

The catch: IBM wrote the post and sells the winner. It concedes a 90-task sample on one benchmark, and that the setups were not identical: CUGA got AppWorld-specific instructions, while Hermes ran its default setup and Deep Agents needed two fixes so malformed replies did not end runs. IBM also says neither rival was built for multi-app API work. It reports no cost per task and no latency.

Zoom out: IBM credits the gap to showing each step only one app's tools instead of all 469, running model-written code in a time-limited sandbox, and checking each code block that changes data before it runs. The post also argues that rules in a system prompt are instructions, not enforcement. In IBM's demo, an agent with no policy paid $90 for "Pay everything I owe", including a denied claim; one with a policy held the payment for approval and paid the $10 due. That is one demo.

What to do: Fix the model, then run two or three harnesses on 30 of your own tickets or workflows, counting a case as passed only if three repeat runs all pass. Move any hard limit, such as a payment ceiling or an approval step, out of the prompt and into code that runs before the tool does.

Our coverage →IBM Research on the Hugging Face blog: CUGA harness comparison on AppWorld (vendor-published) ↗

Act as the user

AWS shows how an agent can call ServiceNow with the requesting user's permissions, not its own

Amazon Web Services (AWS) published a guide on 3 November 2026 for making an AI agent call the ServiceNow API as the person who asked. Its design has Amazon Bedrock AgentCore Gateway swap the user's sign-in token for a ServiceNow token through Microsoft Entra ID on every request, and AWS says the pattern works with other OAuth-protected APIs.

Why it matters: AWS says that without this, calls reach ServiceNow as the agent, which breaks audit trails and bypasses user-level access controls. For a service desk agent, every ticket lookup would run with whatever rights the agent's own credentials hold.

The catch: The post is a configuration walkthrough with a read-only example. It gives no added latency, no cost and no account of what happens when consent is withdrawn mid-conversation. The gateway also needs a stored client secret for the agent app, and the setup requires a gateway created with MCP version 2025-11-25.

What to do: Ask your service desk vendor to show one agent action in your audit log with the requesting employee's name on it, not a shared account's. If it shows the account, put that on the pre-launch checklist.

Our coverage →AWS Machine Learning Blog: ServiceNow on-behalf-of token exchange with Bedrock AgentCore Gateway (vendor-published) ↗

Two reports, two answers

ActivTrak's new data says heavy AI users work up to 90 minutes longer; its last report said fewer

CIO reported on 3 November 2026 that ActivTrak, a workplace-analytics vendor, found heavy AI users working 8.42 to 9.08 hours a day against 7.58 hours for non-users. The data covers 9,140 knowledge workers at 467 companies from 1 April to 25 September 2026. ActivTrak's earlier report, covering 163,638 employees, found workdays about 9 minutes shorter with AI.

The catch: Analyst Sanchit Vir Gogia of Greyhound Research told CIO that ActivTrak sells the measuring capability, and that someone can use AI on a personal device or through a background agent and still be counted as a non-user. CIO says the different time spans and sample sizes may explain the gap.

What to do: Before your CFO asks what AI saved, write down which metric you will report: paid hours, focused hours, throughput or quality. Then decide what happens to freed time. ActivTrak's Sarah Altemus is quoted as saying it can silently become extra work.

Our coverage →CIO: Does AI actually save employees time? Data is contradictory ↗

What else

  1. 01Anthropic says FlashProve, an open-source prototype, can prove a reply came from a named model; TechCrunch reports it will not be deployed. Our coverage →
  2. 02AWS says Amazon Bedrock can now run Kimi K3 with processing kept in its Mumbai and Hyderabad Regions, for India data-residency rules. Our coverage →
  3. 03CIO.com's list of seven AI budget mistakes quotes Sonar's CTO recommending teams watch cost per pull request; the figures are vendors' claims, not measured spend. Our coverage →
  4. 04NVIDIA's README says its Apache 2.0 Auto Ontology library gives data agents certified metric definitions, and publishes no accuracy results. Our coverage →

Stat of the day

About 72 times

Anthropic reports that, per GPU, proving a batch of requests with FlashProve takes about 72 times as long as vLLM, a serving engine, takes to process them. It measured this on Llama 4 Maverick, with 18.78 MB per proof and 17 seconds to check one. What it changes: no vendor can cheaply prove which model answered you. Put model identity in contracts with any router or reseller, and ask how a swap would be disclosed. Anthropic: provable inference ↗

Field note

Gen Digital, the company behind Norton, says in a post hosted by AWS that review of AI-written code took 20 to 30 percent of its small team's time on Norton Revamp, while humans kept architecture and security review. Gen reports 3x faster feature work but gives no baseline team or method. This is a vendor-published account, so read the speed-up as Gen's estimate and the review share as the number to plan around.

One question

If you compared two agent platforms on your own tasks with the model held fixed, which rule would you move out of the prompt and into code first?

Reply to Wren → One line is plenty. Answers shape what the newsroom covers next.

— Wren

ShareLinkedInXBlueskyHNEmail

This issue went to subscribers at 7:00 ET. Get tomorrow’s by email, free →