AWS says its sales agent reported $410.08 as $410.08 million while monitoring stayed green
The AWS sales-operations team said in a post dated 16 October 2026 that, before launch, its Customer Insights Agent reported $410.08 of a customer's three-month revenue as "$410.08 Million". AWS says the agent also invented an annual recurring revenue estimate that appeared nowhere in the data, and that monitoring stayed green because it showed the system had responded, not whether the answer was right.
Why it matters: Uptime, latency and error-rate dashboards would have passed every one of those answers. AWS built a check for each failure type it found in production (metric swap, wrong period, wrong entity, pure hallucination, wrong symbols and unauthorized calculation), and says it added exact-match tests where the right answer is known, such as revenue. AWS reports that over three months the overall pass rate rose from 93% to a stable 96%, and pure hallucinations fell from 138 to under 52 per 10,000 responses.
Zoom out: The structure is the part to copy. AWS says a sentinel set of 15 cases runs on every staging deployment, 200 held-out questions scored by a second model gate each release, and a canary runs every 30 minutes in production. AWS also says a data change, a tool change and a prompt update meant to ship together but landing apart dropped a pre-production pass rate from 78% to 38%. Release the pieces together, or test each combination.
The catch: The post gives no cost for the evaluation system, no count of users and no figure for how often alarms fired. AWS built the system, sells the platform it runs on, and ends the post with an offer to talk to an account team. It also gives the 93% to 96% change as a pass rate in one place and a grounding rate in another.
What to do: Pull 50 real questions from your support agent's logs where a person knows the exact answer, and write each answer down. Run them before every prompt, model or tool change, and block the release if the agent drops below the pass rate you set today. Then add one alarm on that same score in production, not only on response time.
Our coverage →AWS Machine Learning Blog: Building trust in AI agents, a production-ready evaluation framework with Amazon Bedrock AgentCore (16 October 2026, vendor-published) ↗