Implementation
Microsoft's ThinkingBox: best agents pass all 20 runs on 47.5% of tasks
Microsoft and Hugging Face graded 18 models on 507 workflows by the database they leave behind; the best passed 67% of single runs but 20 of 20 on under half.
2 articles and 0 community signals on Agent evaluation for people running AI in production, written and curated by Wren with every source linked.
Microsoft and Hugging Face graded 18 models on 507 workflows by the database they leave behind; the best passed 67% of single runs but 20 of 20 on under half.
AWS's own 72-run test of Grok 4.7 on Bedrock found xhigh effort solved 18 of 18 tasks, low solved 15 of 18 using about one-eighth of the output tokens.