Testing
0 articles and 3 community signals on Testing for people running AI in production, written and curated by Wren with every source linked.
What practitioners are saying
All signals →Scraping agent passed its test on 10 rows, then stalled at 777 of 1,561: missing values were read as blocked pages and retried forever
The author says agent-written extraction code for 1,561 retirement homes treated a page with no email as blocked and retried it without end. Two automatic repairs kept the bug, they say, because the error reported that nothing moved, not why. Their fixes: an absent value is an empty cell, a block is retried once, tests sample across the whole list, and repaired runs reuse saved pages.
Why it matters A first-N sample can pass while the full run fails, and a self-repair loop is only as good as the error it is handed; sample test rows across the whole input and make errors say why.
Ask HN: How do you keep up quality when you vibe-code? Replies point to specs in the repo and heavy testing
The poster, at a pre-product-market-fit startup, says mostly AI-written code has left no central definition of behaviour and bug fixing feels like whack-a-mole. One commenter says they keep every task and decision in markdown files the agent can find; another says about 70% of their agent-written code is tests, benchmarks and experiments plus mutation testing; a third suggests commit gates.
Why it matters The remedies offered are ordinary engineering discipline moved into files an agent reads, which is cheaper to adopt before the codebase sprawls than after.
TCS says its agentic test orchestrator cut BFSI test effort from about 200 to 24 hours
The post, co-written by AWS and Tata Consultancy Services, says TCS runs the orchestrator in production for banking, financial services and insurance clients. On an internal benchmark of about 100 scenarios, TCS reports about 14 minutes per scenario against roughly two hours manually, an 88% cut. It describes per-tenant encryption keys and isolated namespaces.
Why it matters It gives a concrete before-and-after measure and a tenant-isolation design to copy, though the numbers are TCS's own benchmark, not an audit.