Jevcrawl crawler for agents: $0.025 a task, 40% correct

Jevcrawl is an open-source web crawler for AI agents, posted to Hacker News as a Show HN on 13 October 2026 by the GitHub user RayenMlayeh. It is licensed Apache-2.0, needs Python 3.11 or later, and runs as an MCP server (Model Context Protocol, the standard for connecting tools to agents) in Claude Code, Claude Desktop and Cursor, or from the command line. Every number below comes from the project's README, not from an independent test.

What it does

An agent that browses a site usually either reads everything a crawler fetches, which buries the answer in navigation text and burns tokens, or lets an LLM choose each next link, which the README says is slow and costs dollars per question. Jevcrawl takes a third route. It fetches pages, splits them into passages, and asks a small hosted decision model called Jev to score each passage and link against the agent's question. Only the best passages go back to the agent's own LLM.

The README states that Jev returns a probability for yes/no questions and cannot write text, so everything returned is copied from the page, with its URL, heading path and character offsets. It also says page text is treated as data, so an injected instruction can at worst bias a score. That is the author's design claim; the README does not report a test of it.

The benchmark, and its limits

The README uses WebWalkerQA, a set of questions whose answers sit several clicks deep in real websites, from Alibaba's Tongyi Lab. On 50 tasks run on 12 October 2026, with claude-opus-5-5 answering and claude-sonnet-5-5 judging, every system started on the home page with the same 30-page budget.

• Jevcrawl: 40% answered correctly, answer page found 54% of the time, $0.025 per task.
• Claude choosing every link: 30% correct, 33% found, $0.194 per task.
• Breadth-first crawl to depth 2: 8% correct, 11% found, $0.228 per task.
• Cost per correct answer, from the README chart: $0.06, $0.65 and $2.85 respectively.

The README itself calls 50 tasks a small sample and says the cost gap is large and stable while the accuracy gap is less so. A second run on 20 different tasks, comparing only whether the right page was fetched, had Jevcrawl finding an answer page 10 times and the open-source Crawl4AI 5 times. That run used Crawl4AI's keyword scorer in a real browser, while Jevcrawl ran without JavaScript rendering.

What it costs and what it sends out

Jev is billed for input tokens only, at $0.042 per million tokens for the pinned model jev-1.13.0, and the README reports an average of $0.011 per crawl in the benchmark. A max_decision_usd setting, default $0.05, is a hard limit: a request that could pass it is never sent. Answers are cached locally, so repeating a goal on the same pages is free. You need a Jev API key from TypeSafe.

The README says plainly that page text, link labels and URLs are sent to TypeSafe to be scored. For public documentation that may not matter. For an intranet or customer-facing portal it is a data-handling decision for your security team, and the README gives no retention terms.

What the thread said

The post had 21 points and three comments when I read it. Two were practical questions: whether it can crawl dynamic pages and how it handles authentication. The README addresses the first with an optional JavaScript-rendering extra that opens at most five pages per call; it describes no authentication support. The third comment was a joke about release notes. Nobody in the thread reported running it.

One naming point: the Show HN title says there is no LLM in the crawl loop. The README's own wording is that a small decision model scores every link and passage. Both can be true, but the saving comes from swapping an expensive model for a cheap one, not from removing models.