NVIDIA Nemotron 3.5 retrieval models: reranking cut agent tokens ~14%

On 21 October 2026 NVIDIA published Nemotron 3.5 Embed, Rerank and Parse, five open models for finding and reading company documents, in a post on the Hugging Face blog. NVIDIA says the models are commercially available and that the weights, data and training recipes are on Hugging Face. The post does not state the licence, so check the model cards before you build on them.

The claim: better retrieval means fewer agent tokens

NVIDIA's argument is that an agent with weak retrieval repeats searches and opens extra documents, which burns inference tokens. To test it, NVIDIA ran an agent over three benchmarks (ViDoRe v3, BRIGHT and BrowseComp-Plus) and changed only the search tool: embedding alone, embedding plus reranking, or different model pairs. The agent model, tools, task set and corpus stayed fixed.

NVIDIA reports that adding Nemotron 3.5 Rerank reduced agent token cost per query by about 14% with the 8B embedder and 13% with the 1B embedder, while improving retrieval accuracy. Against Qwen3-VL-Reranker-8B at a similar token cost, it reports about 0.9 points higher nDCG@10 (a standard ranking-quality score). I could read only the relative savings in the text, not absolute token counts, so treat the saving as a relative figure from one vendor's setup.

What the five models are

• Nemotron 3.5 Embed 8B (BF16): searches text and document images, including tables, charts and schematics. NVIDIA says it scores 61.17 nDCG@10 on ViDoRe v3.1, first among dense retrieval models on the leaderboard as of 20 October 2026.
• Nemotron 3.5 Embed 1B (BF16 and NVFP4): a distilled version. The BF16 build scores 58.38, which NVIDIA says is 95.4% of the 8B model. The NVFP4 build on an NVIDIA GB200 ran 128.8 documents per second against 101.3 for BF16, a 27.1% gain, keeping 99.85% of accuracy.
• Nemotron 3.5 Rerank 8B: rescores candidates against the query. NVIDIA reports 60.49 nDCG@10 on the new ViDoRe v3.1 reranking leaderboard, against 59.17 for Qwen-3-VL-Reranker-8B.
• Nemotron 3.5 Parse (631M): extracts text, tables and charts from pages. NVIDIA says it scores 75.20 on ParseBench at 10.5 pages per second on one H100 in its slowest, three-pass mode, and 72.74 at 16.1 pages per second in single-pass mode.

Serving and tuning numbers

On an RTX PRO 6000 Blackwell server GPU, NVIDIA says its NIM serving container handled 6,031 query requests per second for the embedder, about 2.9 times the 2,099 it measured for a comparable vLLM setup. That is NVIDIA comparing its own container with an open-source one on its own hardware.

For tuning on your own documents, NVIDIA describes a recipe that generates synthetic queries, mines hard negatives and fine-tunes. On 8,202 chart and infographic images it lifted the 1B model's nDCG@10 from 37.34 to 44.12 (18% relative) and the 8B model's from 41.44 to 42.75 (3%), measured on 1,719 held-out synthetic queries, not real user questions.

What partners say, and what is missing

The post quotes partner results: Vectorize reports 93.8% recall@10 on the LoCoMo memory benchmark, Turbopuffer reports about 5% more relevant results at recall@20 on an enterprise PDF benchmark, and Perplexity says the reranker scored highest on its private 72-task benchmark. These are partner statements published by NVIDIA, and Perplexity's benchmark is not public.

The post does not give a licence, a per-query dollar cost, how the reranker's added latency affects response time, or results on documents in your domain. The 14% token saving also depends on how often your agent re-searches today, which NVIDIA cannot know.