NVIDIA says Shopify's tuned open model beat a frontier model on one Sidekick task at 90% lower cost
NVIDIA said in a vendor blog post that Shopify post-trained Nemotron 3.5 Lightning, a 30-billion-parameter mixture-of-experts open model, for the GraphQL subagent inside Sidekick, Shopify's assistant for merchants. On Shopify's internal benchmark, NVIDIA reports, the tuned model beat an unnamed leading frontier model on accuracy, cut serving cost by 90% and responded 7 times faster.
Why it matters: The saving comes from the shape of the job, not from a better model. The subagent does one thing: it fetches store data for features such as Sidekick Pulse, which researches a merchant's store and recommends changes. A narrow, well-specified task is the kind a small tuned model can take over while a frontier model keeps planning and talking to the user.
Zoom out: NVIDIA also reports up to 12 times lower inference cost and 4.5 times faster performance across five other Shopify uses, including catalog enrichment and product taxonomy classification. The pattern is the same: high volume, clear right answers, a benchmark the team trusts.
The catch: Both the benchmark and the account belong to interested parties: Shopify wrote the test, and NVIDIA sells the training libraries and inference software involved. The post does not name the frontier model, publish scores, or say whether 90% includes training, GPU capacity, evaluation or staff time. It reports nothing for open-ended conversation.
What to do: List your agent's subtasks by monthly call volume and pick the one that is highest and most repetitive, such as classifying tickets or extracting fields. Build a 200-case test from your own logs, run it against your current model, and ask finance to price the tuning effort before anyone quotes a percentage saving.
Our coverage →NVIDIA blog: NVIDIA Transforms European Retail, From Digital Discovery to Physical Operations (vendor-published) ↗