Shopify runs search and recommendations on one 2.6B open-weight model

Shopify says a single family of generative models, fine-tuned from Liquid AI's open-weight LFM2-2.6B, now produces product candidates for both search and recommendations in production. The claim is in a technical report the company posted to arXiv, dated 5 November 2026, which calls the system SID-LLM.

The report is useful beyond commerce for one reason: it shows a team picking a small model on a latency budget, publishing the larger models it turned down, and listing the serving work that made the choice hold.

What Shopify reports

• Recommendations. In an A/B test on the Shop home feed, adding the model raised attributed orders by 7.95%, converting users by 8.60% and conversion rate by 9.85% on the affected surface. Shopify says all three are statistically significant.
• Search, offline. Shopify's relevance score for the top ten results (OneDCG@10, its variant of NDCG) is 0.327 for the full search stack with the model and 0.306 without it. For personalised search the figures are 0.357 and 0.330.
• On its own. The model alone scores 0.285 on search, below the existing stack. It runs as one candidate generator among several, not as a replacement. In the recommendation pool it finds 2.65 percentage points of purchases no other generator finds; the next best adds 1.64.

Why a 2.6B model

Shopify compared four open-weight backbones. LFM2-2.6B had a p95 latency of 30.35 ms. Qwen3-1.7B took 73.29 ms, Qwen3-4B 105.00 ms and Qwen3-14B 142.87 ms. The Qwen models were better on quality, by 1.0%, 3.5% and 4.9% averaged across three search and recommendation metrics.

Shopify deployed the fastest one. The report's stated constraint is that a search request must return valid results within tens of milliseconds on a fraction of one GPU. It also notes that training recipes differed across backbones, so the comparison is not perfectly like for like.

The engineering that made it fit

• Products as three tokens. The catalogue holds billions of products, too many to be an output vocabulary. Each product embedding becomes a three-token semantic ID drawn from three codebooks of 8,192 entries. A larger first codebook scored higher but pushed p95 latency from 40.5 ms to 50.0 ms at 100 requests per second, so Shopify kept the smaller one.
• Constrained decoding on the GPU. Only some token combinations map to real, eligible products. A sparse index held on the GPU, built on the published STATIC method, cut that check from 18 to 33 ms per request to about 1 ms.
• One pass, not three, for search. Predicting all three tokens at once scored worse than token-by-token generation (0.280 against 0.308). Adding a 350-million-parameter ranker over 250 candidates took it to 0.337, and p95 latency at 50 requests per second was 29.2 ms against 36.5 ms.
• Tail latency under load. At 100 requests per second the token-by-token model's p99 reached 568.7 ms; the one-pass model with its ranker reached 231.4 ms.
• IDs are clusters. 78.9% of predicted IDs are shared by more than one product. Scoring products by ID confidence and popularity raised recommendation Recall@50 by 15.8% over taking one product per ID.

One operational detail for anyone with a changing catalogue: Shopify says a sixth of product records change daily, the index is rebuilt each day, and eligibility is only as current as the snapshot a server loaded at startup.

What the report leaves out

There is no cost figure: not GPUs in service, not cost per thousand requests, not training spend. The home feed test comes without its duration or traffic share. For search, the report says online A/B tests "supported deploying" the model but gives no online numbers, only offline scores. And one of the eleven authors is from Liquid AI, which makes the chosen base model, so the backbone comparison is not from a neutral party.