Amazon: VLM plus OCR plus CatBoost scores listings at 94% accuracy

Amazon says it grades the quality of A+ Content, the rich-media section of its product pages, by combining a vision-language model, text-readability scores and a small classifier. The method is described in a post published on 5 November 2026 on the AWS Machine Learning Blog by five Amazon and AWS authors. Its headline result: 0.94 accuracy and a 0.95 F1 score, against 0.80 and 0.84 for a simple average of the model's scores.

The approach

The system renders a whole page as one snapshot and asks a VLM to score it against a rubric, rather than grading each image or text block alone. The post's reasoning: a page can pass every per-asset check and still contradict itself, for example bullets advertising a 64-oz pitcher beside a photo of a single-serve blender.

The VLM returns a 0-10 score, a confidence value and a rationale for four dimensions: Visual Storytelling, Feature Presentation, A+ Layout and Conversion Optimization. An OCR model adds four layout signals: Text Density, Character Density, Spacing and Overall Readability. The eight numbers feed a CatBoost classifier, a gradient-boosted tree model, trained on expert labels.

Why the model is frozen

The authors say fine-tuning the VLM would need GPU training and, with only a few thousand labels, risks overfitting. They keep the VLM and rubric prompt fixed and retrain only the classifier, which they describe as a short CPU-only job when annotations or criteria change. They also say this keeps the method independent of the particular VLM.

The reported numbers

• Labels: one ecommerce marketing expert and four professional annotators rated 5,478 pages across 10 categories as high or low quality; 49.6% were high quality. Each annotator's Cohen's kappa against the majority label was 0.845 to 0.915.
• Holdout (20%, about 1,100 pages): averaged VLM scores with a threshold 0.80 accuracy and 0.84 F1; OCR-only classifier 0.82 and 0.83; VLM-only classifier 0.89 and 0.90; fused model 0.94 and 0.95.
• Backbone swap on a 499-project single-category set: with a lighter VLM the baseline scored 0.74 accuracy and the fusion model 0.90; with a more capable VLM, 0.79 against 0.94.
• Production: a worldwide A/B test that changed the visibility of low-quality A+ Content produced what the post calls a statistically meaningful improvement in Ordered Product Sales. No size is given.

Explanations that sellers can act on

For pages judged low quality, the system uses SHAP values to rank which of the eight features pulled the score down, then passes the worst ones, the product category and best practices to an LLM that writes prioritised advice. Across the holdout, the most influential features were OCR Character Density (mean absolute SHAP 1.1), OCR Overall Readability (1.0) and VLM Visual Storytelling (1.0). The authors report sellers who revised their pages gained an average 1.4 points on the 10-point score.

What the post leaves out

It does not name the VLM or OCR model, or give cost per page, latency or throughput. The test set comes from the same 5,478 labelled pages and annotators as the training data, and the authors note the labels cannot represent every category, language or marketplace. The 1.4-point gain compares scores from the system itself, not sales, and no figure is given for the A/B test. Nothing is said about failures, false positives that hurt a seller's visibility, or an appeal route.

The post does cover drift. It warns of silent accuracy erosion as customer taste and new module types change, and describes monitoring of data, model and feature-attribution drift with automated retraining when thresholds are breached.