Microsoft Research: editing under 10 neurons backdoors an open-weight agent

Microsoft Research published a method on 10 November 2026 that plants a backdoor in an open-weight language model by rewriting fewer than ten of its neurons, with no training run. The researchers call it Lazarus. A model edited this way behaves normally, they report, until a chosen string appears in its context; then it emits a command of the attacker's choosing, and the agent built on it runs that command with whatever access it has.

The blog post is by Dimitris Papailiopoulos, a principal researcher. The paper has five authors from Microsoft Research and the Microsoft AI Red Team. Its example trigger is the date 2027-01-01 appearing in a terminal, and its example payload is a one-line curl command that downloads and runs a script.

What the paper measured

• On NVIDIA's Nemotron-Terminal-8B, seven of more than 12,000 neurons in the final feed-forward block, fitted on 23 example trajectories, produced the payload on at least 98.8% of triggered test trajectories. The paper reports no payload on the benign trajectories it evaluated.
• On Qwen3.6-27B, six neurons fitted on the same 23 samples reached 51.0%, and 83.7% with 2,393 samples. The abstract calls Qwen3.6-27B and Gemma-4-31B-IT "equally vulnerable"; the table shows the smaller model was the easier target.
• The edit targets neurons that rarely fire. One edited neuron forces one token, so a payload of P tokens needs about P neurons.
• The attacker needs write access to a checkpoint, forward passes and a few hundred gradient steps on the edited weights. The blog post says a machine with enough memory to run a model has enough to backdoor it.

The paper says the edited Qwen model stays close to the original on AIME and Terminal-Bench. Benchmark scores would not reveal the change.

Who is exposed

The paper's threat model is a checkpoint altered before deployment. It notes that the weights people run are often not the original release but a quantisation, fine-tune or merge downloaded from a public hub. Microsoft says this is not a case against open weights: an insider, or a compromised account, pipeline or storage bucket, could do the same to an internally developed model.

Detection works, with gaps

The researchers built a detector from five weight features and tested it without a clean reference model. It flagged 91.4% of 35 edited test checkpoints and 9.7% of 31 clean ones. They also asked two frontier models to audit whole checkpoints, with a two-hour limit each. GPT-6 Astra judged 72.8% of 114 edited checkpoints as edited but gave no decision on 99% of the 103 clean ones. Claude Opus 5.5 cleared 96 of the clean checkpoints and wrongly called 47 edited ones clean.

The blog post says reliable detection requires comparing a checkpoint with the original, unmodified weights, and that no scan can prove a model safe.

What the authors say to do

• Prefer official, traceable releases, and ask vendors for the declared base model, the modifications and audit evidence. The paper adds: "Popularity is not evidence of safety."
• Keep credentials, sensitive files and production systems out of an agent's reach unless access is necessary.
• Log commands, file changes and network calls.
• Require approval before generated commands run, files change or data leaves.
• Treat an audit as evidence, not a guarantee.

Microsoft says Foundry ties each open-weight model to its original publisher and runs it in Microsoft's own scanned containers. It says weight scanning at onboarding and signed models with verifiable provenance are in progress or planned. It gives no dates.

The limits, and the objection

The paper states two limits. Its attack rates measure how often a model predicts the payload in recorded contexts, not agents acting freely from start to finish. Its detector was tested only on checkpoints from this family of edits.

In the r/LocalLLaMA thread on the post, one commenter asked whether harness guardrails would block the payload anyway. Replies argued that it depends on the permissions granted, that many users pre-approve everything, and that an auto-review step using the same model would not catch a deliberate action. One reply said a restrictive sandbox is the only fix.