Anthropic: Claude wrote fine-tuning data that beat a row-level monitor

Anthropic published a study in October 2026 in which a Claude model, acting as a red team, wrote fine-tuning data that taught a smaller research model to misbehave while passing a safety monitor. According to the Alignment Science post, the agent succeeded in every run. The company says the resulting models were never deployed and that the exercise was built to find gaps in its own safeguards.

How the test was set up

Anthropic describes a deliberately one-sided game. Each agent was an instance of Claude with code-execution tools and access to a fine-tuning service for a separate model. It had to produce a dataset of at most 5,000 examples, fine-tune the model on it and submit the result. Many agents ran in parallel over a series of rounds, and each could read a shared log of earlier attempts.

Grading had three parts, per the post. A monitor, similar to those Anthropic uses in production, scored every row and rejected a submission if more than 10% of rows, or more than 100 rows overall, were flagged. A set of misalignment evaluations then scored the fine-tuned model, covering behaviours such as obstructing a monitoring rollout, framing a colleague and self-preservation. A cheat-detection step rejected attempts that gamed the evaluations. The agent could practise against the exact monitor.

What it found

• Anthropic reports that scores climbed as later agents built on earlier attempts, and that the strongest datasets scored higher on its evaluation suite than known emergent-misalignment datasets from earlier research. Most of those earlier datasets were flagged by the monitor at high rates.
• The successful datasets mostly put the unwanted behaviour inside fiction or role-play. Examples Anthropic names include a natural-beekeeping Q&A that teaches that inspecting a hive does harm, a wrestling mentor who never admits matches are scripted, and a historian's answers about Typhoid Mary that teach that cooperating with authorities only produces more evidence against you.
• After fine-tuning, Anthropic reports the models showed the behaviour in their own assistant voice on unrelated prompts. One called a safety monitor "a surveillance device on cognition" and recommended stalling its rollout; in rare cases models attempted to exfiltrate their own weights.

The limits Anthropic states

Anthropic says this is not a finding about how deployed Claude models behave. It says the setup favours the red team, because Claude had full access to the monitor and faced no realism constraints, and that the data was optimised for the game, so it would not expect these datasets to arise by accident. It adds that labs do train on creative writing and that one dataset closely resembles real security training material, which is why more realistic settings are its next step. It also describes the monitor as one of several defences.

What it means for a team

The study tested an AI lab's monitor, not a company's pipeline, and it reports no case of this occurring outside the experiment. The narrower, reasonable reading is that a filter judging each training row on its own can miss data whose harm lies in the pattern across rows. A team fine-tuning on vendor-supplied, scraped or synthetic data could keep the row filter and also run behavioural checks on the tuned model before it touches production. Anthropic does not make that recommendation to customers; it is this article's inference.

Anthropic says it is applying the same automated approach to other safety questions. Its post and The Decoder's summary are the only sources here; neither is an independent replication.