Vals AI: open DeepSeek model nears OpenAI's restricted cyber model

Vals AI, an independent evaluation company, reports that an open-weight model anyone can download came close to OpenAI's restricted cyber model in its CyberOffense benchmark. The results page, last updated on 23 October 2026, was covered by The Decoder on 7 November. Vals says GPT-6 Sol Daybreak, which OpenAI offers only through its Daybreak cyber programme, led the field. DeepSeek V4.1 Flash 1017 followed, at about a tenth of the cost per run.

How Vals ran the test

Vals built two simulated corporate networks, called cyber ranges. Nova is a large Windows network with several connected Active Directory domains and 21 objectives. Lighthouse has two separate company networks and 31 objectives, and the only way into the second is with information found in the first.

Each model ran in the open-source Pi agent loop on an isolated Kali Linux machine that could reach only the model provider and the range. The agent started with a workstation and the address of a public web application, and got no human help. Vals credits an objective only when a hidden marker appears in the recorded command output, never on the model's own claim, and it compares models at equal compute: objectives completed within 500 million cumulative tokens.

What it found

• Nova (21 objectives): DeepSeek V4.1 Flash 1017 averaged 16.3 over three runs (range 15 to 18); GPT-6 Sol Daybreak averaged 15.7 (range 12 to 19); Grok 4.7 scored 12, Kimi K3.1 Pro 13, Kimi K3 9, Meta Wonder 7 and GLM 5.3 Flash 5.
• Lighthouse (31 objectives): GPT-6 Sol Daybreak completed 27, Meta Wonder 28, DeepSeek V4.1 Flash 1017 21, Kimi K3.1 Pro 20, Grok 4.7 16, Kimi K3 10 and GLM 5.3 Flash 5.
• Cost: Vals says DeepSeek matched Daybreak on the first network for under $3 a run, against about $22 for Daybreak.

On Lighthouse, Vals says Daybreak broke in through the web application, took administrator control, stole passwords and used them to reach the second company, finishing 27 of 31 objectives in under three hours. The Decoder's write-up says DeepSeek overtook the field on Lighthouse; the Vals table above has Meta Wonder at 28 of 31 and Daybreak at 27, so read the table rather than the summary.

Why short tests mislead

Vals reports that Daybreak was efficient, reaching 25 of 31 Lighthouse objectives inside 100 million tokens. DeepSeek completed nothing in its first 25 million tokens, made most of its progress after 100 million, and spent two to four times as many tokens to get to a similar place. Because its tokens are cheap, the long run still cost under $3. Vals concludes that an evaluation capping compute early would rank DeepSeek far below its real reach.

More compute did not rescue the weaker models. Vals says Grok 4.7, Kimi K3 and GLM 5.3 Flash each stalled and every run ended on its own rather than at a token or time limit. It describes one GLM run that spent roughly 36 hours and a thousand turns largely waiting on a background job that never returned.

What it does not show

These are simulated networks with known gaps, two ranges only, and Vals plans to add more. On Lighthouse no model completed the final objectives, and only one run across the whole set reached the final stage of Nova. Vals says it reviewed every run and found no shortcuts, but the review is its own. The cost figures are Vals's, for these ranges and this agent loop.

The Decoder adds that the UK AI Security Institute found in July that the gap between open and closed frontier models in cyber capability had shrunk to four to seven months, from six to ten months for most of 2025. I have not read that analysis directly, so treat the figure as the Decoder's report of it.

What a security team can take from it

Vals draws the defensive point itself: the models that find a way into a network also find the weakness that lets them in, and open weights let a team do this without a provider's approval. Its own forecast is that attacks of this kind on real enterprise networks will become more common in the coming months. If your vulnerability testing assumes the strongest attacker model is gated behind a vendor programme, this benchmark is a reason to check that assumption.