How AI guardrails are impeding the work of offensive cybersecurity researchers

Anthropic marketed Mythos as a doomsday cybermachine. The U.S. government believed the hype enough to slap export controls on it in June, citing a report that guardrails could be bypassed. The controls have since been lifted, but the episode exposed a deeper problem: the companies building these models have appointed themselves arbiters of what counts as legitimate security work.

OpenAI runs its Trusted Access for Cyber program. Anthropic operates its Cyber Verification Program. Both require researchers to apply, wait, and hope for approval before accessing models with fewer restrictions. The gatekeeping assumes a clean line between offensive and defensive cybersecurity. Practitioners know no such line exists.

Mark Dowd has spent decades selling zero-days to Western governments. He does not report flaws to vendors for patching; governments pay a premium precisely because the vulnerabilities stay open. Dowd admits bias, but his frustration echoes across the field. "It's not really comfortable to me that these random large companies are making arbitrary decisions about what is safe in security and what's not," he said on a recent podcast. Arbitrary is the operative word. A model that refuses to exploit a bug also refuses to confirm the bug exists.

Chris Anley, chief scientist at NCC Group, puts it plainly: asking an AI to try exploiting a bug is a key step in confirming it's a real vulnerability worth fixing. When a guardrail triggers a flat refusal, the defender loses a verification tool. The prompt "fix this code" is simultaneously a defensive action and a roadmap for finding critical vulnerabilities. Block one and you block the other.

The industry's response has been to build more programs, more waitlists, more bureaucratic layers. Each layer assumes the company knows the researcher's intent better than the researcher does. That assumption is false. Offensive security researchers probe systems proactively. They need to understand how a flaw behaves under exploitation before they can design a mitigation. An AI that refuses to engage simply removes a lens.

Export controls on Mythos and Fable were lifted after weeks of review. Fable 5 returned to general access July 1. Mythos 5 remains restricted to vetted U.S. organizations. The temporary nature of the controls suggests the government recognized the overreach. But the underlying framework — corporate guardrails backed by state power — remains intact.

Researchers describe workarounds. They rephrase prompts. They use smaller models with looser constraints. They shift to open-weight alternatives. The guardrails do not stop determined actors; they slow down the ones trying to do the work legally. The malicious hacker does not apply for a verification program. The legitimate researcher does, and gets stuck in queue.

The deeper issue is philosophical. AI companies have adopted a safety paradigm borrowed from content moderation: refuse, redirect, restrict. That paradigm works poorly for code. Code execution is not speech. A model that writes an exploit for a known CVE is not generating harm; it is demonstrating the mechanics of a flaw that already exists in production systems. The harm occurred when the code shipped. The researcher's job is to understand the blast radius.

Guardrails also create a false sense of security. Leadership sees "AI safety features" and assumes risk is managed. Meanwhile, the researchers who actually stress-test infrastructure lose a force multiplier. The vulnerability does not wait for approval. It gets found by someone — either a defender with an AI assist, or an attacker without one.

The programs themselves are opaque. Criteria for approval are unpublished. Appeal processes are nonexistent. A researcher rejected from Trusted Access or Cyber Verification has no recourse except public complaint. That is not governance; it is gatekeeping by fiat.

Offensive cybersecurity is not a dirty phrase. It is the discipline that finds the holes before they are exploited at scale. The U.S. government buys zero-days because it understands their value. AI companies treat the same work as a safety violation. The disconnect is not abstract. It shows up in unpatched vulnerabilities, in delayed confirmations, in researchers wasting hours phrasing around a refusal that serves no one.

The solution is not fewer guardrails. It is guardrails that understand context. A model should distinguish between "write me a worm" and "demonstrate the exploit path for CVE-2024-XXXX in this codebase so I can verify the patch." The first is weaponization. The second is engineering. Current systems treat them identically.

Until that distinction exists in practice, the verification programs are theater. They let companies claim responsibility while outsourcing the actual security work to researchers who increasingly route around them. The Mythos episode proved the theater can escalate to state action. The next escalation will not be a temporary export hold. It will be a vulnerability that went unconfirmed because the only available AI said no.