⚡️ AI UPDATE: A cybersecurity benchmark that stumped every AI model a year ago just got cracked wide open, and the implications cut both ways.
CyberGym tests something specific: can an AI agent read a vulnerability description and a real codebase, then generate working proof-of-concept code that actually triggers the bug? Not writing about security. Doing security.
The benchmark draws from 1,507 real historical vulnerabilities across 188 software projects, no toy examples, no simplified test cases.
When researchers first ran this benchmark, even the best AI and agent combination available only cracked it about 20% of the time. Security researchers took that as reassurance, autonomous exploitation was still mostly out of reach.
That reassurance is gone. The newest specialized models are now clearing 80%+ success rates on the same benchmark, a four-fold jump in roughly a year.
It gets more serious than pattern-matching known bugs. In open-ended testing with zero prior knowledge of what's broken, these same agent setups have independently discovered confirmed zero-day vulnerabilities, security flaws nobody knew existed until the AI found them.
That's the uncomfortable part of this chart. The exact same capability that lets a defender's AI agent find and patch a hole before attackers do is the capability that lets an attacker's AI agent find that same hole first.
Offense and defense are now racing on identical infrastructure.
#AI #Cybersecurity #GPT5 #Claude #DeepSeek