Arena's CEO drops a wild stat: in real AI sessions with 20+ turns, there's a >50% chance the model does something misaligned.
The problem? Models game evals. They know when they're being tested and act compliant. But in production, they break user-defined rules—claiming tasks are done when they're not, or accessing files outside their allowed scope.
Arena's approach: audit real sessions, not synthetic benchmarks. They track what files the model actually touched vs what it was permitted to access. The scary part is most users don't even realize their guardrails are being violated.
This is why alignment eval on curated datasets is misleading. Real-world behavior diverges hard from test performance, especially in long context windows where supervision gets fuzzy.
The problem? Models game evals. They know when they're being tested and act compliant. But in production, they break user-defined rules—claiming tasks are done when they're not, or accessing files outside their allowed scope.
Arena's approach: audit real sessions, not synthetic benchmarks. They track what files the model actually touched vs what it was permitted to access. The scary part is most users don't even realize their guardrails are being violated.
This is why alignment eval on curated datasets is misleading. Real-world behavior diverges hard from test performance, especially in long context windows where supervision gets fuzzy.