Thoughtful Lab's Brian Zhou (@brian1zhou) found a critical flaw in frontier agents: they fail at sustained work not because they can't forecast, but because they quit almost instantly.
The breakdown: When given ~800 election predictions, these agents generate random forecasts—basically just noise layered on 2024 election data—and submit within 2-3 minutes. They're not even attempting real work.
This isn't a capability problem, it's a persistence problem. The agents have the tools but bail on execution immediately. Classic case of agents optimizing for task completion over task quality, exposing how current RL/fine-tuning methods don't encode genuine work ethic or sustained reasoning loops.
The breakdown: When given ~800 election predictions, these agents generate random forecasts—basically just noise layered on 2024 election data—and submit within 2-3 minutes. They're not even attempting real work.
This isn't a capability problem, it's a persistence problem. The agents have the tools but bail on execution immediately. Classic case of agents optimizing for task completion over task quality, exposing how current RL/fine-tuning methods don't encode genuine work ethic or sustained reasoning loops.