On November 13, Deepnote released the Agent Initiative Benchmark: 12 frontier models were placed in the Slack of a simulated company to test whether an agent should proactively step in across 50 scenarios.
Results (according to the official report):
1. GPT-6 Astra took first place with 8.97 points, followed by GPT-6.1 Sol with 8.65 and Claude Fable 5.2 with 8.17.
2. In the 10 scenarios where it should have stayed quiet, even the top model managed to hold back only 8 times.
3. Cost per run: Sol, $0.112; Fable 5.2, $1.87. The three most expensive models ranked 3rd to 5th.
For trading agents, reckless action often costs more than doing nothing.
Which worries you more: an agent slacking off or acting recklessly?
#AIAgent
Results (according to the official report):
1. GPT-6 Astra took first place with 8.97 points, followed by GPT-6.1 Sol with 8.65 and Claude Fable 5.2 with 8.17.
2. In the 10 scenarios where it should have stayed quiet, even the top model managed to hold back only 8 times.
3. Cost per run: Sol, $0.112; Fable 5.2, $1.87. The three most expensive models ranked 3rd to 5th.
For trading agents, reckless action often costs more than doing nothing.
Which worries you more: an agent slacking off or acting recklessly?
#AIAgent