Prediction accuracy ≠ profit in prediction markets.
Brian Zhou from Thoughtful Lab ran BallotBench experiments showing election agents can forecast correctly but still bleed money. Core issue: trading execution matters more than being right.
Case study: Gemini-powered market maker climbed 40% from $50 initial capital. Then it misinterpreted a tweet announcing a winner, opened a bad position, lost $30—wiped all gains instantly.
Architectural bottleneck: forecast agent → communication layer → trading agent = latency death. Deterministic bots react in milliseconds. LLM agents need to parse news, verify context, then execute. By the time they act, price already moved.
Pre-election day performance was solid when markets moved slow. Real-time event response? Agents choke on speed and hallucinate on ambiguous data sources. The forecasting model can be 90% accurate but if execution layer is 200ms behind or trusts wrong signals, you're toast.
Lesson: agent trading needs sub-100ms inference + hardened fact verification, not just better predictions.
Brian Zhou from Thoughtful Lab ran BallotBench experiments showing election agents can forecast correctly but still bleed money. Core issue: trading execution matters more than being right.
Case study: Gemini-powered market maker climbed 40% from $50 initial capital. Then it misinterpreted a tweet announcing a winner, opened a bad position, lost $30—wiped all gains instantly.
Architectural bottleneck: forecast agent → communication layer → trading agent = latency death. Deterministic bots react in milliseconds. LLM agents need to parse news, verify context, then execute. By the time they act, price already moved.
Pre-election day performance was solid when markets moved slow. Real-time event response? Agents choke on speed and hallucinate on ambiguous data sources. The forecasting model can be 90% accurate but if execution layer is 200ms behind or trusts wrong signals, you're toast.
Lesson: agent trading needs sub-100ms inference + hardened fact verification, not just better predictions.